Camelia, the Perl 6 bug

IRC log for #gluster-dev, 2013-08-16

| Channels | #gluster-dev index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:34 avati joined #gluster-dev
00:39 yinyin joined #gluster-dev
01:57 kanagaraj_ joined #gluster-dev
02:14 bharata joined #gluster-dev
02:34 bala joined #gluster-dev
02:38 __Bryan__ joined #gluster-dev
02:43 awheele__ joined #gluster-dev
02:46 awheele__ joined #gluster-dev
02:54 shubhendu joined #gluster-dev
03:34 kanagaraj_ joined #gluster-dev
03:34 yinyin joined #gluster-dev
03:37 kanagaraj joined #gluster-dev
04:01 mohankumar joined #gluster-dev
04:16 ppai joined #gluster-dev
04:22 itisravi joined #gluster-dev
04:39 yinyin joined #gluster-dev
04:52 aravindavk joined #gluster-dev
04:54 itisravi joined #gluster-dev
05:29 lalatenduM joined #gluster-dev
05:30 lala_ joined #gluster-dev
05:45 hagarth joined #gluster-dev
05:53 ababu joined #gluster-dev
06:01 bulde joined #gluster-dev
06:02 bharata-rao Has anyone ever tried "-" (equivalent to stderr) on glfs_set_logging() and found it to be working ?
06:04 bharata-rao avati, bala ^
06:09 bala bharata-rao: not sure about that
06:10 bala bharata-rao: i believe some options need to be working in different way ie non-daemon mode should throw logs to stderr
06:10 bala bharata-rao: and '-' is not needed
06:11 bharata-rao bala, QEMU uses - to imply logging to stderr
06:11 bharata-rao bala, client side logs
06:11 bharata-rao bala, I have never seen any error logs coming to stderr with qemu
06:11 bharata-rao bala, I have a fix, but not sure if that is the right way to fix, let me post the patch anyway
06:12 bala bharata-rao: ok
06:14 bala bharata-rao: however dumping logs to stderr for daemon or background tasks is a mess.  they should go into log files
06:14 bharata-rao bala, but it is ok for client side logs right ?
06:16 bala bharata-rao: u mean gluster mounts?
06:16 bharata-rao bala, no qemu using libgfapi
06:18 bala bharata-rao: it depends on the consumer of the api.  if consumer app is a background service, then i dont think stderr is right choice
06:20 bharata-rao bala, of course, but we are kind of struggling with unintelligible failure messages with qemu-glusterfs, so some meaningful clientside error will help
06:20 bharata-rao bala, http://review.gluster.org/#/c/5637/
06:21 rgustafs joined #gluster-dev
06:24 yinyin joined #gluster-dev
06:28 vshankar joined #gluster-dev
06:54 aravindavk joined #gluster-dev
07:06 bala bharata-rao: does the fix work for u?
07:06 bala bharata-rao: in gf_log(), if log_file is empty, then it goes to stderr
07:07 bala bharata-rao: i think ur fix is not needed
07:17 bharata-rao bala, the fix works for me with qemu-glusterfs. http://dpaste.com/1345173/
07:17 bharata-rao bala, qemu uses glfs_set_logging(glfs, "-", 4) with the intention of getting the logs on stderr
07:28 foster joined #gluster-dev
07:31 bharata-rao bala, I don't see what you mention above in _gf_log()
07:32 asias joined #gluster-dev
07:50 mohankumar joined #gluster-dev
08:12 ababu joined #gluster-dev
08:33 deepakcs joined #gluster-dev
08:58 bharata-rao bala, ping
09:03 bala bharata-rao: yes
09:13 kanagaraj joined #gluster-dev
09:14 kanagaraj joined #gluster-dev
09:16 bharata-rao bala, I looked at _gl_log, but don't find what you said before, can you point me to the file:line no ?
09:17 bharata-rao bala, also not gf_log choosing stderr is one thing, but getting stderr logging working with "-" is another thing
09:17 bharata-rao bala, s/also not/also ^
09:20 bala bharata-rao: logging.c:849
09:22 bharata-rao bala, ok, so if we don't specify logfile, then stderr is chosen, now what if logfile is "-" ?
09:23 bala bharata-rao: that means choose stderr
09:23 bharata-rao bala, I think that case where "-" is used explicitly is still broken. Agree ?
09:24 bala bharata-rao: this is done by leaving ctx->log.logfile is null
09:24 bala bharata-rao: yes
09:24 bala bharata-rao: i think its broken somewhere
09:25 bharata-rao bala, yeah, but somehow populating ctx->log.logfile is making to work
09:25 bala bharata-rao: without ur patch, what is the behavior?
09:26 bala bharata-rao: log goes to file than stderr?
09:26 bharata-rao bala, check the pastebin I gave u earlier
09:26 bharata-rao bala, no logs w/o my patch
09:26 yinyin joined #gluster-dev
09:28 bala bharata-rao: some prob with libgfapi
09:28 asias joined #gluster-dev
09:30 bharata-rao bala, when is ctx->log.logrotate set ?
09:32 bala when it gets sigup
09:33 bharata-rao bala, and who sends sigup ? to whom ?
09:34 bala when u want to rotate log, the file is copied and sigup is sent to gluster process
09:35 bharata-rao bala, ok
09:41 bharata-rao bala, Debugging shows that ctx->log.logfile isn't set to stderr at logging.c:849, but instead set to something else
09:44 bharata-rao bala, so the problem is gf_log_init didn't set ctx->log.logfile at all and hence the log never came to stderr
09:44 bharata-rao bala, and this also explains why my fix is working after setting ctx->log.logfile to stderr
09:49 bharata-rao bala, also "-" is not handled correctly when logrotate is set, logging.c:788 ends up open'ing "-" which won't work!
09:56 bala bharata-rao: actually log.logfile is set to be null
09:57 itisravi joined #gluster-dev
09:58 bala bharata-rao: if something is set, then it wont go to stderr as per 849 line
09:58 bala bharata-rao: however surely there is bug which is not handling '-
09:58 bala '-' properly
10:03 bharata-rao bala, at least I don't see log.logfile being explicitly set to null in logging.c, it's initialized somewhere else ?
10:07 bala bharata-rao: ctx initialization should take of
10:08 bala bharata-rao: anyway this is also wrong, gf_log_globals_init() should do explicitly than excepting ctx is init'ed properly
10:09 bharata-rao bala, right
10:14 bharata-rao bala, so there are two issues then: 1)log.logfile not initialized correctly 2) "_" isn't being handled correctly
10:18 bala bharata-rao: as per current code, if (1) is fixed (2) will work fine
10:21 bharata-rao bala, don't think so
10:21 bharata-rao bala, in case of libgfapi, glfs_new will call glfs_set_logging(glfs, "/dev/null", ..)
10:22 bharata-rao bala, next glfs_set_logging(glfs, "-", ..)will be called explicitly by the client (qemu)
10:22 bharata-rao bala, so what is currently happening is that since you don't set log.logfile for "_", it is using the old setting (/dev/null) and hence we don't see any logs
10:23 bharata-rao bala, so I think my fix is still needed to address 2)
10:34 bala api/src/glfs.c:407:     ret = glusterfs_globals_init (ctx);
10:34 bala glusterfs_globals_init() calls gf_log_globals_init()
10:36 bharata-rao bala, yes, check api/src/glfs.c:423
10:39 bala bharata-rao: yes, it sets /dev/null forcefully
10:39 bala bharata-rao: no idea why
10:39 bharata-rao bala, that's how it should be
10:40 bharata-rao bala, for those clients who don't do glfs_set_logging explicitly, it should be set to /dev/null by default
10:40 bharata-rao bala, so the problem 1) doesn't exist
10:40 bharata-rao bala, log.logfile is initialized correctly to null before it gets set to /dev/null
10:40 bharata-rao bala, the problem is only with "-"
10:41 bharata-rao bala, where log.logfile isn't set and that's what my fix does
10:57 kanagaraj joined #gluster-dev
12:18 hagarth joined #gluster-dev
12:44 bala1 joined #gluster-dev
12:53 awheeler joined #gluster-dev
12:54 awheeler joined #gluster-dev
13:27 hagarth joined #gluster-dev
14:11 wushudoin joined #gluster-dev
14:17 lpabon joined #gluster-dev
14:29 deepakcs joined #gluster-dev
14:32 ababu joined #gluster-dev
14:35 __Bryan__ joined #gluster-dev
16:04 jclift_ joined #gluster-dev
16:04 jbrooks joined #gluster-dev
16:55 ababu joined #gluster-dev
16:55 Technicool joined #gluster-dev
17:13 hagarth joined #gluster-dev
17:16 mohankumar hagarth: in bd_lookup why i get <gfid:gfid> as loc->path ?
17:17 mohankumar instead of regular path?
18:56 jclift_ @seen jayunit100
18:56 glusterbot jclift_: I have not seen jayunit100.
18:57 jclift_ @seen jvyas
18:57 glusterbot jclift_: jvyas was last seen in #gluster-dev 30 weeks, 6 days, 20 hours, 1 minute, and 28 seconds ago: <jvyas> i love thrift.  is it a valid replacement for SWIG?
18:59 avati bfoster: ping?
19:03 foster avati: pong
19:05 avati foster: hey
19:05 foster o/
19:05 avati foster: some details about the read memcpy
19:05 avati the zero-copy patch was really "misnamed"
19:06 avati the intention was to give zero copy in libgfapi in the read path.. but the patch was actually just allowing for a "read straight into my buffer" semantics as a new FOP
19:06 foster right, makes sense
19:07 foster I got a little confused by the zero copy terminology and just looking at the fop
19:07 avati that fop should also help eliminate the memcpy in the qcow2 xlator, if we use the new fop rather than existing readv
19:07 foster ok
19:07 avati so i'll work on making some final changes to the patch and re-post it
19:08 avati once that is merged, we can start the v2 work for external snapshot support
19:08 foster that sounds good...
19:08 avati we'll have to figure out some internal design details - where we will keep the intermediate delta files, whether we hide them from namespace, etc.
19:09 foster when that's up and posted I'll do a more detailed review and some testing to help with the merge
19:09 foster right, so external snapshots means the qcow images have path references to separate files, right?
19:10 avati yes.. I was thinking we will internally use gfid references
19:11 foster does qemu control the snap filenames (i.e., so we'd have to map them)?
19:11 avati while creating the qcow2 file with the required "backing file", we ahve to provide the backing file path as input
19:12 foster ok
19:12 avati since the caller has to provide the backing file as input, we can as well give it in the gluster://gfid URI format and qemu will figure out and call our driver from inside
19:13 avati and in the bottom open handler (bdrv-xlator.c) we decode the gfid, lookup inode by gfid, create anonymous fd, and use it - is my thinking
19:13 foster sounds reasonable
19:14 avati i may be missing something too.. (e.g, not sure if we need to maintain relationship b/w the two inodes in the gluster layer etc.)
19:15 foster yeah, I'll have to play around a bit with the native tools to better understand what's going on
19:16 avati another (possible) puzzle - qcow2 is not "clustered" code.. so we might have to prevent a second open from another client of the same file while first is still opened
19:16 avati if both modify, it can result in corruption
19:17 foster hmm, do you mean by "not clustered," it could be doing caching and things?
19:18 avati yes..
19:18 avati it would be cachign the allocation tables in the client side
19:19 foster heh, ok
19:19 avati multi client access wouldn't be a problem if xlator is loaded on server side
19:19 avati but then, i'm not sure how important multi client access is
19:20 avati so that's still an open question.. some openstack expert can probably answer that right away
19:20 avati (if the use case involves same file getting opened from multiple clients)
19:20 avati *same block image file
19:21 foster in the external snap case, are the new files the snap exception stores?
19:21 foster i.e., writes from a vm to a snap go to the new file
19:21 avati yes
19:21 foster reads come from the backing file
19:21 avati backing fiels are read-only
19:21 foster ok, that answers that then
19:21 foster otherwise yeah, multi-access seems unlikely
19:22 avati two snaps having common backing file should be safe, since qemu makes the origin read-only.. multi client accessing the same snapshot is the question
19:23 foster unless there's a virtual distributed block device mechanism (a la multi-port storage hardware?)
19:23 foster seems kind of out there
19:24 foster but I know nothing about openstack, so maybe lpabon knows :)
19:24 avati what's a multi-port storage hardware :-?
19:24 foster not sure if that's the right term
19:25 foster like an emc array that can be attached to multiple hosts
19:25 foster then run with a parallel fs to the same block device
19:25 avati it may be.. i have zilch knowledge of even terminology of enterprise storage
19:25 avati right.. you will need a clustered filesystem to use such a shared block device
19:25 avati with DLM and all
19:25 foster yeah, so if there's a virtual one of those, perhaps that's the use case for multi-access
19:26 avati yep.. if someone decides to run gfs2 or ocfs in their openstack guests from a shared cinder block device.. that's the use case
19:26 avati *on a shared cinder block device
19:27 avati lpabon not here?
19:27 foster yeah, i guess the infrastructure isn't really even necessary
19:27 foster just export the bdev to multiple vms
19:27 foster don't see him around
19:28 avati i dont know if openstack allows attaching same bd to multiple vms.. for e.g in amazon web services, you cannot
19:28 foster interesting
19:28 avati i hope they don't allow.. one less problem for our use case :p
19:28 foster :)
19:29 foster fwiw, the nice thing about a v1 release is that integration can start happening early
19:29 foster and we can shake out any big issues or weird requirements early too
19:30 avati yes.. we can at least get started with the integration.. not sure how crucial external snapshot is (to even have a basic integration)
19:30 avati a few more things which might show up - where is the command originating (in openstack) to issue a snapshot creation
19:31 avati since snapshot creation is in the setxattr path, the command will have to do a setxattr() from the same mount point since we are now loaded in the client side
19:31 avati i guess we can only know such answers by getting hands dirty w/ openstack
19:32 foster yep
19:33 avati worst case - load it on the server side and deal with ugly self-healing and rebalancing quirks :p
19:33 foster hehe
22:42 awheele__ joined #gluster-dev
23:24 awheeler joined #gluster-dev

| Channels | #gluster-dev index | Today | | Search | Google Search | Plain-Text | summary