Camelia, the Perl 6 bug

IRC log for #gluster-dev, 2013-06-20

| Channels | #gluster-dev index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:21 yinyin joined #gluster-dev
00:43 yinyin joined #gluster-dev
00:52 itisravi_ joined #gluster-dev
00:57 jbrooks joined #gluster-dev
00:59 * jclift looks at the PostgreSQL "Feature Matrix" and thinks we should have one for GlusterFS 3.4: http://www.postgresql.org/about/featurematrix/
01:22 bala1 joined #gluster-dev
01:22 lpabon joined #gluster-dev
01:33 bala1 joined #gluster-dev
01:33 itisravi_ joined #gluster-dev
01:40 bala1 joined #gluster-dev
01:51 hagarth jclift: +1
01:51 jclift hagarth: Hmmmm, should I open a BZ to do just that, so it's not forgotten?
01:52 jclift Actually, yeah, I'll going to make one now. :)
01:53 hagarth yeah, better to start writing one :).
01:54 jclift hagarth: Well, I'm creating the BZ now.  I'll try and be evil and assign it to Eco (who's not here to argue). :)
01:54 jclift hagarth: He may (likely) hand it back to me.  But, at least it won't be forgotten.
01:54 hagarth jclift: ok :)
02:00 jclift Damn, evilness fail.  BZ is created, but BZ refuses to let me give it to Eco.
02:00 jclift Amar is not going to be happy
02:00 jclift :(
02:01 hagarth Eco is not on bugzilla?
02:01 jclift Doesn't seem like it
02:22 jclift Hmmm, I really want to get the RDMA Test Day planning stuff written up in the next few hours, but I'm braindead atm.  I'll have to do it when I get up tomorrow. :(
02:24 hagarth jclift: I have that on my radar for today too. Hopefully something for you when you wake up.
02:24 jclift hagarth: Initial page being worked on is here: http://www.gluster.org/community/d​ocumentation/index.php/GlusterFest
02:25 jclift Got the top part of it organised, and compilation instruction page updated so it can work with release-3.4 branch.
02:25 hagarth jclift: neat, thanks!
02:26 jclift Now I just need to watch that video on the "new testing framework" so I understand it, then write instructions to somehow leverage that to do what we want.
02:26 * jclift suspects midnight friday is going to be a tall order
02:26 hagarth jclift: yeah
02:27 jclift Anyway, good night. :)
02:27 hagarth good night. :)
02:40 bulde joined #gluster-dev
02:40 jbrooks joined #gluster-dev
02:46 yinyin joined #gluster-dev
02:49 lalatenduM joined #gluster-dev
03:00 shubhendu joined #gluster-dev
03:00 bharata joined #gluster-dev
03:28 johnmark aha, so this is where evil is planned ;)
03:28 johnmark don't worry, bulde was only too happy to assign the bug to me :)
03:43 itisravi joined #gluster-dev
03:57 hagarth joined #gluster-dev
04:01 mohankumar joined #gluster-dev
04:57 bala joined #gluster-dev
04:57 lalatenduM joined #gluster-dev
04:58 rastar joined #gluster-dev
05:02 bala joined #gluster-dev
05:29 bulde joined #gluster-dev
05:32 hagarth joined #gluster-dev
05:40 yinyin joined #gluster-dev
05:45 aravindavk joined #gluster-dev
05:46 krishnan_p joined #gluster-dev
05:46 deepakcs joined #gluster-dev
05:58 bala joined #gluster-dev
06:02 raghu joined #gluster-dev
06:18 bala1 joined #gluster-dev
06:28 bala joined #gluster-dev
06:29 badone joined #gluster-dev
06:34 hagarth joined #gluster-dev
06:36 bala joined #gluster-dev
06:58 krishnan_p JoeJulian, there?
07:03 bala joined #gluster-dev
07:32 rgustafs joined #gluster-dev
08:04 yinyin joined #gluster-dev
08:23 bala joined #gluster-dev
08:58 yinyin joined #gluster-dev
09:26 blues-man joined #gluster-dev
09:28 bulde joined #gluster-dev
09:56 yinyin joined #gluster-dev
10:20 krishnan_p joined #gluster-dev
10:47 bulde joined #gluster-dev
10:48 shubhendu joined #gluster-dev
11:01 krishnan_p joined #gluster-dev
11:10 rastar1 joined #gluster-dev
11:17 bfoster joined #gluster-dev
11:22 kkeithley joined #gluster-dev
11:49 krishnan_p joined #gluster-dev
11:49 aravindavk joined #gluster-dev
12:06 bala joined #gluster-dev
12:22 hagarth joined #gluster-dev
12:29 shubhendu joined #gluster-dev
12:33 jbrooks joined #gluster-dev
12:50 krishnan_p joined #gluster-dev
12:57 lalatenduM joined #gluster-dev
13:01 yinyin joined #gluster-dev
13:09 ndevos hagarth: the glusterd-store patches from krish and me are waiting on http://review.gluster.org/4676, could you review+merge that maybe?
13:09 ndevos krish will be working on fixing the FILE* leaks, but he can not ./rfc.sh his patch as mine is not merged yet :-/
13:11 hagarth ndevos: I am sitting with krishnan_p right now ;)
13:11 ndevos hagarth: awesome!
13:16 deepakcs joined #gluster-dev
13:23 bala joined #gluster-dev
13:27 kkeithley we should close bz 918917, 3.4Alpha3 Tracker
13:40 edward1 joined #gluster-dev
13:45 kkeithley joined #gluster-dev
13:52 hagarth ndevos: done!
13:52 ndevos hagarth, krishnan_p: thanks!
14:00 krishnan_p ndevos, cool. Could you review http://review.gluster.org/#/c/5243/ for me?
14:00 ndevos krishnan_p: sure, is that the one you sent by email?
14:05 wushudoin joined #gluster-dev
14:12 lpabon joined #gluster-dev
14:12 johnmark people of the world... join hands... start a love train (a love train)
14:13 johnmark kkeithley: good point. alpha is old old news
14:13 johnmark kkeithley: what's the bug you need someone's attention on? and who's on the clock for it?
14:13 * johnmark looks up email
14:15 johnmark bug 912747
16:07 jclift joined #gluster-dev
17:09 wushudoin joined #gluster-dev
17:15 tg3 does anybody have eco's email?
17:15 tg3 http://www.linkedin.com/in/ecowillson
17:16 jclift tg3: Sent in priv
20:59 a2 foster, ping?
21:02 foster a2: pong
21:02 a2 foster, ah, just pinged you on #fs too.. does ioctl(FIEMAP) on XFS return preallocation as extents?
21:03 foster hmm, I would expect so, but lemme give it a check...
21:05 a2 i'm interested to know if a given block is zero filled in a file because it was written with a zero filled buffer, or because it was never written to before
21:06 tg2 joined #gluster-dev
21:06 foster yep it looks like fiemap shows it
21:07 a2 so, there's no way to know if a given block is having zeros because of preallocation or becaues of a write() syscall?
21:08 foster well I think partial pages are written with zeroes, but preallocation is not
21:08 a2 partial pages are OK for me
21:09 foster e.g., if eof falls within a page, I think the remainder of the page should be zeroed for mmap purposes
21:09 a2 yeah.. that's fine
21:09 a2 i guess even if you write subset of a page into a hole it writes in the full granularity of a page to fill the hole?
21:10 a2 s/page/block size/?
21:10 foster block size I think
21:10 foster yeah
21:10 a2 so fiemap does show preallocation as holes?
21:11 foster fiemap just showed me a single extent that included preallocation
21:11 a2 oh :( you mean extra *blocks*?
21:11 a2 brb
21:12 foster yeah
21:12 foster xfs_bmap seems to stop at eof, fwiw
21:19 foster a2: assuming a block is allocated, it's not clear to me how you could tell
21:20 foster the write code looks like it checks for extending writes and zeroes any blocks between current eof and the new eof
21:46 yinyin joined #gluster-dev
21:52 tg3 joined #gluster-dev
22:02 yinyin joined #gluster-dev
22:19 a2 foster, hmm
22:19 a2 if i truncate a 100GB vm image
22:20 a2 as one big hole
22:20 a2 and then start writing to regions inside, say for some regions contiguously
22:20 a2 will preallocation kick-in and get marked as 'valid' data blocks?
22:21 a2 foster, and therefore, what is FIEMAP_EXTENT_UNWRITTEN?
22:22 foster a2: I don't think so, preallocation should kick in when you write past eof (or it might have been when eof hits the last block in the extent)
22:23 foster i think unwritten extents are fallocated extents
22:23 a2 hmm, speculative preallocation is not an unwritten extent?
22:24 foster i believe it's a delayed allocation extent until some data is written
22:24 foster then it's a normal extent (I think, could be wrong)
22:24 foster data is written to disk I mean
22:24 foster hence requiring to really do the allocation
22:24 a2 you mean this: FIEMAP_EXTENT_DELALLOC      = 0x00000004 # Location still pending.
22:25 foster seems so
22:25 a2 are these flags not returned up to userland in the ioctl?
22:25 foster http://xfs.org/docs/xfsdocs-xml-dev/XFS​_User_Guide/tmp/en-US/html/ch02s07.html
22:25 foster ^ unwritten extents
22:26 foster hmm, I'm not sure
22:27 foster xfs_bmap -v includes flags if present
22:27 foster (or at least, it does if I fallocate an extent)
22:29 foster I imagine if it provides a block mapping, it's not going to show a delalloc flag
22:29 foster but I guess there's always the possibility that fiemap/bmap requests force an allocation to happen
22:29 a2 so, is there a way to reliably detect if a block was "written to" or not? (in a fixed size file)
22:30 foster the truncate case you mentioned doesn't work?
22:30 foster i.e., based on whether a file offset is a hole or not?
22:30 a2 so speculative preallocation is only for past-EOF?
22:31 foster yeah
22:31 a2 i thought even if you are writing into a massive hole, there could be speculative prealloc when the write is "streaming"
22:31 foster you can also turn it off with allocsize=4k
22:33 foster no I dont believe so
22:33 a2 i guess delalloc might kick in when you are writing into a hole?
22:33 foster sure, I think delalloc is always the case
22:33 a2 maybe i got confused b/w delalloc and prealloc
22:33 foster unless sync or whatever
22:34 foster well technically it's the same algorithm, just data is being flushed
22:34 foster but if you have something you're testing or whatever, the allocsize=4k mount options will turn it off in all cases
22:35 a2 i am trying to understand real life behavior when allocsize=4k is not specified
22:35 a2 am trying to evaluate if i can reliably check which blocks in a file have been 'written to' using fiemaps
22:36 foster yeah I'm not sure of a way to do that if a block is allocated
22:36 foster the extent info might not know or care what's been written, if anything
22:37 a2 if preallocation is only past EOF, then we can ignore extents which are past EOF and still get that info reliably, right?
22:38 foster well you could seek past eof and get preallocated extents within eof
22:39 a2 argh
22:39 foster :P
22:39 foster that was the problem we had with stripe, iirc
22:39 a2 can we disable prealloc on a per-file basis? with fadvise?
22:40 foster hmm, not at the moment. but perhaps that's something we could add
22:40 a2 does xfs even know what blocks were written to and which were never?
22:40 foster depends on how its received upstream tbh
22:41 a2 it would have witnessed all the IO after all
22:41 foster my inclination is to say not unless it's explicitly marked as an unwritten extent
22:41 foster otherwise it has no reason to track it
22:41 foster but I could certainly be wrong, I haven't dug into the depths of block mapping much yet
22:43 a2 it would be very little (marginal) work for a filesystem to maintain which regions were written to, and this would work great for 'dirty block tracking' to implement file snapshots on top of
22:44 a2 essentially all non-holes would be dirty blocks (for a VM image file)
22:45 foster don't you need dirty block tracking from a point in time?
22:45 a2 yes, just start writing to a new file
22:45 a2 and read from previous file if requested region is hole in the current file
22:46 foster ok, so you basically want to depend on the block allocation algorithm
22:46 a2 essentially waht qcow2 and qed all of them do for 'external snapshots'.. except all the overhead is added in the format for tracking dirty blocks
22:46 a2 yes, the filesystem already knows it all (or is very little work to retain that info)
22:47 foster so this works if you just disable speculative prealloc?
22:47 foster (and don't use fallocate)
22:47 a2 in theory - yep!
22:47 foster well maybe the unwritten extent flag thingy helps in the falloc case
22:48 a2 if fallocate can distinguish itself as unwritten then falloc is not a problem
22:48 a2 yes, exactly
22:48 foster interesting
22:48 foster i have to run out in a few (and I'm on pto tomorrow)...
22:49 a2 ok enjoy!
22:49 foster if you have time, could you send me a quick mail with the context?
22:49 a2 ok.. i'll do that
22:49 foster going away for the weekend, but I'll try to digest it when I'm back :)
22:50 a2 have a good weekend :-)
22:50 foster thanks. ttyl

| Channels | #gluster-dev index | Today | | Search | Google Search | Plain-Text | summary