Camelia, the Perl 6 bug

IRC log for #gluster-dev, 2013-03-15

| Channels | #gluster-dev index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:55 mohankumar joined #gluster-dev
01:05 copec joined #gluster-dev
01:07 jdarcy joined #gluster-dev
01:40 jules_ joined #gluster-dev
01:58 jules_ joined #gluster-dev
02:30 kkeithley1 joined #gluster-dev
02:40 jclift joined #gluster-dev
03:27 lpabon joined #gluster-dev
03:59 mohankumar joined #gluster-dev
03:59 bharata joined #gluster-dev
04:04 bulde joined #gluster-dev
04:09 aravindavk joined #gluster-dev
04:09 aravindavk joined #gluster-dev
04:18 aravindavk joined #gluster-dev
04:22 sgowda joined #gluster-dev
04:34 mohankumar joined #gluster-dev
04:38 harshpb joined #gluster-dev
05:03 sripathi joined #gluster-dev
05:12 bala joined #gluster-dev
05:17 sgowda joined #gluster-dev
05:22 lalatenduM joined #gluster-dev
05:24 sahina joined #gluster-dev
05:31 bulde joined #gluster-dev
05:35 vshankar joined #gluster-dev
05:45 sripathi joined #gluster-dev
05:51 bala joined #gluster-dev
05:56 sgowda joined #gluster-dev
05:59 lalatenduM joined #gluster-dev
06:03 sripathi joined #gluster-dev
06:03 rastar joined #gluster-dev
06:03 bala joined #gluster-dev
06:04 sripathi1 joined #gluster-dev
06:15 harshpb joined #gluster-dev
06:19 bulde joined #gluster-dev
06:23 bharata joined #gluster-dev
07:02 hagarth joined #gluster-dev
07:57 harshpb joined #gluster-dev
08:21 sripathi joined #gluster-dev
08:34 sripathi1 joined #gluster-dev
08:35 bharata joined #gluster-dev
08:42 badone joined #gluster-dev
09:01 harshpb joined #gluster-dev
09:02 harshpb joined #gluster-dev
09:28 sripathi joined #gluster-dev
10:30 sripathi joined #gluster-dev
10:43 davis_ joined #gluster-dev
11:20 jclift joined #gluster-dev
11:21 harshpb joined #gluster-dev
11:22 harshpb joined #gluster-dev
11:36 jdarcy joined #gluster-dev
11:51 bala joined #gluster-dev
11:51 harshpb joined #gluster-dev
12:11 harshpb joined #gluster-dev
12:13 aravindavk joined #gluster-dev
12:32 deepakcs joined #gluster-dev
12:44 jdarcy joined #gluster-dev
12:51 davis_ Hi, I'm going through Jeff Dary's Translator 101 guide, and I've got a really naïve question about one of the paragraphs. In lesson 2, he says the first 2 arguments to GF_CALLOC are "kind of reversed". As far as I can tell, they could simply be switched so they're not reversed. Am I right about that? If so, is there any reason why they *are* reversed?
13:04 lpabon joined #gluster-dev
13:09 jdarcy davis_: No reason that I know of.  Just a harmless goof, since it doesn't really matter.  The first thing that calloc does is multiply those two anyway.
13:09 jdarcy On older architectures with more arcane structure/array padding rules that stuff mattered a lot more.
13:10 davis_ Yep, ok. Cool. Thought it was totally benign; it's just the note in the article threw me a little bit. I'm just barely starting to grok some of this stuff as it is.
13:10 davis_ Thanks.
13:12 jdarcy I probably should have fixed the order as part of my example.
13:12 jdarcy Glad you're enjoying the articles BTW.  :)
13:13 davis_ They're really good; thanks. Think I have the need to write a translator... so they're pretty much what I'm looking for :)
13:14 jdarcy What are you thinking of doing with a translator?  I love hearing people's ideas on that.
13:15 davis_ Well, we have a number of GlusterFS, and they're backed up by an instance of HP's Data Protector
13:17 davis_ DP can only handle a single backup thread per filesystem (kinda), so we only get one drive to back up an entire Gluster FS. Because the tape library has 4 drives, I'm thinking about an xlator that deterministically (sp?) splits up the FS into 4 different directories, and getting one DP instance to back up each of those directories. Sort of the reverse of DHT, I guess
13:20 jdarcy Interesting.
13:20 edward1 joined #gluster-dev
13:21 davis_ It also should (hopefully!) help the performance of the backup. Prior to running any backup DP runs a TreeWalk to determine what should be backed up (even in the case of a "full" backup, despite my attempts to turn it off!). Because that TreeWalk is lots of directory travesal, and the bricks are all XFS, the TreeWalk takes ages, so DP essentially does nothing for a very long time (around 24 hours sitting about for a 17 TByte FS)
13:23 davis_ I could skip the translator step, and coerce DP into backing up different directories, but I have to manually specify which ones, so I'd rather create an xlator that e.g. always puts "my_dir_path" into "Bin 1", and "an_other_path" into "Bin 2"
13:24 jdarcy Are you marking the directories with xattrs, or something else?
13:25 davis_ I haven't even started yet -- I'm just beginning to get my head back into coding! I'm thinking that simply hashing the top level directory names modulo the number of "bins" (i.e. tape drives) would do it.
13:25 jdarcy I could see having a translator (above DHT) that uses its own consistent-hashing scheme based on the number of DPs instead of the number of bricks, mapping directory GFID to DP using a "magic" xattr (i.e. one that doesn't really exist on disk but gets recalculated on each request).
13:25 davis_ I.e. I don't see that I'd need to mark the top-level directories at all, as they'd always hash/modulo to the same "bin"
13:26 davis_ Ah. I think we've just agreed with each other, sort of.
13:26 davis_ Though I'm missing the understanding of xattr stuff.
13:28 jdarcy I usually think of xattrs as a simple key/value store per file, but since we can intercept all getxattr calls we can have "keys" that we just make up as we go along.
13:29 jdarcy For example, trusted.glusterfs.pathinfo doesn't really exist, but if you ask for it we calculate a value based on the current volume topology.
13:29 jdarcy Maybe I'll write a translator to implement 'user.random.number' as an example.  ;)
13:29 davis_ ah. Ok. I was kind of thinking about storing the hashes in private data structure inside the xlator, but you're saying I should instead calculate the hash and bung it in an xattr associateed with the GFID of the top-level directory?
13:30 davis_ Well, not store it.
13:30 davis_ Have a callback which returns the same value whenever it's asked for, essentially.
13:31 jdarcy davis_: Right.  Once we're at getxattr, we should already have a GFID.  We can map that to a DP number deterministically, and as long as the mapping doesn't change as long as the set of DPs doesn't change then we don't need to store it.
13:31 jdarcy If the DP set does change, the mapping will automatically follow.
13:31 davis_ Well, if the number of DPs change it's because a tape drive is on fire :-)
13:32 jdarcy Or you buy a new unit.
13:32 jdarcy And BTW they do catch on fire sometimes.  ;)
13:32 davis_ yeah, that too. :) Think DP is licensed per drive though so I doubt anyone is looking to fork out any more for HP software :-)
13:33 davis_ Anyway, I'm safe to assume the number of DP bins will not change for the immediate future, and if/when it does I'll be involved anyway.
13:35 jdarcy Sounds like a cool project.  In fact it's not all that different than something we've thought of doing wrt classifying storage into different "bins" that are assigned to different set of bricks (e.g. based on brick kind or location).
13:35 davis_ HSM kind of thing?
13:36 jdarcy That's one possibility.  Also directing certain "hot" data to SSDs, or data for a certain experimenet to the rack that's closest to the compute cluster that's grinding on it.
13:42 davis_ hmmm not sure I see the direct parallel: surely the "hot" data handling would try to keep data in the same namespace as far as clients are concerned: in my case I have DP as a client for whom I'm trying to split out the namespaces.
13:42 davis_ too many colons. Bah
13:43 davis_ ...and that last sentence doesn't parse that well, either. :)
13:48 jdarcy davis_: It would all be in the same namespace, but it would be put on different physical bricks based on the user's policies.
13:48 jdarcy davis_: Relocating it to a different kind of storage would mean re-tagging and rebalancing (just that directory) - all within one volume, transparently.
13:49 davis_ gotcha
13:52 davis_ actually that would be particularly useful for us, too.
13:56 johnmark davis_: oooh. do I hear hte rumblings of a potential developer?
13:57 davis_ johnmark: Hah. I wish I had the ability. I'm an admin-type of bloke who's currently trying to make a backup headache go away. I havne't hacked any C for a long time...
13:59 lpabon joined #gluster-dev
14:13 foster joined #gluster-dev
14:29 wushudoin joined #gluster-dev
14:34 jbrooks joined #gluster-dev
14:42 jdarcy joined #gluster-dev
14:42 lpabon_ joined #gluster-dev
14:43 lpabon_ joined #gluster-dev
14:45 lpabon joined #gluster-dev
14:51 _benoit_ left #gluster-dev
15:27 lalatenduM joined #gluster-dev
15:33 hagarth joined #gluster-dev
15:53 hagarth1 joined #gluster-dev
16:15 vshankar joined #gluster-dev
16:47 gbrand_ joined #gluster-dev
17:13 lalatenduM joined #gluster-dev
17:25 bulde joined #gluster-dev
17:25 puebele joined #gluster-dev
21:24 jdarcy joined #gluster-dev
22:01 badone joined #gluster-dev

| Channels | #gluster-dev index | Today | | Search | Google Search | Plain-Text | summary