Camelia, the Perl 6 bug

IRC log for #gluster, 2012-12-30

| Channels | #gluster index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
01:15 bauruine joined #gluster
02:45 robo joined #gluster
02:51 yinyin joined #gluster
03:48 badone joined #gluster
04:14 badone joined #gluster
04:20 raven-np joined #gluster
04:30 __Bryan__ joined #gluster
04:53 yinyin joined #gluster
05:15 badone joined #gluster
05:20 badone joined #gluster
07:31 yinyin joined #gluster
07:49 hateya joined #gluster
10:50 isomorphic joined #gluster
11:28 lh joined #gluster
11:28 lh joined #gluster
11:36 mohankumar joined #gluster
11:45 Hymie joined #gluster
12:54 mohankumar joined #gluster
13:46 sunus joined #gluster
15:10 nightwalk joined #gluster
15:11 wushudoin joined #gluster
15:41 isomorphic joined #gluster
16:37 mozg_ joined #gluster
16:38 mozg_ hello guys
16:38 mozg_ anyone here?
16:39 mozg_ could some help me with migrating data stored on the server to glusterfs?
16:57 mozg_ joined #gluster
17:19 jjnash joined #gluster
18:03 nightwalk joined #gluster
18:11 mozg__ joined #gluster
18:31 hateya joined #gluster
19:15 _br_ joined #gluster
19:21 _br_ joined #gluster
19:22 mozg__ joined #gluster
19:49 daMaestro joined #gluster
20:19 VeggieMeat joined #gluster
20:19 samppah joined #gluster
20:19 tjikkun joined #gluster
20:19 tjikkun joined #gluster
20:19 ndevos joined #gluster
20:19 jiffe1 joined #gluster
20:19 H__ joined #gluster
20:20 hagarth joined #gluster
20:20 mnaser joined #gluster
20:45 JoeJulian @which brick
20:52 JoeJulian @learn which brick as To determine on which brick(s) a file resides, run "getfattr -n trusted.glusterfs.pathinfo $file" through the client mount.
20:52 glusterbot JoeJulian: The operation succeeded.
21:35 mynameisdeleted hi all
21:36 mynameisdeleted so.. here is sample scenerio.. I have raid5... if I have one computer running locatedb while another is reading a huge media file they compete for disk io
21:36 mynameisdeleted on my netowrk drive
21:36 mynameisdeleted if I upgrade to gluster and make 2 raid5 arrays one can read at full raid5 performance while another uses the 2nd mirrored array?
21:36 mynameisdeleted to locate files?
21:36 mynameisdeleted or do lots of small operations?
21:36 JoeJulian Unrelated question, do you actually use locatedb?
21:37 mynameisdeleted I use it
21:37 mynameisdeleted I have it run at 3 or 4 am
21:37 mynameisdeleted so thats not typical that I have issue
21:37 mynameisdeleted more typi9cal is someone loads kde off network drive
21:37 JoeJulian I disable it on all my systems as I never found myself using it, so I was just curious.
21:37 mynameisdeleted or one person indexes 50k photos while another reads a movie
21:38 mynameisdeleted large file read performance is improved best with raid5 or raid6 with proper lookahead and parameters etc
21:38 mynameisdeleted and caching
21:38 mynameisdeleted so it reads an entire stripe, 1 block per drive or writes taht at a time
21:38 JoeJulian Ok, so your question seems to be if reads to replicated volumes balances the load, right?
21:38 mynameisdeleted I think storing all the filenames on a solid-state drive raid1 would be ideal for performance for that
21:38 mynameisdeleted yup
21:39 mynameisdeleted I'd like read-load balancing
21:39 JoeJulian As of 3.3, not exactly. It reads from the "first to respond" which is generally the one with the least load (but not necessarily).
21:39 mynameisdeleted but it will be an improvement
21:40 mynameisdeleted if the same client has multiple programs running tha tneed disk i/o it can use 2 replicated volumes from one client?
21:40 JoeJulian Though I would expect that if one server is SSD, that one's going to respond first most of the time.
21:40 mynameisdeleted I might replace ext4 with something that saves data to one drive and filenames to a smaller faster ssd array
21:40 mynameisdeleted so file-content to raid5
21:40 JoeJulian Yes. The "first to respond" method is per file descriptor.
21:40 elyograg tangent: I wish xfs would let you specify a stripe size larger than 256k.  i wanted to do a 1MB stripe on my hardware raid, but xfs won't do it.
21:40 mynameisdeleted so both arrays try to read the same file?
21:41 mynameisdeleted and actually perform seeks?
21:41 JoeJulian mynameisdeleted: They both get the lookup() call and whoever responds first gets the fd.
21:41 mynameisdeleted could it be tuned to be round-robin with modification in user-space per measured disk load?
21:41 mynameisdeleted so they dont both physically try to spin the disk
21:41 JoeJulian Right.
21:42 mynameisdeleted if I changed ext4 for somethign that requires 2 extra ssd drives to store filenames and locatios
21:42 mynameisdeleted and only stores file-data on the 6*3TB arrays
21:42 JoeJulian As for whether it /could/ be tuned differently, not with the existing code - though that has been discussed.
21:42 mynameisdeleted then updatedb and find might be faster
21:42 mynameisdeleted can gpfs be tuned?
21:42 mynameisdeleted or pvfs?
21:42 JoeJulian I think that might help your updatedb/find, yes.
21:43 mynameisdeleted what filesystem might support storing names on one device and files on another?
21:43 mynameisdeleted I gues gluster keeps its own names database?
21:44 JoeJulian You've got me on that one. I use xfs over lvm on traditional sata drives.
21:44 JoeJulian No, gluster does not store metadata separately.
21:44 mynameisdeleted lustre could use a metadata drive with high performance ssd raid, and huge drives for file-content
21:44 mynameisdeleted with with ssd cache on each machine for frequently read files could be something
21:45 mynameisdeleted ideally I want to optimize find, reading lots of small txt files, and reading 1 or 2 huge files in the GB range
21:45 mynameisdeleted and parallel io
21:46 mynameisdeleted remember... searching hte LD_LIBRARY_PATH or $PATH variable is also determined by find/updatedb performance
21:46 mynameisdeleted so improving that makes all scripts faster
21:47 JoeJulian Unfortunately, this is an area in which glusterfs does not perform well. Caches only last as long as the fd so a directory read is no longer cached after it's read. This can result in a lot of extra round-trips if you're reading stat data.
21:47 JoeJulian s/read./closed./
21:47 glusterbot What JoeJulian meant to say was: As of 3.3, not exactly. It closed. from the "first to respond" which is generally the one with the least load (but not necessarily).
21:47 JoeJulian hah
21:47 JoeJulian That's not what I meant glusterbot.
21:48 JoeJulian Caches only last as long as the fd so a directory read is no longer cached after it's closed.
21:48 elyograg I'm using 12x4TB disks split into two 6-disk raid5 arrays.  Because of the xfs limitation just mentioned, I went with a 256KB stripe at the raid level.  When I created the gpt partition, I used 1MB units.  Then I split that into four volumes using percentages with LVM, and put my XFS filesystems on those.  I used lvm defaults.  should I have went with a specific (256k?) lvm block size for perfect alignment?
21:48 mynameisdeleted it woudl be fun to make my custom fork which supports performance directory struture, performance small files, load balancing and performance huge files
21:49 mynameisdeleted 48TB total... 40TB visible on raid,   20TB space.. not bad
21:49 JoeJulian The way some have been adding a cache is to mount via nfs which then uses the kernel fscache.
21:49 mynameisdeleted 20 if you replicate twice everything
21:49 elyograg two servers, four 5TB bricks per server, replica 2.
21:49 JoeJulian Theoretically you shouldn't need to fork, just implement a new translator.
21:50 elyograg oh, wait.  8 bricks per server.
21:51 JoeJulian elyograg: I wonder if xfs on lvm would be an option to optimize for the larger stripe sizes.
21:51 mynameisdeleted so I make my version with new translator and maybe submit it back
21:51 JoeJulian mynameisdeleted: Yep, see ,,(hack)
21:51 glusterbot mynameisdeleted: The Development Work Flow is at http://goo.gl/ynw7f
21:51 JoeJulian @translator
21:51 glusterbot JoeJulian: I do not know about 'translator', but I do know about these similar topics: 'translators'
21:52 JoeJulian @translators
21:52 glusterbot JoeJulian: http://goo.gl/APZ49
21:52 JoeJulian That's another good link
21:52 elyograg JoeJulian: the first lv on each raid volume ought to be perfectly aligned, but the other three quite possibly aren't.
21:52 JoeJulian Also http://www.gluster.org/community/d​ocumentation/index.php/Developers
21:52 glusterbot <http://goo.gl/qhNCo> (at www.gluster.org)
21:53 JoeJulian elyograg: As long as your extents match your stripes, it should.
21:53 elyograg JoeJulian: that's my worry.  i went with lvm defaults, which iirc is 4k.  probably should have made it 256.
21:53 JoeJulian Right, it's 4k
21:54 JoeJulian er, 4m
21:55 elyograg well, that's a nice multiple of 256k.  i wonder if i should perhaps match the xfs stripe width - for my arrays, that's 5 x 256k.
21:55 JoeJulian The default PE size in LVM2 is 4MiB.
21:58 JoeJulian Maybe just go for a nice round 5m
21:58 JoeJulian Ok, I probably should get back to work in the garage. I'm just sitting here putting it off.
21:59 JoeJulian Need to get that damned thing organized.
22:14 JoeJulian @ports
22:14 glusterbot JoeJulian: glusterd's management port is 24007/tcp and 24008/tcp if you use rdma. Bricks (glusterfsd) use 24009 & up. (Deleted volumes do not reset this counter.) Additionally it will listen on 38465-38467/tcp for nfs, also 38468 for NLM since 3.3.0. NFS also depends on rpcbind/portmap on port 111.
22:17 _br_ joined #gluster
22:24 _br_ joined #gluster
22:34 raven-np joined #gluster
23:18 daMaestro joined #gluster

| Channels | #gluster index | Today | | Search | Google Search | Plain-Text | summary