Camelia, the Perl 6 bug

IRC log for #gluster, 2013-07-18

| Channels | #gluster index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:00 tom_cruise And follow the instructions, it doesn't get me back to a situation like before 2 was rebuilt.
00:00 JoeJulian Ok. The place to look if you want to try to figure this out before I get there is in /var/log/glusterfs/etc-glusterfs-glusterd.vol.log . This problem is a new once since 3.4 so I haven't had an opportunity to diagnose it before.
00:00 tom_cruise Thanks, I'll look there.
00:01 JoeJulian See you in a bit.
00:01 tom_cruise great, thanks.
00:11 tom_cruise http://permalink.gmane.org/gmane.co​mp.file-systems.gluster.user/12271
00:11 glusterbot <http://goo.gl/zOcJd> (at permalink.gmane.org)
00:16 harish joined #gluster
00:28 aknapp joined #gluster
00:30 yinyin joined #gluster
00:57 bleepblop_ joined #gluster
00:57 bleepblop_ left #gluster
00:58 zombiejebus joined #gluster
01:10 zombiejebus joined #gluster
01:24 Ehtyar joined #gluster
01:26 Ehtyar hi guys I was having some trouble getting two hosts to replicate but after whitelisting the hosts in iptables it worked. Now when I check netstat the two servers are using several ports not listed in the documentation all between 1000 and 1030. Can anyone tell me what they're for please?
01:27 yinyin joined #gluster
01:28 Ehtyar relevant rules and netstat output here: http://pastebin.com/2Say68Je
01:28 glusterbot Please use http://fpaste.org or http://dpaste.org . pb has too many ads. Say @paste in channel for info about paste utils.
01:30 samppah @ports
01:30 glusterbot samppah: glusterd's management port is 24007/tcp and 24008/tcp if you use rdma. Bricks (glusterfsd) use 24009 & up. (Deleted volumes do not reset this counter.) Additionally it will listen on 38465-38467/tcp for nfs, also 38468 for NLM since 3.3.0. NFS also depends on rpcbind/portmap on port 111.
01:30 samppah @3.4 ports
01:30 samppah @34 ports
01:30 Ehtyar duplicated here: https://dpaste.de/Reh14/
01:30 glusterbot Title: dpaste.de: Snippet #234656 (at dpaste.de)
01:31 Ehtyar samppah: all those ports are open
01:31 Ehtyar and none of it explains the lower 1000s being used
01:33 samppah what version you are using?
01:33 Ehtyar 3.4
01:35 samppah Ehtyar: i heard that there has veen some changes in portusage on 3.4
01:35 samppah I just can't find any info about it
01:41 Ehtyar i can't find any either
01:43 samppah those are open on client side, right?
01:43 Ehtyar on both
01:44 tom_cruise left #gluster
02:16 raghug joined #gluster
02:28 bobby_ joined #gluster
02:43 sprachgenerator joined #gluster
02:51 kshlm joined #gluster
02:53 [o__o] joined #gluster
02:56 bulde joined #gluster
03:02 bharata joined #gluster
03:04 bulde joined #gluster
03:15 sprachgenerator joined #gluster
03:32 puebele3 joined #gluster
03:39 yinyin joined #gluster
03:39 _pol joined #gluster
03:47 lalatenduM joined #gluster
03:52 hagarth joined #gluster
03:53 ultrabizweb joined #gluster
03:54 glusterbot New news from resolvedglusterbugs: [Bug 923398] NFS problem <http://goo.gl/0UbO5>
03:59 raghug joined #gluster
04:02 lalatenduM joined #gluster
04:12 chirino joined #gluster
04:19 shylesh joined #gluster
04:26 hagarth joined #gluster
04:32 sgowda joined #gluster
04:34 skyw joined #gluster
04:39 bulde joined #gluster
04:46 _pol joined #gluster
04:47 CheRi joined #gluster
04:48 puebele1 joined #gluster
04:48 yinyin joined #gluster
04:50 rjoseph joined #gluster
04:53 shylesh joined #gluster
04:53 raghu joined #gluster
04:57 badone_ joined #gluster
04:58 _pol joined #gluster
05:03 lalatenduM joined #gluster
05:13 chirino joined #gluster
05:16 _pol joined #gluster
05:19 vpshastry joined #gluster
05:35 sgowda joined #gluster
05:35 badone_ joined #gluster
05:39 [o__o] joined #gluster
05:48 harish joined #gluster
06:06 rgustafs joined #gluster
06:12 ramkrsna joined #gluster
06:12 ramkrsna joined #gluster
06:14 jtux joined #gluster
06:16 satheesh joined #gluster
06:17 vshankar joined #gluster
06:22 thommy_ka joined #gluster
06:22 _pol joined #gluster
06:22 ricky-ticky joined #gluster
06:23 guigui1 joined #gluster
06:27 dpaul joined #gluster
06:29 Recruiter joined #gluster
06:34 mooperd joined #gluster
06:43 lalatenduM joined #gluster
06:43 pkoro joined #gluster
06:44 _pol joined #gluster
06:45 lala_ joined #gluster
06:45 bulde joined #gluster
06:49 piotrektt joined #gluster
06:52 ekuric joined #gluster
06:52 ctria joined #gluster
06:54 psharma joined #gluster
06:57 tjikkun_work joined #gluster
07:01 vpshastry1 joined #gluster
07:09 jtux joined #gluster
07:13 hybrid5121 joined #gluster
07:41 andreask joined #gluster
07:41 shylesh joined #gluster
07:46 ramkrsna joined #gluster
07:52 mooperd joined #gluster
08:01 ricky-ticky joined #gluster
08:05 bulde joined #gluster
08:11 ProT-0-TypE joined #gluster
08:18 ricky-ticky joined #gluster
08:25 Norky joined #gluster
08:32 X3NQ joined #gluster
08:34 atrius joined #gluster
08:38 sgowda joined #gluster
08:44 deepakcs joined #gluster
08:54 baoboa joined #gluster
08:57 harish joined #gluster
09:00 skyw joined #gluster
09:10 lalatenduM joined #gluster
09:19 vpshastry joined #gluster
09:21 satheesh joined #gluster
09:28 andreask joined #gluster
09:35 vshankar joined #gluster
09:37 skyw joined #gluster
09:39 vimal joined #gluster
09:51 mgebbe joined #gluster
09:54 mmalesa joined #gluster
10:00 glusterbot New news from resolvedglusterbugs: [Bug 844584] logging: Stale NFS messages <http://goo.gl/z72b6>
10:12 rastar joined #gluster
10:28 bulde joined #gluster
10:35 satheesh joined #gluster
10:44 kshlm joined #gluster
10:47 spider_fingers joined #gluster
10:53 ekuric joined #gluster
10:54 rastar joined #gluster
11:05 bala joined #gluster
11:08 vshankar joined #gluster
11:13 kkeithley1 joined #gluster
11:20 bala joined #gluster
11:23 CheRi joined #gluster
11:39 chirino joined #gluster
11:44 mmalesa_ joined #gluster
11:44 lpabon joined #gluster
11:46 rjoseph joined #gluster
11:47 Recruiter joined #gluster
11:48 andreask joined #gluster
12:01 rwheeler joined #gluster
12:01 rwheeler Morning/afternoon - ready for that call?
12:01 rwheeler International dial-in number: (212) 729-5016
12:01 rwheeler United Kingdom : 02078970515
12:01 rwheeler Conference code: 3375198528
12:02 * rwheeler pasted into the wrong window, sorry!
12:03 T0aD lawl.
12:08 rastar joined #gluster
12:09 semiosis chirino: yesterday i figured out how to use the heap to call readdir_r with its 'struct dirent **result' parameter.  stumped at first but i just kept readnig the hawtjni dev guide over & over until i got it
12:10 semiosis now i just have to figure out how to use readdir_r, it's strange
12:11 chirino semiosis: ah, so you malloc/free/memove ??
12:11 semiosis yes
12:11 semiosis well, not free... i should probably do that as well
12:11 chirino :)
12:13 chirino FYI: there's a org.fusesource.hawtjni.runtime.PointerMath.add() that can aid in doing pointer math if you need to do stuff like work /w arrays.
12:14 semiosis i suspect that may come in handy when calling seekdir
12:15 clag_ joined #gluster
12:16 clag_ left #gluster
12:16 chirino Once you starting doing C heap allocations, your java code starts to look alot like C code :)
12:17 semiosis pointer arithmetic will really help that along too
12:18 guigui3 joined #gluster
12:18 semiosis ok added free, good tip
12:26 kkeithley_ @later JoeJulian: just because you asked :-).  3.4.0 rpms for epel-5 are on download.gluster.org.
12:27 semiosis s/@later/@later tell/ also s/JoeJulian:/JoeJulian/
12:28 glusterbot semiosis: Error: I couldn't find a message matching that criteria in my history of 1000 messages.
12:28 semiosis glusterbot: meh
12:28 glusterbot semiosis: I'm not happy about it either
12:28 kkeithley_ @later tell JoeJulian: just because you asked :-).  3.4.0 rpms for epel-5 are on download.gluster.org.
12:28 glusterbot kkeithley_: The operation succeeded.
12:31 edward1 joined #gluster
12:36 glusterbot New news from newglusterbugs: [Bug 985874] [posix.c:1754:posix_create] 0-app-storage-posix: setting xattrs on /media/gluster/ftp/sa-prod_ftp_trilogy/​incoming/orderconfirm/orderconfirm.tmp failed (Operation not supported) <http://goo.gl/BsoOu>
12:40 lalatenduM joined #gluster
12:41 CheRi joined #gluster
12:57 bennyturns joined #gluster
13:01 T0aD http://www.bpaste.net/show/3sOIll2vlRowUWmnIafr/
13:01 glusterbot <http://goo.gl/iqqjy> (at www.bpaste.net)
13:02 T0aD *ouch*
13:03 lalatenduM joined #gluster
13:05 bfoster joined #gluster
13:07 shylesh joined #gluster
13:12 bradfirj dats bad
13:12 Debolaz joined #gluster
13:12 deepakcs joined #gluster
13:17 tqrst started rebalancing a 3.4 volume about 40 hours ago. glusterfs's memory usage has gone from 1.5% initially, to 7.4% the day after, and right now up to 12.6%. This is a server with 16G of memory. Is this normal? Some servers seem to be more affected than others.
13:30 T0aD thats very funny
13:30 T0aD it segfaults at 1060th entry
13:30 T0aD when the total number of entries is like 7000
13:30 T0aD when there are 5000 entries, there is no problem
13:31 bala joined #gluster
13:32 kedmison joined #gluster
13:36 failshell joined #gluster
13:42 pkoro joined #gluster
13:54 jebba joined #gluster
14:02 zhashuyu joined #gluster
14:08 raghug joined #gluster
14:08 bugs_ joined #gluster
14:09 daMaestro joined #gluster
14:10 vpshastry joined #gluster
14:11 puebele1 joined #gluster
14:11 vpshastry left #gluster
14:12 puebele1 left #gluster
14:14 raghug joined #gluster
14:14 _pol joined #gluster
14:16 vincent_vdk joined #gluster
14:18 _pol joined #gluster
14:20 mmalesa joined #gluster
14:21 raghug joined #gluster
14:23 plarsen joined #gluster
14:27 lpabon joined #gluster
14:29 lpabon joined #gluster
14:29 hagarth joined #gluster
14:41 chirino joined #gluster
14:43 shylesh joined #gluster
14:45 lwoj joined #gluster
14:46 lwoj Anyone around to answer a few questions about gluster?
14:46 jag3773 joined #gluster
14:47 dbruhn lwoj: what do you need to know?
14:47 daMaestro don't ask to ask, just ask ;-)
14:48 lwoj First timer here ;)
14:48 semiosis hello
14:48 glusterbot semiosis: Despite the fact that friendly greetings are nice, please ask your question. Carefully identify your problem in such a way that when a volunteer has a few minutes, they can offer you a potential solution. These are volunteers, so be patient. Answers may come in a few minutes, or may take hours. If you're still in the channel, someone will eventually offer an answer.
14:48 semiosis lwoj: ^^
14:49 lwoj Ok I have 3 nodes with 6gb SAS cards + MD3200.  I need 1 LUN shared across all 3 nodes allowing reading and writing
14:49 lwoj from what I'm seeing... gluster might assist in that
14:49 dbruhn Why do you need a LUN?
14:49 dbruhn Or are you using the word LUN as a single name space, and aren't concerned about block level
14:49 lwoj fast 6gb/s
14:50 semiosis gluster is a virtual filesystem, not a virtual block device
14:50 lwoj When I say LUN it's what I'm carving off my san and presenting to the hosts
14:50 lwoj Ok so what if I do this then
14:50 lwoj Carve out a separate LUN for each of the 3 nodes
14:50 lwoj mount it
14:50 lwoj and then use gluster to join them together
14:51 lwoj Or am I not understanding exactly how gluster works. Pardon my ignorance I was turned onto gluster yesterday and researched most of the night
14:51 semiosis sounds possible, but gluster is ususally not used with a san
14:51 lwoj Ok
14:51 dbruhn How do you expect to have your clients work against the global name space
14:51 lwoj That's what I was wondering
14:51 semiosis usually it's used to combine direct attached storage of many servers
14:51 semiosis into a unified volume
14:51 lwoj Right
14:52 lwoj So direct attached would also include a direct attached SAN though?
14:52 dbruhn Yep
14:52 lwoj The nodes will see the drives as directly attached
14:52 lwoj So it's still OK to use gluster for this
14:52 dbruhn What protocol would you be using from your gluster servers, to the SAN
14:52 semiosis direct attached storage (DAS) usually means a disk or raid array dedicated to a single server
14:53 semiosis s/dedicated/directly attached/
14:53 glusterbot What semiosis meant to say was: direct attached storage (DAS) usually means a disk or raid array directly attached to a single server
14:53 bala joined #gluster
14:53 lwoj dbruhn SAS cables
14:54 lwoj All 3 servers have 6gb/s SAS HBA cards
14:54 dbruhn What might help us make a recommendation is knowing what you are trying to accomplish
14:54 lwoj True
14:54 lwoj Let me explain
14:55 lwoj We have 3 servers
14:55 semiosis maybe gfs?  http://en.wikipedia.org/wiki/GFS2
14:55 glusterbot Title: GFS2 - Wikipedia, the free encyclopedia (at en.wikipedia.org)
14:55 lwoj We need 1 large common storage system between all 3, utilizing the 6gb/s SAN connection
14:55 neofob joined #gluster
14:55 zhashuyu joined #gluster
14:56 dbruhn What are your three servers doing that they need this shared storage?
14:57 lwoj Running some vertica software "High-speed, self-tuning column-oriented SQL database management software for data warehousing and business intelligence"
14:57 lwoj It's for a client, I don't know much more than that.
14:58 semiosis this is at least an "uncommon" use for glusterfs, just so you're aware
14:58 semiosis at least as far as what people commonly talk about using glusterfs for in here
14:58 lwoj I appreciate the honesty
14:58 rcheleguini joined #gluster
14:58 lwoj So there's probably a better solution to my problem
14:58 semiosis idk
14:59 lwoj See, originally, I had thought I could just share my 1 large SAN LUN to all 3, mount it, and have them all be happy reading and writing at the same time
14:59 lwoj turns out, that will corrupt the data / fs
14:59 semiosis i think that's what GFS2 is for
14:59 lwoj Yea
14:59 lwoj semi
14:59 lwoj thats what I was getting to next
14:59 dbruhn I would say that at minimum you would be needing to use RDMA and Infiniband to make gluster do what you want, and you would be forcing something that you could probably find a better fit for
14:59 lwoj I think this is what I want to do
14:59 lwoj http://mossiso.com/2013/02/01/setting-up-a-hostin​g-environment-part-3-redhat-cluster-and-gfs2.html
14:59 glusterbot <http://goo.gl/5oGJY> (at mossiso.com)
15:02 T0aD jesus this segfault is very wrong
15:02 brosner joined #gluster
15:03 lwoj Thanks everyone for their time/knowledge!
15:03 semiosis good luck
15:03 lwoj Thanks
15:04 lwoj but while I'm here
15:04 lwoj what is the ideal situation for gluster
15:04 lwoj bc from everything I'm reading it's pretty amazing
15:04 lwoj its combining multiple nodes local storage into 1? Keeping everyone in sync?
15:05 brosner joined #gluster
15:05 semiosis "good fit" workloads are things like... scientific datasets operated on by lots of parallel processes, storage for web (especially multimedia)
15:05 ekuric1 joined #gluster
15:05 semiosis more recently also VM image storage
15:07 glusterbot New news from newglusterbugs: [Bug 985946] volume rebalance status outputting nonsense <http://goo.gl/VO7By> || [Bug 892808] [FEAT] Bring subdirectory mount option with native client <http://goo.gl/wpcU0>
15:10 lwoj left #gluster
15:14 bala joined #gluster
15:18 brosner joined #gluster
15:26 T0aD http://www.bpaste.net/show/MZZhWru3GIiF2YeEvb8v/ its funny, its always crashing at 1060th entry no matter how many entries im parsing when their total is > 6000
15:26 glusterbot <http://goo.gl/JAcsh> (at www.bpaste.net)
15:31 hagarth joined #gluster
15:32 daMaestro joined #gluster
15:33 spider_fingers left #gluster
15:39 glusterbot New news from newglusterbugs: [Bug 985957] Rebalance memory leak <http://goo.gl/9c7EQ>
15:42 txdoman joined #gluster
15:42 txdoman left #gluster
15:45 risibusy joined #gluster
15:46 zaitcev joined #gluster
15:48 txdoman joined #gluster
15:54 _pol joined #gluster
15:57 puebele2 joined #gluster
16:01 lalatenduM joined #gluster
16:07 JoeJulian T0aD: What? You're getting a segfault in malloc? That can't be right...
16:07 T0aD yep
16:07 T0aD damn gluster fails me everytime
16:08 T0aD always happens at 1060th entry when total number of entries > 6000
16:08 T0aD so i guess it has something to do with the volume configuration file reader
16:10 kedmison joined #gluster
16:10 Technicool joined #gluster
16:10 glusterbot New news from newglusterbugs: [Bug 976558] NUFA on distributed 2-way replicated volume corrupts inodes for parallel mapreduce jobs (TestDFSIO, TeraGen, TeraSort) resulting in FNF and corrupted filesystem <http://goo.gl/SzbyW> || [Bug 848556] glusterfsd apparently unaware of brick failure. <http://goo.gl/rIjjW>
16:11 JoeJulian Ah, I see. malloc segfaults can be caused by stack corruption further up. :/
16:11 JoeJulian I didn't know that.
16:15 aliguori joined #gluster
16:21 JoeJulian T0aD: I've got some errands to run this morning, but when I get back I should be able to take a few minutes to duplicate your problem. Do you have a test procedure I can duplicate?
16:21 T0aD well yeah i made a set of scripts to supercharge quotas
16:22 T0aD ping me and ill make them available to you
16:25 duerF joined #gluster
16:27 hybrid5123 joined #gluster
16:30 _BuBU joined #gluster
16:31 T0aD yeah thats it
16:31 T0aD when limit-set is reduced, no problem allocating memory for quotas
16:49 kedmison joined #gluster
16:54 T0aD now its a stupid gf_log() who segv
16:56 _pol_ joined #gluster
16:57 _pol joined #gluster
16:59 _pol joined #gluster
17:00 _pol joined #gluster
17:05 _pol joined #gluster
17:10 [o__o] joined #gluster
17:16 bulde joined #gluster
17:20 T0aD yeah there is some bad memory problem, if I replace GF_CALLOC() by a simple malloc(), things go smoothly, otherwise the next gf_log() segv on another malloc
17:30 T0aD [2013-07-18 17:29:10.498552] I [client-handshake.c:1658:sele​ct_server_supported_programs] 0-users-client-0: Using Program GlusterFS 3.3, Num (1298437), Version (330)
17:30 sprachgenerator joined #gluster
17:30 T0aD funny considering im using 3.4
17:31 andreask joined #gluster
17:44 phox joined #gluster
17:44 phox JoeJulian, kkeithley_, etc; any of you around?
17:44 kkeithley_ yes
17:44 phox had a thought on "what to do about the lstat problem"
17:44 phox specifically I think not just Gluster but most of the rest of the world would really benefit from a vectorized lstat
17:45 phox (yeah, I write R a lot, so...)
17:45 phox seems like something to push kernel-ward, and frankly it's really cheap to implement
17:45 phox one syscall to rule them all, and in the darkness... make them faster?
17:46 * T0aD bows before phox
17:46 phox so I was figuring RH involvement in Gluster might be kinda key to this, as it'd probably be easier to have RH kernel devs involved in this than JUST ask LKML-land to go implement the kernel side of this
17:46 phox and obviously libc needs to join the party, etc.
17:47 * phox has a day job or would do this
17:47 kkeithley_ yeah
17:47 T0aD whats the lstat problem ?
17:48 phox listing a directory incurs the wrath of gluster checking every single file in that dir on every single brick because it's paranoid about things being out-of-sync, basically
17:48 phox "whoap, better make sure this actually IS consistent!"
17:48 T0aD ah yeah it will trigger some self-heal capability is that it ?
17:48 phox the kinda-other solution is to basically have a cache coherency table for the relevant VFS
17:49 phox well that isn't the problem, but yes that's its intent IF something isn't kosher
17:49 phox but the general case where that ISN'T triggered is really the issue here
17:52 bulde joined #gluster
17:55 T0aD anyway im already busy digging the source on some other matter
17:58 phox heh
17:59 phox I have too much internal code here to be digging other code :l
17:59 phox kinda miss having the time to thoroughly rip stuff like this apart
17:59 phox free time is cool
18:01 _pol joined #gluster
18:06 kedmison joined #gluster
18:11 cfeller Peformance question:  I upgraded from 3.3.1 to 3.3.2 a couple of days ago, and I'm seeing a _huge_ increase in performance  in some use cases.  One example is directory listings via apache.  I'm seeing at least an order of magnitude increase.  I'm so pleased with the increase that I'm now hesitant to upgrade to 3.4.0.
18:11 cfeller that said, does anyone have performance benchmarks between 3.3.2 and 3.4.0?
18:12 T0aD why dont you give it a shot
18:12 * kkeithley_ thinks: if it ain't broke——
18:16 kedmison joined #gluster
18:18 _pol joined #gluster
18:29 Recruiter joined #gluster
18:29 skyw joined #gluster
18:34 phox cfeller: hm I think I should go install 3.3.2 then :o
18:35 phox am I correct in understanding that 3.4.0 still doesn't have RDMA support back in it?
18:41 duerF joined #gluster
18:41 phox erm, anyone have any idea when 3.3.2 or 3.4.0 debs might be up on the repo?
18:43 bronaugh_ joined #gluster
18:43 bronaugh_ so, whee. just noticed you guys released 3.4.0. congrats :)
18:43 puebele joined #gluster
18:43 bronaugh_ noticed centOS builds up today. how're deb builds coming?
18:54 phox specifically for wheezy
18:56 kedmison joined #gluster
19:05 kkeithley_ both 3.3.x and 3.4.0 have RDMA support. Always have.
19:08 dpaul joined #gluster
19:10 kkeithley_ Patrick Matthäi said he built packages for Debian. Not sure where he put them
19:23 brosner joined #gluster
19:26 mmalesa joined #gluster
19:28 _pol joined #gluster
19:44 * tqrst twiddles his thumbs as he watches glusterfs eat up more and more memory
19:46 kaptk2 joined #gluster
19:50 phox kkeithley_: my understanding was that the code paths are there but it's disabled somehow
19:50 phox i.e. AFAIK our 3.3.1 here is NOT using RDMA
19:51 phox 200MB/sec is not RDMA performance, again AFAIK
19:59 hateya joined #gluster
20:10 semiosis phox: debian packages will be in the repo this week
20:11 semiosis they went into debian unstable this morning: http://packages.debian.org/search?ke​ywords=glusterfs&amp;searchon=names&​amp;suite=unstable&amp;section=all
20:11 glusterbot <http://goo.gl/X3rRx> (at packages.debian.org)
20:12 semiosis well, 3.4 at least.  i'll have both 3.4 & 3.3.2 on download.gluster.org this week though
20:15 phox semiosis: sweet, ok, thanks.
20:15 phox I'm wondering which one to switch to.  inclined to go to 3.4, but also inclined to wait 1-2 weeks to see how a new release does with other people's data first :)
20:16 semiosis i understand.  still on 3.1.7 in prod myself :)
20:16 phox heh
20:16 phox 3.3.1 here.  some gimpy issues. and performance is poor in a good number of ways.
20:16 semiosis re: 200MB/s... that's 1.6 Gbps right?
20:17 semiosis and if that's client throughput, possibly higher with replication taken into account
20:17 ProT-0-TypE joined #gluster
20:18 phox there's no replication in our installation
20:18 semiosis oh
20:18 phox my understanding was that we'd get more like 500MB/sec with RDMA
20:18 phox which AFAIK plenty of people have seen with a single brick etc etc
20:18 phox semiosis: mostly we're doing this because NFS is so -incredibly- broken
20:19 semiosis how broken is it?
20:22 semiosis kkeithley_: is the key you used to sign the debian packages you built way back when something you can share with me?  or should I make a new key to sign the debs i'm going to build?
20:43 edong23 joined #gluster
20:58 elyograg joined #gluster
21:04 elyograg will the existing kkiethley yum repo be updated to 3.4, or will i need a new one?
21:10 tqrst What happens if you reboot a server while a 2-distributed-replicate volume is rebalancing? Will rebalance pause for the files that are on that server's bricks?
21:14 semiosis elyograg: i suspect kkeithley_ is gone for the day (just a guess) we may have to wait until tomorrow
21:15 elyograg i don't need anything to happen now, just wondering whether 3.4 will go into the existing repo.  I don't really have any reason for thinking it won't, just confirming.
21:16 elyograg I just barely sumbitted the internal ticket for the upgrade, ther's no way it will get approved for a little while.
21:16 atrius joined #gluster
21:21 brosner joined #gluster
21:25 semiosis elyograg: there's a repo at ,,(latest) can't you use that?
21:25 glusterbot elyograg: The latest version is available at http://goo.gl/zO0Fa . There is a .repo file for yum or see @ppa for ubuntu.
21:28 elyograg iirc, there was some reason not to use the official repo, the other one was better.
21:29 elyograg for 3.3.0, back when I first started playing with my testbed.
21:30 elyograg hmm.  the repo file is 3.4 specific.
21:30 elyograg actually, 3.4.0 specific.  have to futz with the repo to upgrade.
21:32 phox semiosis: some ridiculous junk where it randomly causes me to have bad file descriptors.  that's pretty dysfunctional, but I guess better than actually losing data.
21:32 phox semiosis: either on reading or actually on deletion too IIRC
21:32 semiosis oh
21:34 kkeithley_ elyograg, semiosis: use the (,,yum repo)
21:35 semiosis ,,(yum repo)
21:35 glusterbot The official community glusterfs packages for RHEL (including CentOS, SL, etc), Fedora 17 and earlier, and Fedora 18 arm/armhfp are available at http://goo.gl/s077x. The official community glusterfs packages for Fedora 18 and later are in the Fedora yum updates repository.
21:35 semiosis kkeithley_: see my question about the deb signing key?
21:36 kkeithley_ I used my personal gpg key to sign those .debs. You should make a new key
21:36 semiosis ok thx
21:40 kkeithley_ phox: RDMA in 3.3.x and 3.4.0 is not disabled. When I was fixing a bug with RDMA in 3.3.1 a while back I tried with 3.4.0alphaX and created an rdma-only transport volume as a baseline to confirm that my IB gear was working.
21:40 kkeithley_ YMMV.
21:43 _pol joined #gluster
21:49 badone_ joined #gluster
22:02 jebba joined #gluster
22:04 rcheleguini joined #gluster
22:12 kedmison joined #gluster
22:12 bronaugh semiosis: NFS is broken over TCP over RDMA to the extent that it basically hangs randomly for long periods requiring you to unmount then remount the mount.
22:12 semiosis interesting
22:12 bronaugh semiosis: it appears to be related to the high throughput; but beyond that I have not debugged it.
22:12 phox bronaugh: or sometimes reboot.
22:13 phox kkeithley_: huh, ok... but yeah our performance numbers do suggest that it's just floating over TCP/IPoIB
22:15 bronaugh hmm; maybe we were trying that on 3.3.0?
22:16 phox we haven't been running 3.3.0 in production
22:16 phox all of the latest deployments were pulled in as 3.3.1 when originally installed
22:16 puebele joined #gluster
22:16 bronaugh how about you play and see if we are either already using RDMA or if it can be enabled?
22:16 phox hard to miss, because the sources.list entry is locked to 3.3.1 :P
22:16 phox I can create a volume that is RDMA-only and see what it'll do for me
22:17 phox and also a TCP-only one and confirm if they have different performance #s
22:17 bronaugh yeah.
22:17 bronaugh sounds like a plan.
22:18 phox very
22:18 phox couple of other things to do and I'm only here for another hour, so maybe today, probably tomorrow
22:18 phox also need to go kick $helplessdesk in the head about FAST =/
22:21 ProT-0-TypE joined #gluster
22:28 sprachgenerator joined #gluster
22:38 mooperd joined #gluster
22:47 * JoeJulian grumbles about idempotence and dynamic configurations...
22:58 * phox grumbles about having to fix the broken attempt at idempotence in mount.glusterfs
22:58 phox why?  because it was fixed a while ago and not backported :)
22:59 phox file under "embarrassing"
23:03 mooperd joined #gluster
23:07 T0aD JoeJulian, still interested in reproducing a segv ? :)
23:08 JoeJulian yep
23:08 T0aD alright
23:08 T0aD let me put a gist together if it can hold several files
23:08 JoeJulian ok
23:09 jyundt joined #gluster
23:13 T0aD https://gist.github.com/T0aD/6033887
23:13 glusterbot 'Title: Files used to reproduce crashes with GlusterFS configured with more than 6000 quotas.\r If quotas > 6000, it will crash on 1060th\r Basically the issue is in the volume file reading procedure, but Im not skilled enough to see where exactly.\r Use:\r \r ./gen.py 10000 (generate 10.000 quotas and store them in pickle file)\r ./inject_quota.py (at gist.github.com)'
23:13 T0aD good luck soldier.
23:17 a2 left #gluster
23:17 a2 joined #gluster
23:29 brosner joined #gluster

| Channels | #gluster index | Today | | Search | Google Search | Plain-Text | summary