Camelia, the Perl 6 bug

IRC log for #gluster, 2012-11-27

| Channels | #gluster index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:29 JordanHackworth joined #gluster
00:53 kevein joined #gluster
01:16 gh5046 joined #gluster
01:16 gh5046 left #gluster
01:16 ninkotech_ joined #gluster
01:26 yeming joined #gluster
01:45 yinyin joined #gluster
02:15 pdurbin left #gluster
02:16 sunus joined #gluster
02:45 sunus joined #gluster
03:13 er|c joined #gluster
03:21 sunus joined #gluster
03:22 er|c joined #gluster
03:22 sunus State: Accepted peer request (Connected) what's that state? i can not create a volume with that state
03:24 JoeJulian That's twice that question's come up in one day...
03:24 JoeJulian Guess I'll have to figure out what the answer is on that.
03:55 morse joined #gluster
03:58 atrius joined #gluster
04:00 genewitch joined #gluster
04:01 bulde joined #gluster
04:10 UnixDev_ joined #gluster
04:16 sripathi joined #gluster
04:19 hagarth joined #gluster
04:19 yinyin joined #gluster
04:21 sgowda joined #gluster
04:42 bharata joined #gluster
04:44 overclk joined #gluster
04:46 wariola joined #gluster
04:51 hagarth joined #gluster
04:52 wariola2 joined #gluster
04:56 UnixDev is there some way to manually delete a crashed gluster volume? volume was stopped, but now will not delete with vol delete. result is "failed: operation failed"
05:01 wariola3 joined #gluster
05:01 daMaestro joined #gluster
05:01 yinyin joined #gluster
05:21 vpshastry joined #gluster
05:23 yinyin joined #gluster
05:23 wariola3 joined #gluster
05:33 wheat_thins joined #gluster
05:37 lhawthor_ joined #gluster
05:44 vijaykumar joined #gluster
05:52 mohankumar joined #gluster
05:57 ankit9 joined #gluster
05:59 bala2 joined #gluster
06:02 raghu joined #gluster
06:27 ankit9 joined #gluster
06:40 ramkrsna joined #gluster
06:40 ramkrsna joined #gluster
06:41 wheat_thins joined #gluster
06:51 yeming` joined #gluster
06:56 johnwoo_lee joined #gluster
06:56 ankit9 joined #gluster
06:57 mdarade1 left #gluster
06:58 mdarade joined #gluster
07:02 mdarade left #gluster
07:06 inodb^ joined #gluster
07:10 Daxxial_ joined #gluster
07:10 inodb joined #gluster
07:14 quillo joined #gluster
07:36 guigui1 joined #gluster
07:36 ngoswami joined #gluster
07:38 rudimeyer_ joined #gluster
07:49 puebele joined #gluster
08:01 ctria joined #gluster
08:07 inodb^ joined #gluster
08:25 andreask joined #gluster
08:25 JoeJulian I think you can add the word force to that, UnixDev.
08:25 JoeJulian @split-brain
08:25 glusterbot JoeJulian: (#1) learn how to cause split-brain here: http://goo.gl/nywzC, or (#2) To heal split-brain in 3.3, see http://goo.gl/FPFUX .
08:26 tjikkun_work joined #gluster
08:29 hagarth joined #gluster
08:34 lkoranda joined #gluster
08:52 Humble joined #gluster
08:56 Alpinist joined #gluster
08:59 gbrand_ joined #gluster
09:02 dobber joined #gluster
09:06 yinyin joined #gluster
09:09 vijaykumar joined #gluster
09:10 ankit9 joined #gluster
09:11 mdarade joined #gluster
09:12 hagarth joined #gluster
09:18 inodb joined #gluster
09:23 inodb^ joined #gluster
09:33 hchiramm_ joined #gluster
09:43 yinyin joined #gluster
09:50 inodb joined #gluster
09:53 DaveS_ joined #gluster
10:00 Nevan joined #gluster
10:01 Nevan is there a problem with replicate on glfs 3.3.0 ? when my client is mounted via nfs ... underlyning filesystem is ext4
10:02 Nevan !pastebin
10:02 Nevan !paste
10:09 duerF joined #gluster
10:19 rgustafs joined #gluster
10:30 mdarade left #gluster
10:32 guigui1 joined #gluster
10:35 sgowda joined #gluster
10:55 yinyin joined #gluster
10:55 ndevos_ ~paste | Nevan
10:55 glusterbot Nevan: For RPM based distros you can yum install fpaste, for debian and ubuntu it's dpaste. Then you can easily pipe command output to [fd] paste and it'll give you an url.
10:56 ndevos joined #gluster
10:57 ndevos Nevan: and there is a ,,(ext4) issue, you should check your kernel version and compare it with the one in the blog port
10:57 ndevos *post
10:57 glusterbot Nevan: Read about the ext4 problem at http://goo.gl/PEBQU
10:57 ndevos @ext4
10:57 glusterbot ndevos: Read about the ext4 problem at http://goo.gl/PEBQU
10:58 ndevos hmm
10:58 ndevos glusterbot: ping
10:58 glusterbot pong
11:09 Nevan its not fixed yet?
11:14 glusterbot New news from resolvedglusterbugs: [Bug 763999] Gluster crashes when starting OpenVZ containers from it <http://goo.gl/Z0Zyg>
11:16 ctria joined #gluster
11:17 ndevos Nevan: not that I am aware of, but there are several workarounds (I suggest to use xfs on the bricks)
11:21 Nevan will itonly solve the glsuterfs - kernel -ext4 problem or as well the nfs - glusterfs 3.3 problem?
11:22 ndevos Nevan: the nfs-server is a glusterfs-client, so it will fix both
11:23 Nevan ok
11:23 ndevos Nevan: Bug 838784 contains a small application to check if you are affected with that ext4 problem
11:23 glusterbot Bug http://goo.gl/CO1VZ high, high, ---, sgowda, ASSIGNED , DHT: readdirp goes into a infinite loop with ext4
11:25 vijaykumar left #gluster
11:36 guigui3 joined #gluster
11:39 mohankumar joined #gluster
11:40 Nevan hmm
11:40 Nevan ok its the ext4 bug, used the test programm :-(
11:41 Nevan swo i need to reformat
11:41 Nevan thx
11:41 ndevos or use an other kernel
11:56 yinyin joined #gluster
12:06 kkeithley joined #gluster
12:09 duerF joined #gluster
12:20 Norky joined #gluster
12:21 Norky left #gluster
12:22 Norky joined #gluster
12:23 Norky HEllo. I'm seriously struggling with NFS access to gluster. Despite gluster telling me its NFS element is online on all servers, no filesystem is exported (showmount -e lists nothing)
12:23 Norky I've restarted everything and even gone so far as to delete the volume and recreate it from scratch
12:26 Norky http://pastie.org/5442197
12:27 Norky lnaapp100 is one of many clients, lnasilo{0..3} are the four Gluster servers running Red Hat Storage
12:27 dobber_ joined #gluster
12:37 edward1 joined #gluster
12:42 bfoster joined #gluster
12:43 kkeithley 3.3.0 or 3.3.1? What distro? Built from source or installed from gluster.org packages?
12:43 Nevan 3.3.1 hopefully patched the ext4 bug..
12:43 kkeithley Oh, you're running RHS
12:44 Nevan @Norky, which version and kernel and filesystem type do you use
12:44 kkeithley No, 3.3.1 does not have the fix for the ext4 issue
12:44 Nevan ok, i will use xfs then
12:46 Nevan @norky, you are aware that glusterfs does not use the default nfs port...
12:47 lkoranda_ joined #gluster
12:48 kkeithley The volume should still show up in a `showmount -e`
12:49 Norky the distro is Red HAt Storage (based on RHEL 6.2)
12:49 Norky kernel-2.6.32-220.23.1.el6.x86_64, glusterfs-3.3.0-22.el6rhs.x86_64
12:49 Norky xfs
12:49 kkeithley Yes, RHS 2.0 == glusterfs-3.3.0
12:50 Norky Nevan, yes, I am, and clients can see the correct NFS port via the portmapper, however they cannot see any exports
12:50 Norky note that showmount running locally on each server lists no exports
12:52 ctria joined #gluster
12:53 yinyin joined #gluster
12:53 Norky this is after "gluster volume delete tid" and gluster volume create
12:53 kkeithley so if I kill the glusterfs (nfs) process on my box, gluster volume status still shows the nfs volume and rpcinfo continues to show the mountd and nfsd.  Just checking, are you glusterfs (not glusterfsd) processes still running?
12:54 kkeithley although the showmount -e will fail if glusterfs isn't running
12:54 kkeithley s/are you/are your/
12:54 glusterbot What kkeithley meant to say was: so if I kill the glusterfs (nfs) process on my box, gluster volume status still shows the nfs volume and rpcinfo continues to show the mountd and nfsd.  Just checking, are your glusterfs (not glusterfsd) processes still running?
12:55 Norky yes
12:55 Norky http://pastie.org/5442303
12:55 glusterbot Title: #5442303 - Pastie (at pastie.org)
12:55 kkeithley And since it's RHS, we know that iptables isn't an issue
12:56 kkeithley weird
12:56 Norky indeed
12:57 Norky I'm trying to submit bugs to RH, however this is a trial and my trial licenses have expired (I'm trying to get new ones) so support aint being very helpful
12:57 kkeithley ugh
13:03 andreask left #gluster
13:05 jdarcy joined #gluster
13:14 lkoranda_ joined #gluster
13:15 lkoranda joined #gluster
13:18 lkoranda_ joined #gluster
13:23 mooperd joined #gluster
13:28 Norky well this is weird
13:28 Norky NFS works, after the 4th (5th?) reboot
13:29 kkeithley and nothing else changed?
13:29 Norky not since the previous reboot
13:29 Norky server reboot, not client
13:31 Norky oh, this is precious
13:31 Norky it was working. I stopped the volume; I started the volume; NFS is no longer working
13:34 Norky restarting the glusterd service does not fix it either
13:35 Norky when the service is started "clush -l root -w lnasilo[0-3] service glusterd start", the volume is not started automatically, however it is after a (re)boot
13:38 hagarth joined #gluster
13:40 balunasj joined #gluster
13:43 kkeithley is glusterd itself running after you do that?
13:43 Norky good question, I will dobule check
13:43 Norky double*
13:45 Norky there are three glusterfs processes remaining on each host
13:46 Norky http://pastie.org/5442527 gluster processes left behind after "service glusterd stop"
13:46 glusterbot Title: #5442527 - Pastie (at pastie.org)
13:47 Norky I'm sure I stopped (unmounted) all clients before stopping the service it
13:48 robo joined #gluster
13:52 kkeithley yeah, I'm not surprised. I installed, created vol, started vol, and removed. glusterd process is gone, but the glusterfsd and glusterd processes are still running. I think we don't consider that a bug for some reason.
13:54 Norky presumably after restarting the service and/or volume on your system, that "showmount -e" shows the volume exported via NFS?
13:57 vicelow joined #gluster
13:57 kkeithley POLA, to me, would suggest that if you stop the volume, it doesn't show up in `showmount -e`. When you start it, either explicitly, or when glusterd starts and then (re)starts all previously running volumes, that it would then show up in `showmount -e`.
14:01 Norky POLA?
14:02 Norky my experience so far is that restarting the service does not restart the volume, one has to manually "gluster volume start foo"
14:02 Norky however, I've just stopped the service, killall'ed the three glusterfs processes that were left, andn then restart the service - the volume is now started automatically
14:03 Norky however it does not fix the NFS problem - showmount -e still shows nothing
14:04 aliguori joined #gluster
14:04 theron joined #gluster
14:06 er|c joined #gluster
14:10 Tekni joined #gluster
14:10 Norky kkeithley, what do you mean by "POLA"?
14:11 mooperd How much memory is recomended with gluster?
14:12 Norky it depends...
14:13 Norky what are you using Gluster for, how big is your storage
14:15 Norky more memory is always better, but you can start with machines of 1GB RAM for basic testing
14:22 Norky hmm, now, after a reboot, the volume is not started
14:30 Norky left #gluster
14:31 Norky joined #gluster
14:31 Norky after another reboot, the volume is started.... but NFS is not working again
14:32 Norky I swear, I'm not changing a thing, just rebooting, each time I get different behaviour
14:44 tqrst is it safe to downgrade from 3.3.1 to 3.2.7? The upgrade to 3.3.1 failed miserably and I can't afford to waste another day on this.
14:46 ron-slc joined #gluster
14:46 tqrst my main concern is the "glusterd --xlator-option *.upgrade=on -N" that the upgrade did on all nodes.
14:52 stopbit joined #gluster
14:52 kkeithley POLA = Principle of Least Astonishment
14:53 Norky <kkeithley> POLA, to me, would suggest that if you stop the volume, it doesn't show up in `showmount -e`. When you start it, either explicitly, or when glusterd starts and then (re)starts all previously running volumes, that it would then show up in `showmount -e`.  -- I agree that SHOULD be waht happens. My experience is quite different, and not consistent.
14:53 kkeithley yes, it's puzzling
14:54 Norky my Gluster setup is demonstrating POMA
14:54 kkeithley wish I had an answer
14:54 hchiramm_ joined #gluster
14:54 tqrst mine is demonstrating PONW
14:54 tqrst principle of not working
14:55 ndevos Norky: do you happen to mount an nfs export on you storae servers? or have an export through a non-gluster nfs-server?
14:57 Norky ndevos, I briefly mounted an NFS export on the storage servers last week, but in general, no
14:58 Norky also, yes, I did disable gluster entirely and export something over (standard) NFS to a clinet (in order to compare speed of ordinary NFS versus Gluster)
14:58 Norky but this was some days ago, and the standard NFS services and client have been turned off long ago
14:58 ndevos Norky: okay, so you may have some nfs-client bits enabled? if the non-gluster nfs-server registers itself in the portmapper, gluster-nfs will fail
14:59 Norky ndevos, I think we've covered that
14:59 kkeithley and he's rebooted about seven times since all that
14:59 ndevos Norky: ah, okay
14:59 Norky rpcinfo shows an nfs program that's proto version 3 only, with ports (38467) that correspond to Glsuter's NFS
15:00 Norky when I stop gluster, rpcinfo stops showing an NFS service (and indeed lock manager and mountd)
15:00 torbjorn1_ On a distribute+replicate setup with 2 boxes, 4 bricks (2 on each box), I was doing a single client read at 100MB/s. Firewalled off the active server, causing a failover to the other server. Now throughtput is down to 25MB/s .. is it expected that performance will go down when failover is in effect ?
15:01 torbjorn1_ When the other server comes back, will the failover be reversed, that the clients it had previously would go back to using it ?
15:09 lh joined #gluster
15:10 tqrst is there a way to write questions on http://community.gluster.org/browse/activity/ with markdown instead of having to use that web UI? I have a long question that I want to write in an editor first.
15:10 glusterbot Title: Browse activity (at community.gluster.org)
15:11 tqrst (not a huge fan of tiny textareas that make my browser lag and behave in unpredictable ways)
15:18 hchiramm_ joined #gluster
15:20 lh joined #gluster
15:20 lh joined #gluster
15:24 jbautista joined #gluster
15:33 jbrooks joined #gluster
15:45 kkeithley torbjorn1_: what was the command you used to create the volume?
15:46 kkeithley I created a two-node, four brick replica+stripe volume this way: gluster volume create the_volume replica 2 f17node1:/tmp/bricks/vol0/0 f17node2:/tmp/bricks/vol0/0 f17node1:/tmp/bricks/vol1/1 f17node2:/tmp/bricks/vol1/1
15:48 kkeithley nm, you're asking about reads
15:48 stat1x joined #gluster
15:49 torbjorn1_ kkeithley: My question is regarding reads for now, yeah .. I created it as $SERVER0:$BRICK0 $SERVER1:$BRICK0 $SERVER0:$BRICK1 $SERVER1:$BRICK1
15:49 kkeithley but yes, when the other server comes back, the clients will resume using it
15:50 torbjorn1_ kkeithley: volume info output at http://dpaste.org/eettA/
15:50 glusterbot Title: dpaste.de: Snippet #213898 (at dpaste.org)
15:50 torbjorn1_ server has been back for almost an hour now, still seing the client reading from the "secondary" server, ie: the server it failed over to
15:51 kkeithley and other than there's a 42 second delay any time a server goes away while the client discovers that the server went away, I can't account for the slow reads after a server fails. Other than perhaps your read test isn't long enough to take that 42 second window into account
15:53 kkeithley It's supposed to send the read to both servers, and use whichever one responds first. Can you see the read requests to the first server in tcpdump or wireshark?
15:53 torbjorn1_ kkeithley: ah, 42 seconds is just right for the timeout value :) .. The failover procedure seemed to work as expected, after some time, I would guess 42 seconds, the client gave up on the first server, and changed to the second one
15:54 kkeithley right
15:54 kkeithley that part's clearly working, it's the failback
15:54 torbjorn1_ kkeithley: okay, so the first-to-respond procedure is still active when the read has started ? .. I'm doing this for big single files, and I guessed that the after receieving the first response, the whole read for that file would only go the first responder
15:56 UnixDev is there anyway to get the sync between nodes to work with multiple threads/streams?
15:57 torbjorn1_ kkeithley: I fired up the read request again, this time looking at the client talking to the was-primary server .. there is some chatter, ̃~15 packets worth, but that's it
15:57 torbjorn1_ kkeithley: the read still goes to the secondary server
15:58 torbjorn1_ kkeithley: there is some more chatter on a regular basis, but not a lot of data
16:02 tqrst JoeJulian: hm, looks like I finally fixed this whole upgrade mess
16:02 tqrst JoeJulian: I was going through everything we tried yesterday so I could write this up as a mailing list post
16:02 tqrst JoeJulian: and for some reason, rsyncing another node's /var/lib/glusterd worked this time around
16:02 UnixDev JoeJulian: what branch should i be checking out for production?
16:04 daMaestro joined #gluster
16:05 overclk joined #gluster
16:09 tqrst I'm still getting '[2012-11-27 11:09:17.761004] I [socket.c:1798:socket_event_handler] 0-transport: disconnecting now' every 3 seconds, though.
16:12 bennyturns joined #gluster
16:12 bennyturns ethics
16:12 bennyturns oops wrong channel :)
16:17 UnixDev why would it say "operation failed" when i try to get volume status?
16:19 nhm joined #gluster
16:21 tqrst how do I go from a bug id that was on bugs.gluster.com to whatever new bug tracker you guys use now? I'm trying to find where http://bugs.gluster.com/cgi-bin​/bugzilla3/show_bug.cgi?id=3631 is now.
16:21 glusterbot <http://goo.gl/SgG0v> (at bugs.gluster.com)
16:21 glusterbot Bug 3631: low, low, ---, dkl, CLOSED CURRENTRELEASE, dependancy problem
16:22 tqrst it's the one where quick-read would make perl delete files sometimes (!)
16:25 kkeithley https://bugzilla.redhat.com/show_bug.cgi?id=765363
16:25 glusterbot <http://goo.gl/iVx5c> (at bugzilla.redhat.com)
16:25 glusterbot Bug 765363: urgent, urgent, 3.3.0, rgowdapp, VERIFIED , zero byte files when perl search and replace script
16:25 tqrst JoeJulian: I spoke too quickly - 54 is getting rejected again (sigh)
16:26 tqrst kkeithley: thanks
16:27 lkoranda__ joined #gluster
16:29 Humble joined #gluster
16:34 tqrst kkeithley: it looks like a patch was submitted for this back in March, and approved (http://review.gluster.com/#change,3028), yet the bug report is still 'verified'. Is that normal?
16:34 glusterbot Title: Gerrit Code Review (at review.gluster.com)
16:38 tqrst it looks like the bug doesn't happen any more if I re-enable quick-read
16:41 jiffe98 anyone running vmware images over gluster/nfs?
16:41 kkeithley Verified means that QA has confirmed the issue is resolved (i.e. the bug is fixed). Despite the fact that I work here, I don't know what's involved with going behind later and closing the bz. Maybe someone just forgot?
16:41 jiffe98 wondering how well that works
16:42 tqrst oh, I thought "This bug has been tested by QE, and passed the testing." meant more like "this was reproduced and is really a bug"
16:42 tqrst (from https://bugzilla.redhat.com/p​age.cgi?id=fields.html#status)
16:42 glusterbot <http://goo.gl/LwZEZ> (at bugzilla.redhat.com)
16:49 lkoranda joined #gluster
16:58 mohankumar joined #gluster
16:59 semiosis :O
17:00 tqrst :0
17:05 tqrst more glusterfsd segfaults in 3.3.1: http://pastebin.com/QeRJQrCJ (pastie's down right now)
17:05 glusterbot Please use http://fpaste.org or http://dpaste.org . pb has too many ads. Say @paste in channel for info about paste utils.
17:05 tqrst or http://fpaste.org/Vcrn/ I guess
17:05 glusterbot Title: Viewing Paste #255303 (at fpaste.org)
17:07 andreask joined #gluster
17:08 JoeJulian UnixDev: release-3.3
17:14 JoeJulian tqrst: Could that machine have memory issues? I haven't seen a SIGBUS in forever...
17:14 tqrst JoeJulian: this has happened on 3 different machines so far
17:15 tqrst all fairly new
17:15 tqrst (brb)
17:33 spn joined #gluster
17:33 aliguori joined #gluster
17:35 raghu joined #gluster
17:35 Mo__ joined #gluster
17:48 tqrst any idea why 'volume rebalance myvol status' is outputting the hostname for most nodes except for one for which it outputs the ip instead? http://fpaste.org/bh4s/
17:48 glusterbot Title: Viewing Paste #255317 (at fpaste.org)
17:48 semiosis ~hostnames | tqrst
17:48 glusterbot tqrst: Hostnames can be used instead of IPs for server (peer) addresses. To update an existing peer's address from IP to hostname, just probe it by name from any other peer. When creating a new pool, probe all other servers by name from the first, then probe the first by name from just one of the others.
17:49 tqrst semiosis: thanks, that fixed it
17:49 semiosis cool
17:49 Humble joined #gluster
17:50 tqrst JoeJulian: make that 5 different machines
17:51 tqrst ...6
18:12 rudimeyer joined #gluster
18:29 nueces joined #gluster
18:40 phil______ joined #gluster
18:47 XmagusX joined #gluster
18:48 zwu joined #gluster
18:51 hattenator joined #gluster
18:52 Bullardo joined #gluster
18:56 y4m4 joined #gluster
18:57 rudimeyer_ joined #gluster
19:08 bauruine joined #gluster
19:23 kkeithley JoeJulian: I'm going to put up new rpms in my yum repo. (Two minor tweaks to the glusterfs.spec.: Requires: rpcbind and safer migration of /etc/glusterd to /var/lib/glusterd) Make sure your auto-update isn't going to bite you in the middle of a self-heal!
19:23 JoeJulian :)
19:23 JoeJulian Thanks. I did fix that after the last time.
19:25 JoeJulian I still think having a way to test that the volumes are clean would be good before killing the bricks though. :)
19:25 nueces joined #gluster
19:25 kkeithley yeah, I need to do that
19:27 tc00per joined #gluster
19:38 NuxRo JoeJulian: any news on the gluster UFO (folsom version that is)?
19:53 kkeithley NuxRo: there is no news on a folsom-based UFO other than we're working on it.
19:54 kkeithley And grizzly might be out in time that we'd go straight to grizzly and skip folsom.
19:54 kkeithley we'll see
19:55 tc00per After a peer in a dist-repl cluster goes down then comes back what makes it start serving files again?
19:57 semiosis tc00per: clients will need to notice it's back in business and reconnect to it
19:58 NuxRo kkeithley: cool. do you know if current UFO works ok with the rest of folsom openstack?
19:59 Azrael808 joined #gluster
20:01 kkeithley I haven't tried any of the other openstack pieces to know.
20:02 kkeithley I've heard people say that Swift-1.7.4 isn't so different from 1.4.8. I might guess that they'd play well together, but that's only a guess. YMMV.
20:04 tc00per semiosis: I see all transfers timeout when peer goes away then they carry on after timeout without the missing peer. Transfers seem to come back 'magically' after the peer returns without doing anything, again after a timeout period. This is ideal and I assume expected behavior. I just want to understand and know how to explain it to users.
20:04 semiosis define "peer goes away"
20:05 Technicool joined #gluster
20:05 tc00per host unavailable due to reboot
20:05 semiosis if the host drops off the network abruptly, or glusterfsd ,,(processes) are killed with sigkill, the clients wait for network.ping-timeout (default 42s)
20:05 glusterbot the GlusterFS core uses three process names: glusterd (management daemon, one per server); glusterfsd (brick export daemon, one per brick); glusterfs (FUSE client, one per client mount point; also NFS daemon, one per server). There are also two auxiliary processes: gsyncd (for geo-replication) and glustershd (for automatic self-heal). See http://goo.gl/hJBvL for more information.
20:06 semiosis when you reboot gracefully the glusterfsd processes should be sent sigterm and so not cause a ping-timeout in clients
20:06 semiosis after ping-timeout expires a client will continue working with the remaining servers
20:07 Bullardo joined #gluster
20:08 tc00per are you suggesting that a reboot WILL NOT cause a 42s delay in transfers from remaining peers in the cluster?
20:09 semiosis afaik, that is true for gracefull reboots
20:10 semiosis but it's been a while since i tried, so lets say "iirc"
20:10 semiosis :)
20:10 Bullardo joined #gluster
20:10 semiosis i have used kill (with default sigterm -15) to quit glusterfsd processes and that did not cause ping-timeout in clients
20:11 tc00per ok. I understand. my experience today is that graceful reboot results in a timeout. haven't 'measured' the length.
20:11 semiosis linux sends that to processes at shutdown, so i figure a gracefull reboot should be similar
20:11 semiosis hrm
20:13 elyograg I have noticed a graceful reboot on my testbed causing clients to hang until they time out.  using kkeithley's 3.3.1 repo.
20:15 semiosis s/gracefull/graceful/g
20:15 glusterbot What semiosis meant to say was: linux sends that to processes at shutdown, so i figure a graceful reboot should be similar
20:18 hijakk joined #gluster
20:19 hijakk howdy all, I'm curious about using a gluster volume as a backup destination for a CrashPlan Pro E local install. Anyone know of this being done, or a reason it shouldn't be?
20:20 hijakk I've also asked the crash plan folks directly
20:20 tc00per elyograg: same here, kkeithley's 3.3.1 repo.
20:22 semiosis hijakk: should be fine, as long as the crashplan server can use a locally mounted directory on linux, just make a glusterfs fuse client mount and point crashplan at it
20:22 semiosis hijakk: but i'm curious, what benefit do you expect from using glusterfs under crashplan?
20:23 semiosis looking at their web site, seems like they do distribution and replication already
20:23 hijakk Yeah, true enough
20:23 hijakk I'm considering doing a gluster build anyhow and this seemed like an interesting use case
20:24 hijakk and I'm betting it would be more flexible across more areas we'd want to use a gluster system than running a bunch of crashplan servers
20:25 tc00per Hmmm... updates in kkeithley's repo for CentOS5 but not for CentOS6. Expected?
20:25 semiosis tc00per: are they very recent?  maybe the rest are still in progress
20:25 hijakk I was figuring it should work, but I've learned that "should" doesn't mean does, heh
20:26 kkeithley ??? there are new 3.3.1-3 rpms in epel-6.
20:27 kkeithley as of 30 minutes ago or so
20:27 gbrand_ joined #gluster
20:28 kkeithley none of the source has changed, only some minor tweaks to the glusterfs.spec file, so if you're running 3.3.1-2 or even 3.3.1-1, you've essentially got the latest and greatest.
20:32 tc00per Cleaned my yum cache and they appeared along with a couple of others that got applied to some of my peers but not all.
20:33 tc00per kkeithley: thanks for confirming what I was seeing in the release notes. just trying to document/solidify my server update process to maintain access to data in the cluster.
20:34 kkeithley what were you seeing?
20:34 tc00per * Fri Nov 16 2012 Kaleb S. KEITHLEY <kkeithle[at]redhat.com> - 3.3.1-3
20:34 tc00per - add Requires: rpcbind for minimum install systems where rpcbind isn't
20:34 tc00per installed; usually this is a no-op.
20:34 tc00per - Better logic to preserve contents of /etc/glusterd
20:36 tc00per ...although my peers don't seem to have this directory. :(
20:37 semiosis that was for glusterfs 3.1-3.2
20:37 semiosis if you started with 3.3 you wouldn't have it
20:37 kkeithley Not sure I follow you. You would only have /etc/glusterd if you built from source or installed old gluster.org/gluster.com RPMs from before 3.3.0.
20:38 kkeithley My RPMs have logic to migrate old /etc/glusterd to "standard" /var/lib/glusterd
20:38 tc00per semiosis/kkeithley: I started with 3.3.x
20:38 tc00per A non-issue for me. :)
20:39 kkeithley exactly.
20:39 dberry joined #gluster
20:39 dberry joined #gluster
20:41 tc00per kkeithley: is it safe to assume that changes to rpm version numbers AFTER the '-' are isolated to packaging?
20:42 kkeithley It depends. So far in 3.3.1 that's the case.
20:42 semiosis sometimes packaging can include patches to the source
20:43 kkeithley There were a couple hard crashes in 3.3.0 that I added patches for.
20:43 tc00per OK... rtfm
20:44 kkeithley If I add a patch, it'll be only after it's accepted _upstream_.
20:47 tc00per Is it 'expected' that when applying an update from the repo that the servers will be stopped AND restarted?
20:48 kkeithley The RPM install will stop and restart glusterd. The glusterfsd, glusterfs, and glustershd processes should continue to run until you restart them.
20:48 tc00per glusterd dead but subsys locked
20:49 tc00per glusterfsd is stopped
20:49 elyograg it strikes me as a little odd for an RPM to restart a service.  Not complaining, though.  extremely commonplace for a deb to do so.
20:49 tc00per peer not participating in transfers
20:50 rudimeyer_ joined #gluster
20:51 kkeithley stopping and (re)starting glusterd doesn't affect your storage. (If it does, that's a bug, file a bug report)
20:51 glusterbot http://goo.gl/UUuCq
20:51 tc00per manual start of glusterd required after update, glusterfsd started automatically as expected, peer now participating in transfers.
20:51 tc00per Will do... three chances to replicate... :)
21:05 hijakk left #gluster
21:14 tc00per kkeithley: Verified 3x... manual stop/pause/start of glusterd does NOT affect storage availability.
21:16 tc00per kkeithley: Verified 3x... yum update of gluster* DOES affect storage. manual restart of glusterd required. Are you sure this isn't a package/update issue? Can I do something with my last peer to help capture required information?
21:20 aliguori joined #gluster
21:34 davdunc joined #gluster
21:34 davdunc joined #gluster
21:40 Bullardo_ joined #gluster
21:50 mooperd joined #gluster
22:02 TSM joined #gluster
22:17 tc00per kkeithley: Verified 4x... yum reinstall of gluster* packages does NOT affect storage availability but DOES NOT restart glusterd. manual restart of glusterd required.
22:21 copec joined #gluster
22:48 aliguori joined #gluster
22:55 MalnarThe joined #gluster
22:56 Humble joined #gluster
22:56 MalnarThe hey, how terrible of an idea is it to use gluster to store files that are re-exported as iscsi via loop?
22:57 MalnarThe files would be 1-2TB
23:01 ChiTo joined #gluster
23:01 ChiTo Hi everybody
23:01 ChiTo i have a 8-node architecture to deploy a replicated gluster
23:01 ChiTo every server has 30 Tb
23:01 ChiTo but i have some confussion about the replica count
23:02 ChiTo if i want the 8*30Tb/2 replicated info do i need to have 4 replicas?
23:02 ChiTo or how does it work?
23:10 elyograg ChiTo: with replica 2, you'll have 120TB of space.  With replica 4, you'll have 60TB of space.
23:11 twx_ @repo
23:11 glusterbot twx_: I do not know about 'repo', but I do know about these similar topics: 'repository', 'yum repo', 'yum repository', 'git repo', 'ppa repo', 'yum33 repo', 'yum3.3 repo'
23:11 twx_ @yum repo
23:11 glusterbot twx_: kkeithley's fedorapeople.org yum repository has 32- and 64-bit glusterfs 3.3 packages for RHEL/Fedora/Centos distributions: http://goo.gl/EyoCw
23:17 ChiTo elyograg: And the 8 servers are grouped by 2 for a distributed-replicated? for example node1 and node2 are mirrored node3 and node4 are mirrored, or gluster knows where to mirror?
23:18 sensei joined #gluster
23:19 elyograg ChiTo: the grouping is determined by the order you list the bricks on the create statement (or add-brick).  so you'd do server1:/brick/path1 server2:/brick/path1 server3:/brick/path2 server4:/brick/path2 ... and if you're going to have more than one brick per server, just keep the same pairing.
23:19 elyograg with replica 2, every group of 2 bricks will make a replica set.
23:22 ChiTo elyograg: so if for example i want to use all the 120 Tb but each server has maximum capacity of 30 Tb i only can do one brick for 30 Tb is that correct? my objective is to host virtual machines for openstack in one brick mounted as glusterfs in the /var/lib/instances/nova, does the mount will show 120 Tb or just 30Tb?
23:22 y4m4 joined #gluster
23:30 elyograg ChiTo: the brick mounts will show 30TB.  The entire volume mount will show 120TB.  the largest single file you'll be able to put on the cluster is the available disk space on whichever brick will receive that file.
23:34 ChiTo elyograg: Oh perfect, so the max file, if the brick is empty, will be maximum 30 Tb. Thank you very much for your guidance!!
23:34 ChiTo but the volume will be 120 Tb because it is replicated with "2"
23:35 elyograg yes, you've got it.
23:35 ChiTo elyograg: :) Thank you!
23:36 ChiTo I have my bricks on top of LVM, i think there is no problem with that
23:36 ChiTo i can extend the LVM if i scale up vertically
23:37 elyograg ChiTo: that seems to be the way people recommend going.  LVM is very nice even if you have no plans to extend it, because you can use the same thing to mount it even if Linux decides to shuffle your scsi device names around.
23:37 ChiTo elyograg: I have a last question, i dont know if create the xfs brick with a size inode of 512 or 1024, i noticed it is recommendable to have a large value because of the metadata
23:38 elyograg ChiTo: If you know you'll never need anything in your file metadata other than gluster, you can probably go with an inode size of 512.  If you think you might ever use any other metadata, like ACLs, make it 1024.
23:39 ChiTo elyograg: I got it, thank you very much elyograg
23:39 elyograg that's not an expert recommendation, so if you ever get one of those, ignore me. ;)
23:43 aliguori joined #gluster

| Channels | #gluster index | Today | | Search | Google Search | Plain-Text | summary