Camelia, the Perl 6 bug

IRC log for #gluster, 2013-10-19

| Channels | #gluster index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:08 RobertLaptop joined #gluster
00:23 DV joined #gluster
00:28 plarsen joined #gluster
00:37 KORG joined #gluster
00:38 RobertLaptop joined #gluster
00:53 zerick joined #gluster
01:06 davidbierce joined #gluster
01:08 F^nor joined #gluster
01:46 _ilbot joined #gluster
01:46 Topic for #gluster is now Gluster Community - http://gluster.org | Patches - http://review.gluster.org/ | Developers go to #gluster-dev | Channel Logs - https://botbot.me/freenode/gluster/ & http://irclog.perlgeek.de/gluster/
01:50 dbruhn joined #gluster
01:50 dbruhn gah? all of my volume files were just over written by a yum update
01:51 dbruhn hmm maybe not
01:53 dbruhn does anyone have an issue where gluster peer status and other gluster commands will return connection failed, but if you keep running them they will finally return results
01:54 diegows joined #gluster
01:59 kevein joined #gluster
02:09 dbruhn does anyone know what it takes to upgrade from 3.3.1 to 3.3.2?
02:56 Skaag JoeJulian, I still get the same error
02:56 Skaag making the volume worked fine in 3.2.5
02:57 Skaag but now it keeps saying: volume create: ghome: failed: Glusterfs is not supported on brick: xgnt-vps-001:/ghomedata
03:11 JoeJulian Skaag: fpaste /var/log/glusterfs/etc-glusterfs-glusterd.vol.log from xgnt-vps-001
03:12 Skaag well the main operation that stands out is this one: Setting extended attributes failed, reason: Operation not permitted.
03:12 Skaag I found on google some people had similar issue with newer versions
03:12 Skaag but there is no proposed fix
03:13 JoeJulian The difference would be that the old version didn't check to see that it was going to fail first.
03:13 Skaag this is the full log: https://paste.fedoraproject.org/47917/
03:13 JoeJulian Why does /ghomedata not support extended attributes?
03:13 glusterbot Title: #47917 Fedora Project Pastebin (at paste.fedoraproject.org)
03:13 Skaag I don't know... I tried to chattr and lsattr and I can set/get extended attributes
03:14 dbruhn JoeJulian, I know you didn't want to spend your friday night helping people out? but i have an issue
03:14 Skaag it's an openvz container, maybe something's fishy about it
03:14 JoeJulian Hehe, I'm here... :D
03:14 dbruhn I have had a shit of a day, and one of my production volumes is down and won't mount
03:15 JoeJulian Skaag, Ah, that's it. openvz has some sort of funny restrictions on trusted.* and security.* extended attributes.
03:15 dbruhn it's RDMA, and i just upgraded to 3.3.2 this evening trying to fix it
03:16 JoeJulian Skaag: There is a setting for openvz to allow that, but I only know that by what other people have said.
03:16 JoeJulian dbruhn, logs please
03:16 dbruhn mnt log first?
03:16 JoeJulian yes
03:18 dbruhn http://pastie.org/pastes
03:18 glusterbot Title: Pastes - Pastie (at pastie.org)
03:19 JoeJulian Hehe, perhaps the specific one?
03:19 dbruhn [2013-10-19 03:07:08.695826] E [dht-common.c:1374:dht_lookup] 0-ENTV04EP-dht: Failed to get hashed subvol for /
03:19 dbruhn [2013-10-19 03:07:08.696065] E [dht-common.c:1374:dht_lookup] 0-ENTV04EP-dht: Failed to get hashed subvol for /
03:20 dbruhn is the first thing I see
03:20 dbruhn and this [2013-10-19 03:07:08.688458] E [afr-common.c:3722:afr_notify] 0-ENTV04EP-replicate-9: All subvolumes are down. Going offline until atleast one of them comes back up.
03:21 dbruhn there are some ib errors too, but I have tested everything and it's all talking without issue
03:24 dbruhn hmm just found this in the etc log
03:24 dbruhn E [rpcsvc.c:491:rpcsvc_handle_rpc_call] 0-glusterd: Request received from non-privileged port. Failing request
03:26 JoeJulian dbruhn: What distro is this?
03:26 dbruhn redhat 6.4
03:27 JoeJulian Do you have EPEL installed?
03:27 dbruhn yeah
03:28 JoeJulian If you want to paste things easily, "yum install fpaste". Then you can "fpaste /var/log/glusterfs/blah-blah-blah.log"
03:28 dbruhn ok
03:30 dbruhn installed
03:31 Skaag JoeJulian, yes, found this: https://bugzilla.redhat.com/show_bug.cgi?id=764245
03:31 glusterbot <http://goo.gl/QEXHDI> (at bugzilla.redhat.com)
03:31 glusterbot Bug 764245: low, medium, ---, vbellur, ASSIGNED , [FEAT] glusterfs requires CAP_SYS_ADMIN capability for "trusted" extended attributes - virtualization unfriendly
03:37 MrNaviPacho joined #gluster
03:38 kevein joined #gluster
03:43 dbruhn http://paste.fedoraproject.org/47918/38215419
03:43 glusterbot Title: #47918 Fedora Project Pastebin (at paste.fedoraproject.org)
03:50 dbruhn I am having a really weird deal here too, where if I keep spamming gluster peer status at the command line, it won't produce anything, and then after a handful of times it will
03:51 dbruhn and apparently the mount command is exhibiting the same behavior?
03:53 dbruhn http://fpaste.org/47920/38215480/
03:53 glusterbot Title: #47920 Fedora Project Pastebin (at fpaste.org)
04:14 JoeJulian Back.... wife wanted me to start a fire.
04:14 dbruhn no worries
04:14 dbruhn all sorts of weird behavior here
04:16 JoeJulian gluster volume info ENTV04EP | fpaste
04:16 dbruhn http://paste.fedoraproject.org/47924/82156183
04:16 glusterbot Title: #47924 Fedora Project Pastebin (at paste.fedoraproject.org)
04:17 dbruhn http://fpaste.org/47925/82156215/
04:17 glusterbot Title: #47925 Fedora Project Pastebin (at fpaste.org)
04:17 dbruhn if you look at this one, I am getting this behavior with every single gluster command
04:17 dbruhn http://fpaste.org/47920/38215480/
04:17 glusterbot Title: #47920 Fedora Project Pastebin (at fpaste.org)
04:18 dbruhn eventually if I keep spamming the commands they will work
04:25 JoeJulian That sounds like something is spamming all the ports < 1024 so glusterd's refusing untrusted connections.
04:26 dbruhn the weird thing is I have another system sitting here on the exact same hardware minus sata/sas
04:26 dbruhn even sharing the same IB switch
04:26 dbruhn and the other system isn't having a problem at all
04:29 JoeJulian Is one a server and the other not?
04:30 dbruhn one is a 6 server 12 brick dht x2 replication system
04:30 dbruhn and the other is a 10 server 20 brick dht x2 replication system
04:30 dbruhn all physical hardware
04:35 JoeJulian The question is more about one client that works vs one client that doesn't.
04:35 hagarth joined #gluster
04:36 dbruhn the servers are all clients to the volume they are attached to
04:36 dbruhn no other clients
04:36 JoeJulian Do they all mount from themselves?
04:36 dbruhn I do have the mount command running back to the server that's mounting
04:37 JoeJulian Just for giggles, try mounting using some other server.
04:38 dbruhn same behavior
04:38 dbruhn and just another point of interest no matter what the gluster command it takes spamming it to make it work
04:39 JoeJulian /var/log/glusterfs/etc-glusterfs-glusterd.vol.log probably has a lot of "insecure" something...
04:40 dbruhn Request received from non-privileged port. Failing request
04:40 dbruhn you mean like that
04:41 JoeJulian Yeah, that's the one.
04:41 JoeJulian netstat might tell you what's hogging all the ports.
04:41 dbruhn I found this, haven't tried it yet
04:41 dbruhn http://www.gluster.org/pipermail/glu​ster-users/2012-December/035047.html
04:41 glusterbot <http://goo.gl/TbBkN> (at www.gluster.org)
04:43 JoeJulian You can probably get through the weekend by "gluster volume set ENTV04EP server.allow-insecure on"
04:44 dbruhn volume set unsuccessful
04:44 JoeJulian Same reason. Spam it 'till it's successful.
04:44 JoeJulian Or do it from another server.
04:44 RameshN joined #gluster
04:45 dbruhn when it can't connect it says "Connection failed. Please check if gluster daemon is operational."
04:46 dbruhn when it gets through it says "Set volume unsuccessful"
04:46 dbruhn I have not been able to stop and start this volume for months now either
04:46 JoeJulian Meh
04:46 JoeJulian @meh
04:46 glusterbot JoeJulian: I'm not happy about it either
04:46 dbruhn lol
04:47 dbruhn when does that .vol file get overwritten?
04:47 dbruhn i know the other files get overwritten all the time and it's not advisable to change the files manually
04:47 JoeJulian Good question.
04:51 mohankumar joined #gluster
04:51 JoeJulian Check /var/log/glusterfs/etc-glusterfs-glusterd.vol.log to see why it says it's failing. It may just tell you which server failed and you'll have to check that log on that server.
04:53 dbruhn this looks odd
04:53 dbruhn http://fpaste.org/47928/82158416/
04:53 glusterbot Title: #47928 Fedora Project Pastebin (at fpaste.org)
04:57 dbruhn doesn't belong to the cluster. Ignoring request.
04:58 dbruhn it's saying that about my first server
04:59 dbruhn and sure enough there isn't a file for it in the peer directory /var/lib/glusterd/peer
05:04 johnbot11 joined #gluster
05:08 dbruhn ugh.. my uuid's are all over the place?
05:09 JoeJulian ?
05:09 dbruhn like mismatches between servers
05:09 dbruhn server2's peer file for server1 glusterd.info file didn't match
05:09 dbruhn and it seems to be the case for a bunch of them
05:09 JoeJulian wierd
05:10 dbruhn really weird
05:10 dbruhn fixed server 1 on all the nodes and the commands are now working on server1 without any issues
05:10 dbruhn guess I need to go to each server and check them, and fix them
05:12 MrNaviPacho joined #gluster
05:12 JoeJulian I wish we knew how that happened, but this sounds like a bug that needs fixed. Can you file a bug report on that?
05:12 glusterbot http://goo.gl/UUuCq
05:12 dbruhn yeah, I think I know how it happened
05:12 dbruhn 8 months ago or so my /var file system filled up
05:13 dbruhn and someone helped me fix it
05:13 JoeJulian There should be some way of recognizing that the peer uuids mismatch.
05:13 JoeJulian Oooh, I see where that's going. :(
05:13 dbruhn I seem to remember something about treating one peer as an authority
05:13 dbruhn I had one working server
05:13 dbruhn yeah
05:13 dbruhn I will file a bug after I fix it
05:13 glusterbot http://goo.gl/UUuCq
05:14 dbruhn thanks for helping me look at this, I was going nuts after 8 hours of beating my head against it
05:15 dbruhn it's weird that it's been working at all for this long
05:17 JoeJulian It was probably working because of some bug that got fixed. :D
05:22 ppai joined #gluster
05:28 hateya joined #gluster
05:41 JoeJulian dbruhn: by the way, if you manage with puppet, purpleidea's puppet module would have been able to help ensure the uuids are consistent if anything like that ever happens again.
05:49 mohankumar joined #gluster
05:50 ProT-0-TypE joined #gluster
05:55 dbruhn kk
06:10 KORG|3 joined #gluster
06:33 ababu joined #gluster
06:40 dbruhn You still there by chance?
06:43 m0zes_ joined #gluster
06:47 ProT-0-TypE joined #gluster
06:48 m0zes joined #gluster
06:51 vynt joined #gluster
06:59 hateya joined #gluster
07:06 tg2 joined #gluster
07:28 ccha joined #gluster
07:32 wgao joined #gluster
07:33 MrNaviPacho joined #gluster
07:43 rotbeard joined #gluster
07:44 m0zes_ joined #gluster
07:59 hagarth joined #gluster
09:02 jkroon joined #gluster
09:03 jkroon hi guys, i chatted here earlier about glob() being extremely slow on top of glusterfs
09:03 jkroon as in a factor of around 12x slower
09:04 jkroon the suggestion was made to upgrade to glusterfs 3.4 and retest, which I've done now, with little to no difference.
09:04 jkroon any ideas?
09:16 Remco Lots of files?
09:19 Remco I'd say it's slow because of http://stackoverflow.com/questions/16461​510/why-does-glob-lstat-matching-entries
09:19 glusterbot <http://goo.gl/i31iLQ> (at stackoverflow.com)
09:23 jkroon hmm, that would cause a self-heal check on pretty much every file
09:23 Remco Yep
09:23 jkroon and yes, it does contain a lot of files.
09:24 jkroon if 7500 / folder can be considered a lot
09:24 Remco For glusterfs that is a lot I'd say
09:25 jkroon shees, ok
09:26 jkroon what else is there in terms of redundant distributed file systems that makes sense then for archive-type storage
09:26 jkroon that is somewhat searchable
09:26 Remco I just know of drbd
09:27 Remco You could switch to NFS instead of native glusterfs
09:28 Remco Since it can do local caching, it'll probably be a lot faster
09:28 Remco (of course it has its own downsides, but that's for you to test and decide)
09:29 jkroon no, hell no, i'm not going the drbd route again
09:29 jkroon ok, so run glusterfs with nfs exports and mount the nfs exports instead?
09:29 jkroon the problem with raw NFS is that if the NFS server goes down ...
09:30 Remco No, glusterfs already does NFS
09:31 Remco So you should be able to just switch directly without any additional configuration
09:31 jkroon yea, i know, what i'm saying, mount the gluster share as nfs ... sorry if i'm not clear.
09:31 jkroon raw NFS == bad idea in this case.
09:31 jkroon what's the downsides of doing glusternfs ? if any?
09:31 Remco NFS can do failover too I think
09:32 Remco Different locking support is the biggest one I think
09:32 Remco Next to not having the automatic failover, but you can do that yourself
09:33 jkroon well, it's simple currently, two servers both running glusterfs with 500G allocated on storage for glusterfs bricks, normal 1x2 distribute/replicate setup.
09:33 jkroon then I fuse-mount the glusterfs's against the local machine's IP (no ucarp/keepalive required)
09:34 jkroon so I reckon NFS-mount against the local instance should be just fine.
09:34 jkroon for remote clients something like ucarp would be beneficial (but since it's udp may require some smart iptables + routing trickery)
09:36 Remco I didn't do enough playing around with glusterfs to know about all the special setups. I just did used a different mount command and it worked
09:37 dbruhn any idea what the hell to do about this
09:37 dbruhn [posix-handle.c:387:posix_handle_init] 0-ENTV04EP-posix: Different dirs /var/brick14 (64/2050) != /var/brick14/.glusterfs/00/00/000​00000-0000-0000-0000-000000000001 (1125126/2050)
09:39 jkroon ouch, that doesn't look good.  should be GUID?
09:39 dbruhn not sure what it's supposed to be, but the brick won't start because of it
09:40 jkroon hahaha, time to rtfs then and find posix-handle.c, go to line 387 and scope out the fine source.
09:42 dbruhn well I have no idea what that means? and I am about 3 hours shy of 24 hours of working on this damn system
09:42 dbruhn of course it's a production machine
09:43 jkroon always is ...
09:43 jkroon gluster fs version?
09:43 dbruhn well mid all of this shit I have upgraded from 3.3.1 to 3.3.2
09:43 jkroon and yea, i helped move 13 servers last night, and another 9 the night before, average 3 hours sleep/night, so i know your feeling.
09:44 jkroon ok, just downloading the source quick
09:46 dbruhn this has been 2 weeks of 7am to 12am/2am from various things
09:46 dbruhn this is just the latest mess
09:46 jkroon lol, sucks to be you :p
09:46 dbruhn hmm, so it seems every other location links that to ../../..
09:46 dbruhn [root@ENTSNV04002EP peers]# ls -lha /var/brick02/.glusterfs/00/00/000​00000-0000-0000-0000-000000000001
09:46 dbruhn lrwxrwxrwx 1 root root 8 Apr  8  2013 /var/brick02/.glusterfs/00/00/000​00000-0000-0000-0000-000000000001 -> ../../..
09:46 dbruhn [root@ENTSNV04002EP peers]# ls -lha /var/brick03/.glusterfs/00/00/000​00000-0000-0000-0000-000000000001
09:46 dbruhn lrwxrwxrwx 1 root root 8 Apr  8  2013 /var/brick03/.glusterfs/00/00/000​00000-0000-0000-0000-000000000001 -> ../../..
09:48 jkroon perhaps you need to re-init the brick, but I've no idea how safe it is to just do gluster volume start <VOLNAME> force
09:48 jkroon there is also a reset which I was told to run before that, but as I say, I do not understand all the implications
09:48 dbruhn reset?
09:48 jkroon volume reset <VOLNAME> [option] [force] - reset all the reconfigured options
09:50 jkroon ok, that log entry makes more sense after looking at the code, but not sufficiently for me yet to explain what's wrong
09:50 jkroon it expects the st_ino and st_dev values from stat()s on two different files/folders to be the same, and it's not it outputs that
09:50 dbruhn I am digging through bricks right now
09:51 dbruhn I have a second system that is running well here to look at too
09:51 dbruhn all of my UUID's on this system were hosed from a previous repair job
09:52 dbruhn which was causing issues I believe
09:52 jkroon hmm, it's saying the symlink is wrong.
09:52 dbruhn now it's clean up
09:52 jkroon the st_ino and st_dev values for stat() on those two paths results in different values
09:52 jkroon and it's moaning about brick14, not brick02
09:53 jkroon so perhaps just look what those symlinks are in brick14
09:53 dbruhn yep
09:53 dbruhn I was just showing the other bricks all had something different than brick14
09:54 jkroon what is it on brick14?
09:54 dneary joined #gluster
09:54 Remco So it's a split brain?
09:55 jkroon don't think so
09:55 jkroon split brain won't stop gluster(fs)?d from starting up I don't think
09:55 jkroon at least, not in my experience.
09:55 Remco http://www.joejulian.name/blog/fix​ing-split-brain-with-glusterfs-33/
09:55 glusterbot <http://goo.gl/FzjC6> (at www.joejulian.name)
09:55 Remco Can be a single file/dir
09:55 jkroon it just results in EIO being returned to the client
09:56 jkroon which is not what dbruhn is reporting
09:56 dbruhn yeah not a split brain issue
09:56 dbruhn so on my good system it just links back to /var/brick18/.glusterfs/00/00/000​00000-0000-0000-0000-000000000001 -> ../../..
09:56 ProT-0-TypE joined #gluster
09:56 dbruhn on every brick
09:57 jkroon that looks right, i'm betting on the bad system /var/brick14/.glusterfs/00/00/0000....1 links elsewhere other than ../../..
09:58 dbruhn yep
09:58 dbruhn do you know if these are hard links or symlinks
09:58 jkroon you can't hard-link folders
09:58 dbruhn oh duh
09:58 jkroon but merely the fact that it gives you  path when you ls it makes it a symlink :)
09:59 jkroon # mount -t nfs -o nolock localhost:/stage-cfg /tmp/r <-- nfs mount is hanging :(
10:04 dbruhn ugh 5am?.
10:04 jkroon don't worry, well, i've got to go
10:04 dbruhn I think I got it
10:04 dbruhn thanks for the help
10:04 jkroon but i'm betting if you fix that symlink the brick will start up
10:05 dbruhn i did and it did
10:05 dbruhn and my logs stopped growing at 1gb every 10 min
10:05 dbruhn lol
10:05 jkroon mount.nfs: requested NFS version or transport protocol is not supported :(
10:05 glusterbot jkroon: make sure your volume is started. If you changed nfs.disable, restarting your volume is known to work.
10:06 dbruhn need 4?
10:06 Remco You need to use TCP
10:06 jkroon don't think gluster supports v4 (kerberos)
10:06 jkroon remco, i'm passing vers=3,tcp to mount
10:06 dbruhn in 3.3.1/2 it didn't
10:07 jkroon NFS Server on 192.168.1.1                               2049    Y       6418
10:07 Remco Is that your own NFS server?
10:08 Remco Or the built-in gluster one?
10:08 dbruhn remco, gluster has a built in hfs server
10:08 jkroon built-in gluster
10:08 Remco They can't run at the same time
10:08 dbruhn oh yeah
10:08 dbruhn good point
10:08 jkroon that line is from gluster volume status
10:08 jkroon ok, changing the IP causes things to hang ...
10:09 jkroon no firewall
10:09 dbruhn selinux?
10:09 jkroon nope
10:10 ababu joined #gluster
10:10 dbruhn disable nfs on the server?
10:10 dbruhn the os provided nfs
10:10 jkroon no os provided nfs server is even installed
10:10 jkroon got the client tools only
10:10 dbruhn wht distro
10:10 jkroon gentoo
10:11 Remco Do you have nfs-common installed on both the server and the clients?
10:11 jkroon oh i lie, the server comes as part of nfs-utils
10:11 Remco Well, that would break stuff :P
10:11 jkroon it's not running.
10:11 jkroon port 2049 belongs to gluster
10:12 Remco Doesn't matter I think
10:12 jkroon ?!
10:12 jkroon ok, that one i'd like to understand
10:12 Remco The rpc stuff gets in the way IIRC
10:12 jkroon rpcbind is now shut down.
10:12 jkroon and that immediately causes the mount command to hang
10:12 jkroon with rpcbind running it moans about version stuff
10:13 jkroon but you also can't mount without nolock without some kind of rpc
10:13 jkroon anyway, that's something I'll have to sort out at a later stage...
10:15 Remco If only one of the experts was here, would have been so much faster
10:15 monotek joined #gluster
10:16 jkroon well, for where I can test the performance isn't an issue and at the other site it's been that way for a while now, so seriously, another couple of days won't hurt.  but i want my money before I tell the client to take his attitude and shove it.
10:17 jkroon 12:17:31.518382 IP 192.168.1.2.46083 > 192.168.1.1.111: Flags [S], seq 2173282620, win 14600, options [mss 1460,sackOK,TS val 1050963663 ecr 0,nop,wscale 6], length 0
10:17 jkroon 12:17:31.518419 IP 192.168.1.1.111 > 192.168.1.2.46083: Flags [R.], seq 0, ack 2173282621, win 0, length 0
10:18 jkroon got it
10:18 jkroon you need to first start rpcbind *before* you start glusterd such that the nfs server can register with rpcbind
10:19 jkroon ooh, no, this feels even worse than native mount ...
10:19 Remco That makes sense. But why didn't gluster start it itself?
10:19 jkroon because when glusterd started up a couple of days ago rpcbind wasn't installed yet :p
10:19 Remco Ah :D
10:20 jkroon and because the maintainer for gentoo didn't think to put need rpcbind in the init script, which I'm not going to do either, i'm going to make it after, because otherwise if rpcbind isn't installed
10:21 Remco The first time the glob will be slow, but since nfs caches, it should be a lot faster afterwards
10:21 jkroon ok, nowhere *near* ext4 on local disk, but MUCH better (around 5x faster than gluster native fuse)
10:21 jkroon that's exactly what I'm seeing yes
10:22 Remco NFS has lots of tunables, so you might be able to get it even faster
10:23 jkroon 0m7.962s vs 0m6.445s on initial on nfs vs native fuse.
10:23 jkroon 0m5.202s vs 0m0.979s thereafter nfs vs fuse
10:24 jkroon so gluster native seems to still be faster on full ls.
10:24 dbruhn jkroon, RDMA, or TCP?
10:24 Remco o.O
10:24 jkroon TCP
10:24 dbruhn 1gb/10gb?
10:24 jkroon 1gbit
10:24 jkroon in the glob() case though there is no comparison
10:25 jkroon 0.002s for nfs picking 12 files out of 10k vs 0.513 for gluster fuse
10:25 jkroon so the nfs caching makes a HUGE difference.
10:25 jkroon in my use-case a full ls is unlikely to happen often and it's mostly glob() calls, so I'm going to test that this afternoon.
10:26 dbruhn are you using dht/replicated?
10:27 Remco [11:33] < jkroon> well, it's simple currently, two servers both running glusterfs with 500G allocated on storage for glusterfs bricks, normal 1x2 distribute/replicate setup.
10:28 jkroon dbruhn, what's the difference?
10:28 Remco Lunch time, bbl
10:28 dbruhn well you can use them together
10:28 jkroon Type: Replicate
10:28 dbruhn but dht is a cluster with no replication
10:28 dbruhn ahh ok
10:28 jkroon no, need replication :)
10:28 jkroon this is all about redundancy ...
10:29 dbruhn the nfs server makes the replication async if I remember right
10:29 dbruhn I could be wrong about that though
10:29 dbruhn I am running dht and replication
10:29 dbruhn replication with the native gluster client speeds up reads once you start scaling out
10:31 dbruhn gross? it's 37 degrees out, and raining? lovely 40 min motorcycle ride home to wrap my night up
10:31 dbruhn well gents i need to get cold/wet and then sleep
10:32 dbruhn thanks for the help
10:39 khushildep joined #gluster
11:43 dbruhn joined #gluster
11:48 jmaes joined #gluster
11:49 jmaes joined #gluster
11:50 jmaes Top of the morning to you all - hate to ask - but just wanted to confirm - GlusterFS (client to mount) really isnt a very good option on SLES 11.2 right now is it?
11:52 jmaes I've got a the group of RHEL machines hosting the pool just fine - but have a large number of SLES boxes that need to access it - and was hoping to use the gluster client - but after several hours - i am not seeing any RPM's and the build env on SLES seems significantly different enough to make a clean build not a valid option
11:55 Remco If that version of SLES doesn't have the required packages, I'd say use the NFS mount if you really need it
11:56 jmaes That's the direction I am headed now - just wanted to make sure that I wasn't missing something.  ie - some magical repo of SLES rpm's that are only discoverable via a blood sacrifice of a sysadmin
11:58 Remco http://www.gluster.org/pipermail/gl​uster-users/2012-August/033928.html
11:58 glusterbot <http://goo.gl/XcPkkS> (at www.gluster.org)
11:58 Remco Someone there compiled it in 11.2, so it should be possible
11:59 Remco (or it was a package from some unknown location)
12:00 Remco But that's as much as I can find about gluster on SLES 11.2
12:00 kopke joined #gluster
12:01 jmaes ya - I spent most of yesterday and today on it - seems like a little used combination.  The servers I've built out our 3.4 - any special concerns with connecting to them with a 3.3 client (assuming I can get 3.3 to compile - 3.4 seems especially angry on SLES 11.2)
12:01 jmaes are **
12:03 Remco I have no idea, sorry
12:03 jmaes no problem.  Thank you all the same.
12:08 ababu joined #gluster
12:09 cyberbootje joined #gluster
12:15 jmaes fingers crossed - looks like I was able to get 3.3 to compile on SLES 11.2
12:15 m0zes joined #gluster
12:19 jmaes very silly question - but I am not seeing it in the Quick Start guide or in the documentation - when I mount a share via a client using gluster - it seems like I am asking it to mount a particular node - not a higher level "object" that will survive the failure of the particular node that I happen to mount - ie, my goal is to HA across the fileshare service regaurdless of the availability  of a single node.
12:21 jmaes if anyone could point to me to correct manual - more than happy to rtfm - just cant find it
12:30 foster joined #gluster
12:31 Remco You point it at a node where it fetches the config
12:31 jmaes gotcha - beautiful
12:32 Remco If you have it connect to a round-robin DNS name, you should always get a connection
12:33 Remco I do remember reading about how that works somewhere, but can't find it now
12:35 jmaes even better - I am all AWS - so a nice DNS HA RR on a higher level name is perfect
12:35 jmaes thank you
12:49 samppah jmaes: also there is backupvolfile-server option which you can use to set another server for mounting.. ie. mount -o backupvoplfile-server=server2.domain.tld
12:51 diegows joined #gluster
12:51 jmaes Samppah  - thank you
12:52 jmaes on the client - I am trying to mount the first node int the server - and getting this "Mount failed. Please check the log file for more details.
12:52 jmaes "
12:52 jmaes in the logs - I am seeing the following
12:52 jmaes [2013-10-19 12:51:27.150132] I [glusterfsd.c:1670:main] 0-/usr/local/sbin/glusterfs: Started running /usr/local/sbin/glusterfs version 3.3.2
12:52 jmaes [2013-10-19 12:51:27.166884] E [glusterfsd-mgmt.c:1550:mgmt_getspec_cbk] 0-glusterfs: failed to get the 'volume file' from server
12:52 jmaes [2013-10-19 12:51:27.166951] E [glusterfsd-mgmt.c:1618:mgmt_getspec_cbk] 0-mgmt: failed to fetch volume file (key:/gv0)
12:52 jmaes [2013-10-19 12:51:27.167268] W [glusterfsd.c:838:cleanup_and_exit] (-->/usr/local/lib/libgfrpc.​so.0(rpc_clnt_notify+0x129) [0x7f43b7e38399] (-->/usr/local/lib/libgfrpc.so​.0(rpc_clnt_handle_reply+0xa4) [0x7f43b7e37bb4] (-->/usr/local/sbin/gluster​fs(mgmt_getspec_cbk+0x30b) [0x40d0bb]))) 0-: received signum (0), shutting down
12:52 jmaes [2013-10-19 12:51:27.167312] I [fuse-bridge.c:4655:fini] 0-fuse: Unmounting '/mnt/gv0/'.
12:53 jmaes seeing nothing in the server logs
12:54 samppah hmm.. i just read bit of the earlier conversations. are you trying to connect version 3.3 client to 3.4 server?
12:55 jmaes I am
12:55 jmaes good chance that is a huge no go?
12:57 Remco http://vbellur.wordpress.com/2013/​07/15/upgrading-to-glusterfs-3-4/
12:57 glusterbot <http://goo.gl/SXX7P> (at vbellur.wordpress.com)
12:57 Remco Says it's compatible
12:57 samppah i'm not sure but iirc someone said few days ago that he was having problems with connect 3.3 client to 3.4 server
12:58 ProT-0-TypE joined #gluster
12:59 Remco "If a volume option that corresponds to 3.4 is enabled, then 3.3 clients cannot mount the volume."
13:00 Remco Besides that, no idea
13:01 jmaes ya - not sure - info doesnt tell anything about a "version" on a volume - ie
13:01 jmaes gluster> volume info gv0
13:01 jmaes Volume Name: gv0
13:01 jmaes Type: Replicate
13:01 jmaes Volume ID: cd8c6964-07af-4baf-9949-b38de979f6d5
13:02 jmaes Status: Started
13:02 jmaes Number of Bricks: 1 x 2 = 2
13:02 jmaes Transport-type: tcp
13:02 jmaes Bricks:
13:02 jmaes Brick1: fileshare01-i.infopluswms.com:/export/brick1/sdb1
13:02 jmaes Brick2: fileshare02-i.infopluswms.com:/export/brick1/sdb1
13:04 jmaes let me go back to looking at getting 3.4 to comile on SLES 11.2
13:04 jmaes just a random question - almost everyone running on RHEL or some version of that now days?
13:04 jmaes ie, SLES seems to be in the far minority
13:07 samppah can't remember when was the last time i heard someone speaking about SLES here :)
13:07 samppah btw, have you tried gluster rpms?
13:07 jmaes ya - tried all the RPM's I could find
13:08 jmaes they worked great on my RHEL boxes
13:08 jmaes which are the fileshares for this
13:08 jmaes but all of my clients are SLES boxes
13:08 jmaes none of the RPM's i could find worked for SLES
13:10 squizzi joined #gluster
13:13 hagarth jmaes: can you try mounting after disabling open-behind translator?
13:13 hagarth jmaes: you can disable that through - volume set gv0 open-behind off
13:14 hagarth have to run now .. will bbl
13:15 * Remco runs debian
13:15 Remco Work servers are on centos
13:17 squizzi left #gluster
13:20 jmaes trying now
13:21 jmaes that worked!
13:22 jmaes can anyone clue me in on what "volume set gv0 open-behind off" does / did ?
13:25 Remco It disables a feature introduced with 3.4
13:25 Remco (which is all I can say about it)
13:27 jmaes beautiful - thank you all very much - making a long weekend a little shorter for me
13:54 ababu joined #gluster
13:55 shylesh joined #gluster
14:07 jmaes one last question I hope - trying to get the glusterfs client working through the filewalls - I have been able to confirm everything works when I allow in all traffic - but trying to narrow that down - here are the ports that I am allowing - but still getting trouble
14:07 jmaes 24000 - 24999TCP10.10.0.0/16ALLOW
14:07 jmaes 111UDP10.10.0.0/16ALLOW
14:07 jmaes 24000 - 24999UDP10.10.0.0/16ALLOW
14:07 jmaes 38465 - 38467TCP10.10.0.0/16ALLOW
14:07 jmaes 38465 - 38467UDP10.10.0.0/16ALLOW
14:08 jmaes and 111 TCP from the same mask
14:08 jmaes those are all the ports that I am seeing in the documentation - are there others that I am missing
14:08 jmaes or?
14:09 jmaes looks like maybe some ICMP traffic?
14:13 hagarth joined #gluster
14:32 mohankumar joined #gluster
14:59 Remco @ports
14:59 glusterbot Remco: glusterd's management port is 24007/tcp and 24008/tcp if you use rdma. Bricks (glusterfsd) use 24009 & up for <3.4 and 49152 & up for 3.4. (Deleted volumes do not reset this counter.) Additionally it will listen on 38465-38467/tcp for nfs, also 38468 for NLM since 3.3.0. NFS also depends on rpcbind/portmap on port 111 and 2049 since 3.4.
14:59 Remco jmaes: ^^
14:59 ProT-0-TypE joined #gluster
15:30 ababu joined #gluster
15:48 davinder joined #gluster
16:25 bennyturns joined #gluster
16:26 ababu joined #gluster
16:27 ProT-0-TypE joined #gluster
16:37 ProT-0-TypE joined #gluster
17:04 anands joined #gluster
17:10 diegows joined #gluster
17:11 samkottler joined #gluster
17:34 ProT-0-TypE joined #gluster
17:34 VerboEse joined #gluster
17:53 hagarth joined #gluster
17:57 vipulnayyar joined #gluster
17:58 vipulnayyar left #gluster
18:10 ProT-0-TypE joined #gluster
18:34 ProT-0-TypE joined #gluster
18:37 kr1ss joined #gluster
19:34 anands joined #gluster
20:36 jmeeuwen_ joined #gluster
20:38 FooBar joined #gluster
20:53 kbsingh joined #gluster
21:32 diegows joined #gluster
21:33 khushildep joined #gluster
21:46 dbruhn joined #gluster
22:40 StarBeast joined #gluster
23:10 Elico joined #gluster
23:11 Elico I was wondering if there is an iso that comes with gluster and some interface??

| Channels | #gluster index | Today | | Search | Google Search | Plain-Text | summary