Camelia, the Perl 6 bug

IRC log for #gluster, 2013-05-15

| Channels | #gluster index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:04 MrNaviPacho joined #gluster
00:10 jclift joined #gluster
00:10 yinyin joined #gluster
00:24 hchiramm__ joined #gluster
00:28 DEac- joined #gluster
00:51 andrewjs1edge joined #gluster
00:55 robos joined #gluster
01:03 yinyin joined #gluster
01:11 kevein joined #gluster
01:14 Goatbert joined #gluster
01:44 majeff joined #gluster
01:47 bharata joined #gluster
01:51 bala joined #gluster
01:57 ancaster joined #gluster
02:13 atrius_ joined #gluster
02:17 chirino joined #gluster
02:24 nickw joined #gluster
02:33 lalatenduM joined #gluster
02:48 vex I have two gluster nodes - and for some reason all the network traffic seems to be going to one node and hardly anything to the other
02:49 vex is this how it should be?
02:49 jclift "all network traffic" - Is that from several clients, or just from one client?
02:49 vex lots of clients
02:49 kshlm joined #gluster
02:50 vex i'm talking about seeing lots of I/O on one node and almost none on the other
02:50 jclift k, that's weird then
02:50 majeff joined #gluster
02:50 vex a number of volumes.. and bricks spread over both nodes equally
02:51 * jclift doesn't have enough depth with stuff yet to really offer assistance other that "that sounds weird then" though. :(
03:14 majeff joined #gluster
03:18 aravindavk joined #gluster
03:28 aravindavk joined #gluster
03:53 aravindavk joined #gluster
04:02 anand joined #gluster
04:03 majeff left #gluster
04:13 saurabh joined #gluster
04:20 nickw joined #gluster
04:23 yinyin joined #gluster
04:28 nickw joined #gluster
04:29 shylesh joined #gluster
04:32 aravindavk joined #gluster
04:36 lalatenduM joined #gluster
04:38 ngoswami joined #gluster
04:41 hagarth joined #gluster
04:57 lalatenduM joined #gluster
04:57 flrichar joined #gluster
05:03 coagen joined #gluster
05:03 tjikkun joined #gluster
05:03 NuxRo joined #gluster
05:03 bulde joined #gluster
05:03 mynameisbruce__ joined #gluster
05:03 theron joined #gluster
05:03 eightyeight joined #gluster
05:03 SteveCoo1ing joined #gluster
05:03 sjoeboo_ joined #gluster
05:03 MinhP_ joined #gluster
05:03 dxd828 joined #gluster
05:03 xavih joined #gluster
05:03 wgao__ joined #gluster
05:03 NeatBasis joined #gluster
05:03 haakon_ joined #gluster
05:04 pithagorians joined #gluster
05:05 sgowda joined #gluster
05:09 kshlm joined #gluster
05:18 harish joined #gluster
05:18 bala joined #gluster
05:23 kshlm joined #gluster
05:23 flrichar joined #gluster
05:25 yinyin joined #gluster
05:31 bala joined #gluster
05:31 vpshastry joined #gluster
05:34 raghu joined #gluster
05:39 deepakcs joined #gluster
05:55 bharata_ joined #gluster
05:58 harish joined #gluster
06:01 vshankar joined #gluster
06:10 aravindavk joined #gluster
06:10 guigui1 joined #gluster
06:14 ricky-ticky joined #gluster
06:24 jtux joined #gluster
06:30 vpshastry1 joined #gluster
06:34 ollivera_ joined #gluster
06:36 badone_ joined #gluster
06:38 glusterbot New news from newglusterbugs: [Bug 963081] Gluster cli for `volume info --xml` return wrong opRet <http://goo.gl/fEci4>
06:48 aravindavk joined #gluster
06:51 bala joined #gluster
06:56 anand joined #gluster
07:16 andreask joined #gluster
07:30 redsolar joined #gluster
07:36 StarBeast joined #gluster
07:36 hchiramm__ joined #gluster
07:37 aravindavk joined #gluster
07:39 pithagorians joined #gluster
07:41 mohankumar joined #gluster
07:45 vpshastry1 joined #gluster
07:48 tshm_ Hi! I'm going to use a script that's supposed to trigger self-heal, as a cron job or similar, using
07:48 tshm_ find /storage -noleaf -print0 | xargs --null stat
07:48 tshm_ , which outputs information on all files. Now, is there any way to run a command which outputs whether something went wrong while trying to self-heal, such as a split-brain or other errors?
07:49 NuxRo tshm_: not as far as I know, but could be wrong. why are you using that cron job, what version are you on? Recent glusterfs comes with its own self-heal daemon
07:51 tshm_ GlusterFS 3.1.3 (or may be 3.1.1, not entirely sure). There is currently a decision on not upgrading, for various reasons.
07:51 karoshi joined #gluster
07:52 tshm_ Plus, unfortunately, that we run a bit of a "custom" setup. We do have a log that outputs debug information when an error occurs, so an idea would be to parse that, but the script would be so much more complicated.
07:53 tshm_ Don't know whether that's a standard Gluster log, or something home-built.
07:53 tshm_ such as "[2013-05-14 17:29:05.252105] E [afr-self-heal-data.c:645:afr_sh_data_fix] 0-replicate-0: Unable to self-heal contents of '/00140-test2/in11.dat' (possible split-brain). Please delete the file from all but the preferred subvolume."
07:54 bala joined #gluster
07:55 anand joined #gluster
07:57 aravindavk joined #gluster
07:58 hchiramm__ joined #gluster
08:01 jtux joined #gluster
08:06 NuxRo tshm_: looks like gluster-style log
08:08 majeff joined #gluster
08:13 tshm_ Okay, good. What generates this log?
08:13 tshm_ Or, rather... I'd love to find a way to "manually" produce such output.
08:13 tshm_ But I guess my best bet for the time being is using that already existing log, trying to parse that.
08:18 x4rlos joined #gluster
08:20 x4rlos Hi all. I'm running gluster 3.3-1 in a simple 2-node mirror with one brick each. One of the machines recently suffered from a powercut, and whilst it looked like when it was started that everything has come back online magically, a backup last night
08:20 x4rlos ran rather quicker than i expected.
08:21 x4rlos I checked this morning to see that the files had copied across to both bricks, and this is not the case :-/
08:21 x4rlos self heal processes say they are running.
08:22 vpshastry joined #gluster
08:24 mohankumar joined #gluster
08:27 x4rlos logs suggest that no attempt has been made to try and reconnect :-/
08:27 x4rlos How best can i trigger a self heal - anyone?
08:28 tshm_ ls -l   is a simple way of doing it
08:29 tshm_ a less simple way is
08:29 tshm_ find <storage_mount> -noleaf -print0 | xargs --null stat
08:29 tshm_ (as recommended on the Gluster homepage)
08:29 tshm_ The latter will give you a bunch of output, so you might want to redirect that.
08:30 x4rlos ahh. I tried: gluster volume heal database-archive
08:30 x4rlos Operation failed on 192.168.110.4
08:30 x4rlos maybe i specified the name wrongly though.
08:30 tshm_ Oh.
08:31 x4rlos ls -l is something i ran on the mountpoint to find my latest .tar - however it hasn;t triggered a self heal on that file :-/
08:31 x4rlos when i mean mountpoint, i mean the glustered mountpoint.
08:32 x4rlos should it work on a per-file basis?
08:36 x4rlos When I tried: gluster volume heal database-archive info - it shows the files that are missing correctly :-/
08:39 x4rlos I have restarted the gluster daemon on the 'broken' server - but that does not seem to have helped :-/
08:40 tshm_ Umm, sorry, I'm not proficient enough to answer those questions. Pretty much a beginner myself.
08:41 tshm_ I'm having some troubles with self-healing, myself.
08:43 x4rlos tshm_: No probs, thanks for trying :-) I am also on debian, and had many bugs and troubles :-) But i thought i had sorted most of them and applied to a couple of live servers. Only to find this :-)
08:57 lh joined #gluster
08:57 lh joined #gluster
09:09 rotbeard joined #gluster
09:09 hchiramm__ joined #gluster
09:14 shireesh joined #gluster
09:21 tziOm joined #gluster
09:22 x4rlos I think this may be a quick fix from JoeJulian :-)
09:23 pithagorians_ joined #gluster
09:51 x4rlos If anyone fancies taking a look: http://pastebin.com/vu0Xw1Uk :-) Much apprieciated if anyone knows what the problem may be :-)
09:52 glusterbot Please use http://fpaste.org or http://dpaste.org . pb has too many ads. Say @paste in channel for info about paste utils.
10:00 Nagilum_ x4rlos: kinda misses what the problem is, you say one is broken
10:00 Nagilum_ +only
10:02 x4rlos Oh. Hmm. Yeah, i should really expand. sorry.
10:06 puebele joined #gluster
10:07 karoshi joined #gluster
10:09 harish joined #gluster
10:11 puebele1 joined #gluster
10:14 edward1 joined #gluster
10:14 harish joined #gluster
10:17 hchiramm__ joined #gluster
10:18 x4rlos Maybe this post better explains the problem... http://pastebin.com/6fPGsV3p     (Sorry glusterbot :-))
10:18 glusterbot Please use http://fpaste.org or http://dpaste.org . pb has too many ads. Say @paste in channel for info about paste utils.
10:19 Nagilum_ x4rlos: I'd delete the files from the brick of the faulted server
10:19 Nagilum_ then do a "ls -lR" on the gfs
10:21 x4rlos I'll certainly give it a go. Would this not work leaving in files in place? ( it's 82 Gig of files :-))
10:22 Nagilum_ from you write it looks like gfs is unable to determine which file to use
10:22 Nagilum_ by deleting one you make the decision easy enough for gfs to decide :>
10:22 Nagilum_ you can test it first with just one file of course
10:24 x4rlos I had tried this before, (without R argument) and it did not work. :-/
10:24 Nagilum_ you deleted a broken file from one brick, then did a ls -l on it and was not healed?
10:24 Nagilum_ or what?
10:27 x4rlos It was a file that was not there. I will delete one and try just that one as you suggest.
10:27 x4rlos 1 sec.
10:29 x4rlos nope. That did not work :-(
10:29 Nagilum_ check your /var/log/glusterfs/glustershd.log
10:30 x4rlos I deleted a single file from the 'non-working' server that was present on both servers, and then from the client i ls -l the file and it listed it correctly. However it did not re-copy the file to the broken server.
10:30 x4rlos I will check logs.
10:31 puebele joined #gluster
10:32 Nagilum_ hmm, why does geo-replication insist on ssh when doing geo-replication between to local gfs mounts? :/
10:32 tziOm I have been searching and reading for some days now about optimizing gluster for small files, but documentation/performance tests on this issue is slim to none.. (setup is 6x brick servers with 4x4TB (1 brick each) rep 3, ctdb samba on each brick server)
10:34 tziOm I have not done much performance testing yet, but just a directory listing shows me there is some room for improvement...
10:34 Nagilum_ yeah, gfs is damm slow when it comes to readdir() and stat()
10:35 x4rlos Nagilum_: i see this in the logs from the working server:
10:35 x4rlos [2013-05-15 11:27:40.807801] E [afr-self-heald.c:418:_crawl_proceed] 0-database-archive-replicate-0: Stopping crawl as < 2 children are up
10:35 Nagilum_ x4rlos: what did you set cluster.quorum-type to?
10:36 tziOm I am planning to use cachefilesd (cifs) on clients to take some pressure off glusterfs, and also try negative-lookup translator... any more reading I should do? Is it not possible to do some local caching?
10:37 x4rlos Nagilum_: nothing explicitly. I have left this setting.
10:38 Nagilum_ x4rlos: not sure what the default is, I have this set to "auto"
10:39 glusterbot New news from newglusterbugs: [Bug 963176] G4S: there is no automounting for gluster volumes for UFO thus all REST request fails,PUT with 404 ,GET/HEAD with 503 <http://goo.gl/3c3we>
10:40 x4rlos how can i get this info? must be a gluster attribute list somewhere. the default is auto though i think.
10:40 Nagilum_ gluster volume info <gv>
10:40 Nagilum_ man gluster
10:41 Nagilum_ gluster volume set <gv> cluster.quorum-type auto
10:44 Nagilum_ hmm, odd, on RHEL6 there doesn't seem to be any manpage mentioning the "fsc" mount option for nfs
10:44 Nagilum_ it works just fine though
10:45 tziOm I think it was redhat that implemented it, so thats odd
10:46 Nagilum_ tziOm: maybe they keep a bit secret so they can have a small edge for their paying customers .-)
10:46 tziOm hehe,,
10:47 tziOm might be..
10:47 tziOm very strange that glusterfs does not have any caching..
10:47 Nagilum_ annyoing really
10:48 Nagilum_ I also looked at a few others
10:48 Nagilum_ Ceph
10:48 tziOm atleast directory structure should be possible to cache
10:48 Nagilum_ Xtreemfs
10:48 tziOm yeah, ceph is awesome, but cephfs is not ready
10:48 Nagilum_ at least on RHEL6 you're limited to the fuse mount which is quite unstable
10:49 Nagilum_ the cluster itself was stable when I tested it
10:49 tziOm yeah.. you could update your kernel, but its a fact that cephfs is not usable for production atm, but beeing worked on.
10:49 tziOm you also have moosefs
10:49 Nagilum_ don't know that one
10:50 hchiramm__ joined #gluster
10:50 tziOm havent looked at xtreemefs
10:50 tziOm what license?
10:50 Nagilum_ bsd
10:51 tziOm so they will close it down when it starts working
10:51 Nagilum_ ;)
10:51 Nagilum_ don't think so
10:51 Nagilum_ its funded by universities and othe rpublic entities
10:52 x4rlos Nagilum_: I was looking through the docs, and trying to understand what this option does. The docs also state that None is the default (so what mine is likely set as). If i set to auto - what will it actually attempt to do?
10:52 tziOm have you done any testing?
10:52 Nagilum_ yeah
10:52 Nagilum_ it's stable
10:52 tziOm pros/cons?
10:53 Nagilum_ at first it seemed faster that glusterfs
10:53 tziOm but?
10:53 Nagilum_ but when you start replicating to multiple nodes which all have the same xtfs mounted you notice there is a bottleneck somewhere
10:54 Nagilum_ also they currenty can only have one metadata server iirc
10:54 Nagilum_ so you have a SPOF
10:55 Nagilum_ otherwise I would it in the middle between glusterfs and Ceph
10:55 tziOm seems gluster would actually benefit from a metadata server..
10:55 Nagilum_ yeah, readdir() and stat() would benefit
10:55 Nagilum_ but changes need to be serialized
10:55 tziOm yeah..
10:56 tziOm atleast caching ability of directory structure and some multicast "directory changed" would be good
10:56 Nagilum_ true
10:57 Nagilum_ with 0mz sockets that would be relatively easy to build I think
10:57 Nagilum_ 0mq
10:57 Nagilum_ but I'm just devop, not dev .-)
10:58 NuxRo 3.4 supposedly knows SSL, but can't find any documents about it anywhere. pointers?
10:59 tziOm 3.4 people?
11:00 Nagilum_ x4rlos: Method used for quorum enforcement. "None" means no quorum enforcement, which is the historical behavior. "Auto" means quorum is set to be more than half of the bricks in a subvolume, or exactly half if that includes the first listed brick. "Fixed" means quorum is set to a value specified by cluster.quorum-count. If quorum is not met, then modifing operations such as write will fail with
11:00 Nagilum_ EROFS. This prevents most cases of "split brain" which result from conflicting writes to different bricks.
11:03 tziOm so boiled down, glusterfs is not ment for a) listing directory structure b) small files?
11:04 Nagilum_ tziOm: at least its not optimized for these cases
11:05 tziOm Nagilum_, I think what perhaps all clustered filesystems lack is some basic documentation on what use cases it has and what it should not be used for.
11:06 Nagilum_ tziOm: they seem to focus more around object storage these days it seems
11:06 tziOm Everyone does, and I dont like it!
11:06 * Nagilum_ neither
11:07 Nagilum_ for my use-case I switched to a local ext4 fs
11:07 tziOm my everyting depend on fs
11:07 Nagilum_ synced via rsync
11:07 tziOm my problem is big data
11:08 Nagilum_ luckily my fileset is split
11:08 Nagilum_ I have a big (size) set with big files
11:08 Nagilum_ and a set with many small files
11:08 Nagilum_ so I keep the small files on a local fs and only put the big set on a gfs
11:09 glusterbot New news from newglusterbugs: [Bug 962450] POSIX ACLs fail display / apply / set on NFSv3 mounted Gluster filesystems <http://goo.gl/nRrHg>
11:09 x4rlos Nagilum_: If i am honest, i don't quite understand what that really means. Should i mean it to say that if there are less than half bricks available then writes will not occur?
11:11 Nagilum_ x4rlos: just set it to auto then do the "ls -l " on the file again and see if it works
11:14 x4rlos Nagilum_: Whilst i could kiss you (it worked) - could this cause any undesired effects i could be aware of? :-)
11:15 x4rlos could/should
11:15 hagarth joined #gluster
11:16 Nagilum_ x4rlos: while it's healing there will be increase load and of course you only have one copy the files until its done
11:16 Nagilum_ but that should be obvious
11:16 x4rlos it seems to have finished already.
11:17 x4rlos (very fast).
11:17 x4rlos But why would this not be a default option? There must be a caveat to using this 'feature'
11:17 chirino joined #gluster
11:18 Nagilum_ its a newer setting that wasn't available in old versions
11:18 Nagilum_ and as a responsible software developer you don't change the default behaviour between versions
11:18 Nagilum_ (within reason)
11:19 yinyin_ joined #gluster
11:20 aravindavk joined #gluster
11:21 tziOm Nagilum_, is it web you are using gl for?
11:21 Nagilum_ yep
11:22 tziOm big deployment? many sites?
11:23 x4rlos Nagilum_: it appears i owe you one :-)
11:24 Nagilum_ tziOm: currently one 6-box gluster out of about 50 boxes
11:25 Nagilum_ another setup is being prepared
11:25 tziOm ok, and using native nfs?
11:25 Nagilum_ no
11:25 Nagilum_ not in that setup
11:25 tziOm using glusterfs cifs?
11:26 Nagilum_ no, just mount.glusterfs
11:26 tziOm ok.
11:27 Nagilum_ I only once mounted the glusterfs via nfs, from a HP-UX box, but using anything but glusterfs as fstype wont give you the benefits of gfs
11:27 tziOm I did a quick test with nfs, and was (for some reason) only able to do aroung 300 mounts from a client and it was slow. With cifs I mounted 5k mounts relatively quick
11:27 raghu joined #gluster
11:27 tziOm Nagilum_, what are the benefits you mean?
11:28 Nagilum_ that you can read from multiple nodes in parallel
11:28 Nagilum_ if you use nfs everything has to go through your nfs server node
11:28 tziOm What up with ripe today? whois down seems
11:29 tziOm Nagilum_, yeah.. but my setup is having brick servers mount glusterfs, and serve everything from there using samba
11:29 Nagilum_ no better option if you need to use Windows I suppose
11:30 tziOm Seems like cifs is a better choice anyway so far.
11:30 tziOm mount time and so on
11:31 andrei_ joined #gluster
11:33 dustint joined #gluster
11:33 tziOm Nagilum_, when it comes to automount it does make a diff..
11:34 Nagilum_ I only mount it once! ;)
11:36 dustint joined #gluster
11:37 dustint joined #gluster
11:38 tziOm ok
11:38 Nagilum_ *hrm* bloody geo-replication status output is so useless..
11:41 nueces joined #gluster
11:44 andrewjs1edge joined #gluster
11:46 Durzo joined #gluster
11:47 Durzo hi people, need some help.. have 2 gluster servers in replicated volume. one of the servers is flooding the glusterfs/brick/export-brick.log logfile with GETXATTR (null) (glusterfs.gfid2path) ==> -1 (No such file or directory) - its growing so quickly its chewing through about 6GB per 20 minutes.. any ideas? i have tried restarting it with no success
11:54 Nagilum_ Durzo: stop glusterfs, rsync the bricks including xattribs, start it up again
11:54 Nagilum_ the one with the big log will be the destination..obviously
11:54 tziOm hmm..
11:54 Durzo would that result in data loss?
11:55 tziOm sounds like a hacky solution
11:55 Durzo this is a production cluster
11:55 Nagilum_ Durzo: I don't think so
11:55 Durzo any idea how this happened??
11:55 Nagilum_ Durzo: I see, then do something else
11:56 Nagilum_ do ".glusterfs" and ".landfill" exist on the faulty brick?
11:57 Durzo yes
11:57 yinyin_ joined #gluster
11:57 Durzo wait, .landfill does not
11:58 Durzo it does on the other node
11:58 Durzo should I make it? rsync it?
11:58 Nagilum_ Durzo: yeah
11:58 Durzo which?
11:58 Nagilum_ Durzo: but don't get your hopes up yet :>
11:59 Nagilum_ Durzo: you said you have a replicated set so copy it from the mirror-brick
11:59 manik joined #gluster
11:59 Durzo it appears to be empty on the other node
11:59 Durzo il just mkdir it
12:00 Nagilum_ not sure about the xattribs
12:00 * Nagilum_ is out for lunch
12:06 MaRoe joined #gluster
12:08 robos joined #gluster
12:10 MaRoe hey there. I have a problem with gluster and puppet. The installation fails with "/usr/sbin/glusterd: option requires an argument -- 'f'" if installed via puppet. If I install it manually, everything is fine. Any hints?
12:13 tziOm why puppet?
12:14 MaRoe why not?
12:14 MaRoe our deploy scripts are written in puppet
12:14 manik1 joined #gluster
12:18 karoshi joined #gluster
12:24 bennyturns joined #gluster
12:30 shireesh joined #gluster
12:30 waldner joined #gluster
12:31 waldner hi, I've got a webserver serving static content (lots of small files) out of a glusterfs replicated volume.
12:32 waldner I had to replace one of the bricks and now I tried to just readd the new empty one and let self-healing do its job, but this kills the server.
12:32 waldner How do I go about minimizing the impact of readding the empty brick?
12:32 edward1 joined #gluster
12:34 aliguori joined #gluster
12:34 bala joined #gluster
12:39 glusterbot New news from newglusterbugs: [Bug 963223] Re-inserting a server in a v3.3.2qa2 distributed-replicate volume DOSes the volume <http://goo.gl/LqgL8> || [Bug 962226] 'prove' tests failures <http://goo.gl/J2qCz>
12:40 hchiramm__ joined #gluster
12:41 yinyin_ joined #gluster
12:42 bennyturns joined #gluster
12:43 * H__ sobs :(
12:43 samppah ?? :(
12:43 H__ that last bug glusterbot posted is mine.
12:45 H__ I need a way to re-insert a server without bringing down the entire production environment
12:46 StarBeast joined #gluster
12:47 waldner H__: sounds similar to what I'm looking for as well
12:48 H__ waldner: what problem do you see ? is it already a bugticket ?
12:49 waldner I'm trying to insert an empty brick into a replicated volume, but this kills the webserver (which is the client using the volume)
12:49 waldner because replication happens at the client, and it'd need to replicate hundreds of files per second
12:49 H__ well, I can help you out there, if you don't have too many writes
12:50 waldner no write, in fact
12:50 waldner it's a webserver serving static content
12:50 waldner so I only have reads
12:50 waldner though lots of them
12:50 H__ you lucky &^%&*^&^% ;-)
12:50 waldner :)
12:50 H__ what gluster version ?
12:50 waldner 3.3.something on debian
12:50 * waldner checks
12:50 waldner 3.3.1
12:51 H__ good. here's the workaround :
12:51 waldner \o/
12:51 H__ is this on a single server or over multiple servers ?
12:51 waldner I though about a brick-to-brick rsync, but I'm unsure about the implications of doing that
12:51 waldner the client is a pair of servers
12:52 waldner each one mounts the gluster volume
12:52 H__ i mean the gluster cluster. is that 1 server or multiple ? anyway, here's a *way* faster-than-rsync workaround :
12:52 waldner ah, the gluster
12:53 andreask joined #gluster
12:53 waldner it's a 2-brick replicated volume, each brick is a different server (only one is active atm, the second one is the one I'm trying to insert)
12:53 dustint joined #gluster
12:53 H__ bsdtar -cf - -C /bricks/1/ --one-file-system . | ssh otherglusterserver "bsdtar -xpvf - -C /bricks/1/"
12:54 H__ that is a factor 10 faster than rsync where you have to explain to rsync that it must honour hardlinks
12:54 waldner ok
12:54 H__ when that step is done :
12:55 waldner however, that's still a direct write onto the brick that bypasses gluster, isn't it?
12:55 waldner ok
12:55 pkoro joined #gluster
12:55 waldner I'll shut up now :)
12:55 H__ setfattr -x trusted.glusterfs.volume-id /bricks/1
12:55 H__ and
12:55 H__ setfattr -x trusted.gfid /bricks/1
12:56 H__ those two setfattr are on the target server, the 'otherglusterserver' in above bsdtar pipeline
12:56 waldner yes
12:56 H__ then I add the brick with a force : gluster volume replace-brick ... commit force
12:57 waldner ok
12:57 waldner thank you very much, I'll try this
13:02 lalatenduM joined #gluster
13:05 Durzo H__, whats the command to list the xattrs - to make sure the dirs already have that flag?
13:12 abyss^_ joined #gluster
13:15 H__ Durzo: i think this : getfattr -d -e hex -m . /bricks/1
13:15 plarsen joined #gluster
13:17 MrNaviPacho joined #gluster
13:18 vpshastry joined #gluster
13:19 hagarth joined #gluster
13:21 ctria joined #gluster
13:23 dewey joined #gluster
13:25 jskinner_ joined #gluster
13:29 robos joined #gluster
13:30 abyss^ joined #gluster
13:30 bit4man joined #gluster
13:37 hchiramm__ joined #gluster
13:39 atrius_ joined #gluster
13:39 andrewjs1edge joined #gluster
13:44 Durzo hi guys
13:44 Durzo having an issue rsyncing my bricks
13:44 Durzo im using rsync -aHAX --delete, afterwards the file/size count on each is waaay different
13:44 majeff joined #gluster
13:51 StarBeast joined #gluster
13:53 Durzo sigh.. even after rsyncing i still get this error... 0-data-server: 40671: UNLINK <gfid:7d5085c3-4793-4696-9686-68bb42b04e​8e>/72a54553-242c-4012-867f-6d493fdd92f5 (7d31da30-9db6-407a-8bc1-43e0da2c5644) ==> -1 (No such file or directory)
13:53 Durzo times a trillion
13:53 Durzo all difference gids
13:56 waldner H__: just for my future reference, would the procedure you explained also work for adding a new brick to an existing volume?
13:57 manik joined #gluster
13:58 H__ waldner: on a replicate setup bricks come in pairs, two empty bricks can be added without issue. (rebalancing data is a whole other story though)
13:58 waldner H__: well in case you start out with a single brick
13:58 waldner which AFAICT gluster lets you do
14:00 Durzo also getting this: 0-data-server: 261821: GETXATTR (null) (glusterfs.gfid2path) ==> -1 (No such file or directory)
14:00 waldner and also replica 3 is possible if I'm not wrong
14:00 Durzo any ideas? H__ ??
14:03 daMaestro joined #gluster
14:07 puebele joined #gluster
14:11 piotrektt_ joined #gluster
14:14 lpabon joined #gluster
14:16 bugs_ joined #gluster
14:17 kaptk2 joined #gluster
14:22 jtux joined #gluster
14:28 brian_ joined #gluster
14:30 dmojorydger joined #gluster
14:32 brian_ I am getting an error trying to mount my gluster volume (called gv0 over NFS). When I try a manual mount with this command: mount -t nfs -o vers=3 node02:/gv0 /gluster-mnt-dir/   I get this error: "mount.nfs: access denied by server while mounting node02:/gv0"   I have looked in /var/log/messages and nothing useful shows in there. Also in nothing in /var/log/glusterfs that shows a problem. It does this wether iptabels off… I have other NF
14:32 brian_ mounts working though.
14:33 brian_ the other NFS mounts that are working are not gluster mounts though
14:37 Goatbert joined #gluster
14:38 lalatenduM joined #gluster
14:41 dmojorydger in a geo-replication setup, what does gsyncd do?
14:49 x4rlos Nagilum_: are you back? :-/
14:53 x4rlos Nagilum_: whilst your method fixed the missing files from the mirror, they are not replicating by th elooks of it :-(#
14:54 hagarth joined #gluster
14:56 StarBeast joined #gluster
15:00 brian_ left #gluster
15:05 x4rlos Nagilum_: FYI i restarted the gluster daemon on the 'working' gluster server, and it seems to have prompted the copy. Will see how it goes.
15:08 Nagilum_ k
15:12 jthorne joined #gluster
15:12 brian_ joined #gluster
15:14 Supermathie brian_: output of 'volume info gv0'?
15:14 brian_ question: I'm trying to mount gluster over NFS… do I need to set up exports for the nodes (the ones with the "bricks"?
15:15 Supermathie brian_: nononono
15:15 semiosis ~nfs | brian_
15:15 glusterbot brian_: To mount via nfs, most distros require the options, tcp,vers=3 -- Also an rpc port mapper (like rpcbind in EL distributions) should be running on the server, and the kernel nfs server (nfsd) should be disabled
15:15 brian_ ok I didn't think so.. I don't have that set up
15:15 Supermathie touching the files on the bricks will break things badly. Brick communication is done via the glusterfsd daemons
15:15 brian_ do I need to set up ANY exports to mount gluster over NFS?
15:16 Supermathie brian_: Ah! You're trying to use kernel nfs server and gluster nfs server at the same time?
15:17 brian_ yes I guess I am because I have existing non-gluster NFS mounts already
15:18 Supermathie brian_: you need to pass the port number of gluster's nfs server to your mount command. Also, disable nfs.register-with-portmap.
15:18 manik joined #gluster
15:19 dbruhn joined #gluster
15:19 brian_ here is the output of my gluster volume info gv0: http://fpaste.org/12311/86311541/
15:19 glusterbot Title: #12311 Fedora Project Pastebin (at fpaste.org)
15:20 dbruhn is there a way to tell which bricks a file is stored on in a distributed and replicated system
15:22 semiosis ~pathinfo | dbruhn
15:22 glusterbot dbruhn: find out which brick holds a file with this command on the client mount point: getfattr -d -e text -n trusted.glusterfs.pathinfo /client/mount/path/to.file
15:22 dbruhn Thanks semiosis
15:22 semiosis yw
15:24 vpshastry1 joined #gluster
15:25 brian_ Supermathie: how do I know if I'm running the "kernel version" of NFS… The NFS I have running is just the NFS service that starts when CentOS boots.. is that the same as the "kernel version"?
15:25 Supermathie brian_: Yeah. The distinguishment I'm trying to make (is that a word?) is between Linux's NFS server and GlusterFS's NFS server.
15:26 brian_ so in order to run GlusterFS's NFS server, I should shut down the Linux NFS server?
15:26 Supermathie brian_: Only one can register with portmap.
15:26 brian_ ok i see
15:27 tshm_ http://en.wiktionary.org/wiki/distinguishment   ;-)
15:27 brian_ i think :)
15:27 glusterbot Title: distinguishment - Wiktionary (at en.wiktionary.org)
15:27 brian_ so it isn't possible to run both then?
15:27 Supermathie tshm_: :D
15:27 Supermathie Yes, it's possible to run both, but only one can register with portmap. I told you above how to do it...
15:27 Supermathie "you need to pass the port number of gluster's nfs server to your mount command."
15:28 Supermathie otherwise, the client has no way of finding it.
15:28 brian_ ok
15:28 brian_ sow the port is 7997?
15:28 Supermathie brian_: 'gluster volume gv0 status' will show you
15:28 lalatenduM joined #gluster
15:29 brian_ ok
15:29 Supermathie I have no idea about the lockd interaction and how to resolve that though.
15:33 brian_ hmmm I specified the port, but I'm still getting "mount.nfs: access denied by server while mounting node03:38467:/gv0"
15:33 brian_ 38467 is the port showing in volume status
15:33 Supermathie how did you specify the port? show the command
15:33 brian_ here is the command I used to try and mount: mount -t nfs node03:38467:/gv0 /gluster-mnt-dir
15:34 brian_ http://fpaste.org/12320/32076136/  <--- this is my volume info
15:34 glusterbot Title: #12320 Fedora Project Pastebin (at fpaste.org)
15:35 Supermathie mount -t nfs node03:/gv0 /gluster-mnt-dir -o port=38467 is correct
15:36 brian_ hey I tried your command and now I'm getting a different error. It saying that the NFS version or transport is not supported, but from the docs I think I know how to fix this one
15:36 Supermathie ...-o defaults,port=38467,vers=3...
15:40 Shdwdrgn joined #gluster
15:42 brian_ does this look right?
15:42 brian_ mount -o proto=tcp,vers=3,port=38467 -t nfs node03:/gv0 /gluster-mnt-dir
15:43 Nagilum_ /sbin/fs/nfs/mount -o vers=3,port=38467 nfs://gfs-server/gv01 /mnt
15:43 Nagilum_ thats what I used on HP-UX
15:43 Supermathie yeah well HP-UX is crazy :p
15:43 Supermathie brian_: yeah try it
15:43 duerF joined #gluster
15:44 brian_ i did, I get this same error again: mount.nfs: access denied by server while mounting node03:/gv0
15:45 Supermathie add mountport=38465
15:46 brian_ ok
15:46 Supermathie brian_: When you get it all figured it out, post the problem and how you solved it on serverfault :)
15:47 Supermathie there should be a writeup on this somewhere but I can't find it
15:47 brian_ you mean substitute port=38467 for mountport=38467  right?
15:47 Supermathie no I didn't.
15:47 Nagilum_ brian_: did you nfs.rpc-auth-allow <IP> your IP?
15:48 brian_ Nagilum_: no
15:48 Supermathie Nagilum_: He hasn't set any volume options so he should be fine from that side, right?
15:48 Nagilum_ I don't know what he has set
15:49 Nagilum_ brian_: http://gluster.org/community/documentation/i​ndex.php/Gluster_3.2:_Setting_Volume_Options
15:49 glusterbot <http://goo.gl/dPFAf> (at gluster.org)
15:49 Supermathie Nagilum_: nothing, he posted it above.
15:49 Nagilum_ oh, k
15:49 tc00per left #gluster
15:49 Supermathie brian_: mount -o proto=tcp,vers=3,port=38467,mountport=38465 node03:/gv0 /gluster-mnt-dir
15:50 brian_ Supermathie: SUCCESS!!! it mounted
15:50 Supermathie brian_: You may have weird locking problems, BTW.
15:51 brian_ still don't understand what I was doing wrong
15:51 brian_ I guess adding the mountport fixed it, but dont know why
15:51 Supermathie brian_: mount was trying to contact the mountd registered with portmap (kNFS mountd) which didn't know about /gv0
15:52 brian_ ok
15:52 brian_ thanks supermathie… once again your awesome! :)
15:52 portante|ltp joined #gluster
15:53 majeff joined #gluster
15:53 * Supermathie <morpheus>I know.</morpheus>
15:53 Supermathie thx ;)
15:53 brian_ lol
15:54 Shdwdrgn joined #gluster
15:56 brian_ here is something wierd… before mounting over NFS, I tried to run bonnie++ benchmark (using my gluster-mnt-dir mount point as the directory for writing the data), and it got started and never finished… it was so slow I had to stop it after letting it run over night… I'm going to try the same bonnie++ benchmark now running it over this NFS mount to see what happens.
15:57 StarBeast joined #gluster
15:58 semiosis brian_: what do you hope to gain using bonnie++?  it's not a good benchmark for a distributed cluster fs
15:58 brian_ supermathie: I noticed this successful mount command you gave me didn't have nfs specified in it (-t nfs )
15:59 brian_ well I'm just doing what the boss told me …  :)
15:59 failshell joined #gluster
15:59 Supermathie brian_: Right. mount will infer that from the node03:/gv0 path.
15:59 Supermathie I'm a minimalist
15:59 failshell is there any other form of authentication possible with a volume other than IP ACLs?
16:02 brian_ supermathie: in your opinion, what is the best way to benchmark a glusterfs? I did try iozone too (and it actually did complete the test), but my results weren't good…
16:02 cfeller joined #gluster
16:03 * semiosis has an opinion on that too
16:03 semiosis if you want :)
16:03 Supermathie brian_: Well, I'm using Oracle and https://github.com/khailey/fio_scripts
16:03 glusterbot Title: khailey/fio_scripts · GitHub (at github.com)
16:03 Supermathie but I'm relatively new to glusterfs :)
16:04 brian_ ok, I'll try that one too
16:04 zaitcev joined #gluster
16:05 failshell brian_: i tried bonnie++, iozone and some dirty dd. writes will always be much slower than regular disks. but reads are ok.
16:06 brian_ failshell: my bonnie++ run was so slow I had to stop it.. It ran over night and never even got past "Rewriting Intellignently.. "
16:07 failshell well my tests did finish in a timely fashion
16:07 brian_ my iozone completed, thought it had  crappy results
16:07 semiosis brian_: what is your use case for glusterfs?
16:07 failshell define crappy results
16:07 Supermathie brian_: glusterfs fails hard when you have multiple concurrent things accessing the same file
16:07 semiosis what kind of workload?
16:07 semiosis Supermathie: really?
16:08 failshell ^^
16:08 semiosis Supermathie: is that a new bug?
16:08 Supermathie semiosis: No idea.
16:08 semiosis Supermathie: glusterfs' origins in supercomputing focused heavily on that particular use case
16:08 Supermathie well fails hard as in it slows down to ~100KBps aggregate
16:08 brian_ this is just a demo cluster her in our office.. Nobody is going to use it. My boss wanted me to install gluster on it and do some benchmarking and compare the results to just using regular NFS
16:09 failshell things i use gluster for: centralized mongodb/mysql dumps. backend for spacewalk.
16:09 semiosis brian_: the best thing to show off glusterfs' capabilities is to use many bricks over multiple servers, and many clients accessing many files in the cluster.
16:09 failshell we even serve one of our websites with it
16:09 semiosis brian_: it will shine in aggregate performance, even though you sacrifice some single-thread performance for all the network overhead
16:09 Supermathie brian_: http://pastie.org/7912050 some sample numbers from fioscripts on my gluster setup with fuse
16:09 glusterbot Title: #7912050 - Pastie (at pastie.org)
16:10 brian_ semiosis: Yeah, this thing only has 3 nodes (which I'm using as the brick servers). I'm guessing that only three nodes with gluster is pointless right? I think this is what he wants me to discover for myself with this little training exercise.
16:11 semiosis brian_: fun exercise
16:11 brian_ well you guys helping me has improved the "fun" aspect… I feel completely lost with the stuff
16:11 brian_ :/
16:11 Supermathie brian_: 3 nodes? I'm only using 2 :)
16:11 semiosis brian_: three servers is very reasonable.  the only number of servers (imho) that's pointless with gluster is 1
16:11 tqrst I found a mailing list post that mentions you can "setfattr -n trusted.distribute.fix.layout -v 'blah' /some/client/path" to fix-layout a specific folder on a volume. Is this supposed to work for 3.3.1? It doesn't seem to do anything here.
16:16 Supermathie hah I tried and it crashed my fuse client
16:17 semiosis Supermathie: you have issues
16:17 semiosis hehe
16:18 sprachgenerator joined #gluster
16:18 Supermathie [michael@fleming1:/gv0/common]$ setfattr -n distribute.fix.layout -v 'blah' flippantfile.txt
16:18 Supermathie setfattr: flippantfile.txt: Software caused connection abort
16:19 Supermathie semiosis: Dude, I can make anything fail.
16:20 Supermathie semiosis: check out this ticket I opened: http://i.stack.imgur.com/m1J1x.png
16:21 manik joined #gluster
16:21 semiosis those attached images must be great
16:22 Supermathie semiosis: You should see the x-rays... don't think I can show them though.
16:23 Supermathie tqrst: the correct attribute to set is distribute.fix.layout (without the trusted). It might crash :)
16:24 Supermathie http://pastie.org/7912112
16:24 glusterbot Title: #7912112 - Pastie (at pastie.org)
16:25 Gugge joined #gluster
16:25 jskinner_ joined #gluster
16:27 vpshastry1 left #gluster
16:28 bdperkin joined #gluster
16:31 jag3773 joined #gluster
16:32 Supermathie Does that work for anyone running stock 3.3.1?
16:34 majeff joined #gluster
16:36 Mo__ joined #gluster
16:36 semiosis Supermathie:  i get operation not supported with distribute.fix.layout
16:37 Supermathie semiosis: fuse not nfs right?
16:37 semiosis right
16:37 semiosis want me to try nfs?
16:37 Supermathie no that's where I get 'not supported'
16:42 majeff1 joined #gluster
16:43 bulde joined #gluster
16:46 Supermathie tqrst: did you try that?
16:49 jkroon_ joined #gluster
16:50 jkroon_ hi all, given a 2x2 distribute-replicate setup, would it be advisable to switch to a pure replicated setup?
16:50 Supermathie jkroon_: why?
16:50 jkroon_ i've got a number of 2x replicate setups that's running like a charm, but this 2x2 thing is given me all kinds of hell.
16:51 jkroon_ as in bricks just goes missing
16:51 jkroon_ getting situations where files are inaccessible even though all four machines are up, and a few seconds later everything seems OK again.
16:52 jkroon_ just trying to figure out how much of my issues are gluster and how much could be external factors (other setups are dedicated ethernet links)
16:53 jkroon_ gluster version 3.3.1 ...
16:53 Supermathie I haven't had that problem with using distribute (8x2) - sounds like you have hw issues.
16:53 jkroon_ Supermathie, what should I look at?
16:53 Supermathie logs
16:54 jkroon_ hmm, yea, that's probably a good place to start digging.
16:54 jkroon_ also, is there any serious CPU discrepency between mostly-read and mostly-write setups?
16:55 jkroon_ i would wager that writing is much more cpu intensive?
16:55 Supermathie jkroon_: mmmmm all my tests have been write-heavy
16:57 jkroon_ specific setup is 4 VMs (http://isoho.st), so hardware should not be a problem, getting <1ms round-trip between all hosts, and i've worked around the iptables issue already, so -A INPUT -i eth1 (private VLAN) is the first rule there.
16:57 glusterbot Title: isoho.st | real virtual machines (at isoho.st)
16:57 jkroon_ thanks glusterbot
16:57 StarBeast joined #gluster
17:04 jskinner_ joined #gluster
17:06 rotbeard joined #gluster
17:09 StarBeast joined #gluster
17:10 glusterbot New news from newglusterbugs: [Bug 963335] glusterd enters D state after replace-brick abort operation <http://goo.gl/pGy04>
17:14 jkroon_ Supermathie, what kind of overall IO are you getting out of that 8x2?
17:14 pithagorians_ joined #gluster
17:14 jkroon_ i'm seeing major slowdown of IO even under low load
17:15 jkroon_ probably getting about 1MB/s out of it and that's consuming around 50% CPU
17:15 jkroon_ suspect part of the issue is that the client's application has a bunch of processes all opening the same file with append and writing logs to it ...
17:24 Supermathie jkroon_: on a transactional (Oracle) workload, peak performance is average of 5700 transactions/sec (3317B/transaction)
17:25 Supermathie So not much. Also that's using kNFS instead of DNFS so it's slower than it could be.
17:26 jkroon_ hmm
17:27 premera_g is there a way to mount glusterfs fuse in 'sync' mode ? something like  mount -t glusterfs -o sync  server:/brick1 /mnt  (this throws 'unknown option sync (ignored)') I am on 3.3.1 on ubuntu 12.04
17:27 failshell Supermathie: am i getting this right. you're running oracle on gluster?
17:28 Supermathie failshell: yeah
17:28 failshell Supermathie: dedicated hardware i assume for the bricks? and SSDs?
17:29 Supermathie failshell: yeah, trying 8x2 setup with 400GB Enterprise SSDs now, will also be trying a 1x2 with 3.2TB ioScale cards as bricks
17:29 failshell Supermathie: and Oracle supports you on that kind of support?
17:30 failshell s/support/setup
17:30 Supermathie failshell: not yet ;) researching this for a client, Making It Work.
17:30 failshell ok
17:31 Supermathie customer wants to stop paying ridiculous amounts of money for NetApp, I'm researching alternatives.
17:31 failshell id be curious to hear what Oracle has to say about Gluster
17:31 failshell i doubt they'll support that
17:32 Supermathie customer is big enough that they'll care.
17:33 Supermathie It's been productive though, tickled out a number of bugs in gluster and one in Oracle.
17:33 tqrst Supermathie: sorry about that, got drive-by-meeting-ed
17:34 tqrst Supermathie: doesn't crash here, and is supposedly doing something according to the logs
17:34 failshell Supermathie: hopefully, you'll blog about this
17:34 avati Supermathie, what do you mean kNFS instead of DNFS? are you referring to NFS clients?
17:34 Supermathie failshell: perhaps, I'll see.
17:34 Supermathie avati: Oracle's DNFS (userspace) client
17:35 Supermathie ... instead of Linux's NFS client
17:35 avati ok
17:35 failshell Supermathie: is that a RAC setup?
17:36 Supermathie DNFS can multipath and is much more efficient than Linux's NFS client, so would be ideal for hitting Gluster, but: https://bugzilla.redhat.com/show_bug.cgi?id=960141
17:36 glusterbot <http://goo.gl/RpzTG> (at bugzilla.redhat.com)
17:36 glusterbot Bug 960141: urgent, unspecified, ---, vraman, NEW , NFS no longer responds, get  "Reply submission failed" errors
17:36 Supermathie failshell: no
17:37 avati yeah.. DNFS + gluster would be pretty cool, if we can get it to work ;)
17:38 Supermathie avati: sooooooo close!
17:39 Supermathie At this point I've had to say "DNFS won't work with this setup" and test with kNFS, next I'll be trying GPFS. Which I suspect will smoke gluster but isn't free.
17:40 Supermathie Red Hat really should be sponsoring us so we can get this working properly in Red Hat Storage Server ;)
17:42 jskinner joined #gluster
17:43 avati I'm not sure Red Hat would be interested in that market segment.. though gluster community would surely like to see it work
17:46 Supermathie avati: It'd be a big checkmark for taking out NetApp
17:46 avati +1
17:47 jskinner_ joined #gluster
17:47 avati Supermathie, is there a way to reproduce the issue without using oracle?
17:48 avati Supermathie, another useful thing would be to characterize performance without replication
17:48 Supermathie avati: Sure! Replay the traffic from the packet capture I provided :)
17:54 anands joined #gluster
18:03 Supermathie avati: Not the answer you were looking for, I know, but I unfortunately can't spend any more time on this right now.
18:04 andreask joined #gluster
18:39 statix_ joined #gluster
18:53 MrNaviPacho joined #gluster
19:02 lpabon joined #gluster
19:06 lbalbalba joined #gluster
19:08 thebishop joined #gluster
19:08 MaRoe joined #gluster
19:09 lbalbalba hi. im trying to troubleshoot prove test 'bug-808400.t'. it uses './bug-808400-fcntl.c
19:10 lbalbalba and prodoces an error: fstat failure (Value too large for defined data type
19:11 lbalbalba running it under strace, show it calls 'fstat64'. but im on a 32 bit fedora linux system. i dont get it
19:12 lbalbalba this is the c code: http://fpaste.org/12385/64514813/
19:12 glusterbot Title: #12385 Fedora Project Pastebin (at fpaste.org)
19:13 Supermathie fstat64 doesn't refer to the bitness of the OS, it refers to the size of the call. Under the hood, your fstat gets translated to a system call to fstat64.
19:13 Supermathie Under the hood -> in glibc I think
19:13 lbalbalba ah. so you should be able to run fstat64 on a 32-bit os ?
19:13 Supermathie eyah
19:14 Supermathie s/(e)(y)/\2\1/
19:14 lbalbalba then i dont get why i get this message: fstat failure (Value too large for defined data type)
19:14 Supermathie Damn, did I confuse glusterbot?
19:15 H__ seems so
19:16 Supermathie lbalbalba: check output of 'strace stat randomfile'
19:18 lbalbalba yeah, thats fstat64 as well. ok. my mistake. but the prove test still fails. and strace -f stat /tmp/randomfile doesnt
19:18 Supermathie lbalbalba: I'm guessing it's going to be the inode number is too large for the stat structure
19:19 Supermathie The underlying size of the default stat structure used on 32-bit may be too small to hold the inodes that gluster returns
19:19 Supermathie That's my guess.
19:20 Supermathie run 'stat' on a random file on glusterfs on your 32-bit system
19:20 lbalbalba strace stat /etc/motd runs fine
19:21 lbalbalba tyhe testfile im using is 0 bytes: /mnt/glusterfs/testfile
19:22 lbalbalba running the c file used in the test:    ./bug-808400-fcntl /mnt/glusterfs/testfile "gluster volume set glustervol01 performance.write-behind on"
19:22 lbalbalba and produces the error
19:22 ninkotech joined #gluster
19:22 ninkotech_ joined #gluster
19:22 lbalbalba stat /mnt/glusterfs/testfile runs fine
19:24 lbalbalba 'mnt/glusterfs/testfile' is on a client mounted glsuterfs volume
19:25 lbalbalba glusterfs volume
19:30 plarsen_ joined #gluster
19:32 lbalbalba strace on the test cmd shows this : http://fpaste.org/12392/13686462/
19:32 glusterbot Title: #12392 Fedora Project Pastebin (at fpaste.org)
19:34 andrei_ joined #gluster
19:40 harold_ joined #gluster
19:50 lbalbalba well i guess i really need to learn c now before i even attempt to look into this one ;)
19:59 lbalbalba hrm. all of the failing tests use bug-808400-flock.c and/or bug-808400-fcntl.c.
20:00 andrewjs1edge joined #gluster
20:11 andreask joined #gluster
20:26 lbalbalba joined #gluster
20:29 jiku joined #gluster
20:29 jiku joined #gluster
20:37 lbalbalba hrm. looking at the difference in output between 'stat /mn/glustervol/file' and 'bug-fcnt /mnt/glustervol/file': http://fpaste.org/12413/50158136/
20:37 glusterbot Title: #12413 Fedora Project Pastebin (at fpaste.org)
20:38 lbalbalba good: 'st_ino=3' vs bad 'st_ino=12634102522194315616'
20:39 lbalbalba looks like Supermathie is right. but i still dont understand *why* the difference
20:40 ThatGraemeGuy joined #gluster
20:58 jiku hello all..
20:58 jiku we have a replicate gluster setup with 2 nodes.
20:58 jiku unfortuntaly we need to change the IP  of both the peers due to datacenter limitations.
20:59 jiku can this be done on the run after stopping glusterd service ?
20:59 jiku should i destroy the volume before doing this?
21:02 JoeJulian jiku: Yes, you /can/ do it by hacking files directly under /var/lib/glusterd (make a backup first) with everything stopped. The "correct" way would be to recreate your volumes in order to change them.
21:02 semiosis also ,,(hostnames) FTW
21:02 glusterbot Hostnames can be used instead of IPs for server (peer) addresses. To update an existing peer's address from IP to hostname, just probe it by name from any other peer. When creating a new pool, probe all other servers by name from the first, then probe the first by name from just one of the others.
21:02 JoeJulian jiku: ... also, that's why we recommend using ,,(hostnames) instead of IP addresses.
21:02 glusterbot jiku: Hostnames can be used instead of IPs for server (peer) addresses. To update an existing peer's address from IP to hostname, just probe it by name from any other peer. When creating a new pool, probe all other servers by name from the first, then probe the first by name from just one of the others.
21:02 JoeJulian hehe
21:02 jiku :-( i should have followed that.. hmm..
21:05 JoeJulian lbalbalba: It may have to do with compile options. There's an option to compile with 64bit inode capabilities. It's been a while since I looked at that though, so I can't remember what it is off the top of my head.
21:05 JoeJulian Otherwise that 64 bit inode will be truncated when it's read into the st_ino structure.
21:08 avati ah, we need to have enable-ino32 turned on by default on 32bit builds
21:08 avati *32bit systems
21:08 avati it is disabled by default everywhere
21:09 badone joined #gluster
21:12 avati lbalbalba, can you apply http://shell.gluster.com/~avati/32bit-ino.diff and check if it fixes your problem?
21:18 lbalbalba avati: done patch, make, make install: same error
21:19 lbalbalba :(
21:20 avati what is the error?
21:21 lbalbalba avati: same as before: fstat failure (Value too large for defined data type)
21:22 jurrien_ joined #gluster
21:24 lbalbalba avati: reverted the patch
21:27 brian_ I want to clean all files from all of my "bricks", is there a command that will do that?
21:29 lbalbalba avati: this fixes the error: http://fpaste.org/12429/86533731/
21:29 glusterbot Title: #12429 Fedora Project Pastebin (at fpaste.org)
21:30 lbalbalba avati: change fstat into fstat64
21:30 lbalbalba avati: but dont ask me why. i can google good ;)
21:34 JoeJulian -D__USE_FILE_OFFSET64
21:36 hchiramm__ joined #gluster
21:39 lbalbalba JoeJulian: yes, 'gcc  -D__USE_FILE_OFFSET64' works, too
21:40 JoeJulian Aha... Not bad considering I only found that once and it was back when I was looking for something else: http://joejulian.name/blog/br​oken-32bit-apps-on-glusterfs/
21:40 glusterbot <http://goo.gl/4T31C> (at joejulian.name)
21:42 lbalbalba well, i attached the fstat/fstat64 patches to my bug report: https://bugzilla.redhat.com/show_bug.cgi?id=962226 but feel free to change the test files to gcc -D_foo ;)
21:43 glusterbot <http://goo.gl/J2qCz> (at bugzilla.redhat.com)
21:43 glusterbot Bug 962226: unspecified, unspecified, ---, vbellur, NEW , 'prove' tests failures
21:46 duerF joined #gluster
21:50 Supermathie lbalbalba: woot what do I win? :)
21:51 Supermathie Man it would REALLY be nice to be able to mount a subdir vi the gluster client
21:51 lbalbalba Supermathie: hahaha. but what do *i* win ? as a overly active sysadmin but non-dev ?
21:53 Supermathie lbalbalba: bah all sysadmins are devs. They just develop things that work ;)
21:53 Supermathie OK, all good sysadmins. That also happen to code.
21:54 Supermathie And know what they're doing.
22:03 lbalbalba Supermathie: this (former) sysadmin semi-knows what he is doing. but all the 'coding' i can do is limited to shell. But those do 'just work'.  ;)
22:32 lbalbalba So.... even after applying the patches, the following two tests still fail: ./tests/bugs/bug-823081.t  ./tests/bugs/bug-921072.t   output: http://fpaste.org/12445/36865708/
22:32 glusterbot Title: #12445 Fedora Project Pastebin (at fpaste.org)
22:36 JoeJulian Just curious if those would pass if you mounted a scratchmonkey at /d
22:37 lbalbalba JoeJulian: huh ? what ?
22:38 JoeJulian @lucky scratchmonkey
22:38 glusterbot JoeJulian: http://en.wikipedia.org/wiki/Scratch_monkey
22:38 StarBeast joined #gluster
22:42 lbalbalba mount -t glusterfs 192.168.126.129:/glustervol01 /d        Mount failed. Please check the log file for more details.
22:42 lbalbalba crap
22:42 JoeJulian TEST 3 (line 29): gluster --mode=script volume create patchy lbalbalba.localdomain:/d/backends/patchy
22:43 lbalbalba volume create: patchy: success: please start the volume to access data
22:44 JoeJulian So /d/backends/patchy is the brick. If /d were a mounted xfs filesystem then you would avoid the "is is being created in the root partition" failure
22:44 JoeJulian note the bug? ^
22:44 JoeJulian "is is" hehe
22:46 lbalbalba JoeJulian: I need to go to bed. seriously. :(
22:46 JoeJulian hehe
22:46 JoeJulian sleep feels like such a waste of time...
22:47 yinyin joined #gluster
22:47 lbalbalba true. but so does wasting valuable volunteers time :)
22:47 JoeJulian hehe
22:48 JoeJulian You're helping make something that I use in production better. My time spent helping you do that is valuable to me.
22:48 JoeJulian So don't stress over it.
22:49 lbalbalba thanks. but im seriosuly gonna go now. see you later.
23:39 wgao joined #gluster
23:42 wgao__ joined #gluster
23:43 lpabon joined #gluster
23:56 lyang0 joined #gluster

| Channels | #gluster index | Today | | Search | Google Search | Plain-Text | summary