Perl 6 - the future is here, just unevenly distributed

IRC log for #gluster, 2017-03-27

| Channels | #gluster index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
01:07 shdeng joined #gluster
01:15 kettlewell joined #gluster
01:48 ilbot3 joined #gluster
01:48 Topic for #gluster is now Gluster Community - http://gluster.org | Documentation - https://gluster.readthedocs.io/en/latest/ | Patches - http://review.gluster.org/ | Developers go to #gluster-dev | Channel Logs - https://botbot.me/freenode/gluster/ & http://irclog.perlgeek.de/gluster/
02:02 pioto joined #gluster
02:31 vinurs joined #gluster
02:43 prasanth joined #gluster
03:03 buvanesh_kumar joined #gluster
03:09 kramdoss_ joined #gluster
03:21 ankush joined #gluster
03:43 dominicpg joined #gluster
03:48 kdhananjay joined #gluster
03:49 magrawal joined #gluster
03:55 riyas joined #gluster
04:01 itisravi joined #gluster
04:18 ppai joined #gluster
04:43 karthik_us joined #gluster
04:48 skumar joined #gluster
04:50 Shu6h3ndu joined #gluster
04:59 ashiq joined #gluster
05:08 kotreshhr joined #gluster
05:21 ndarshan joined #gluster
05:23 poornima_ joined #gluster
05:29 hgowtham joined #gluster
05:31 skoduri joined #gluster
05:33 jiffin joined #gluster
05:37 ahino joined #gluster
05:40 sbulage joined #gluster
05:42 gyadav joined #gluster
05:46 kharloss joined #gluster
05:49 kotreshhr joined #gluster
05:52 rastar joined #gluster
05:53 sona joined #gluster
05:54 itisravi joined #gluster
05:54 prasanth joined #gluster
05:59 kblin morning folks
06:00 kblin I've got a gluster volume that is somewhat misbehaving on one of the machines. I've got a pretty long list of pending heal operation
06:00 kblin that list doesn't seem to be getting any shorter, despite a consistently high load on one of my machines caused by glusterfsd eating lots of CPU
06:02 kblin this glusterfs is backing a bunch of compute nodes that I've drained over the weekend, so there's not a lot of disk IO going on anymore
06:02 kblin still, the amount of data printed by "gluster volume heal <volume> info" doesn't seem to go down
06:03 vinurs joined #gluster
06:03 kblin is there any way to figure out what's going on there?
06:03 jwd joined #gluster
06:06 ankush joined #gluster
06:12 Karan joined #gluster
06:15 mk-fg kblin, Not an expert advice, but I'd check the logs for heal failures - maybe some i/o errors there make gluster keep retrying these heals with no luck
06:18 kharloss joined #gluster
06:21 msvbhat joined #gluster
06:29 jtux joined #gluster
06:38 kharloss_ joined #gluster
06:44 jkroon joined #gluster
06:44 Saravanakmr joined #gluster
06:49 jtux left #gluster
06:52 kblin mk-fg: my old log has quite a bunch of timeout errors in the log there, but the log rotated yesterday morning, and has been empty since
06:53 Manikandan joined #gluster
06:53 kblin glustershd.log has quite a bunch of "conflicting entries" messages from today, though
06:55 apandey joined #gluster
06:56 ivan_rossi joined #gluster
06:56 ivan_rossi left #gluster
07:01 Manikandan left #gluster
07:01 ppai joined #gluster
07:12 [diablo] joined #gluster
07:14 hgowtham joined #gluster
07:18 mbukatov joined #gluster
07:36 susant joined #gluster
07:40 [diablo] joined #gluster
07:40 susant left #gluster
07:59 flying joined #gluster
08:00 steveg78 joined #gluster
08:00 hgowtham_ joined #gluster
08:09 ghenry joined #gluster
08:17 subscope joined #gluster
08:23 Philambdo joined #gluster
08:25 prasanth joined #gluster
08:31 ashiq joined #gluster
08:40 kdhananjay joined #gluster
08:45 itisravi joined #gluster
08:50 kotreshhr joined #gluster
08:54 Humble joined #gluster
08:55 anbehl joined #gluster
08:57 sbulage joined #gluster
09:01 Wizek_ joined #gluster
09:01 siel joined #gluster
09:02 karthik_us joined #gluster
09:06 Seth_Karlo joined #gluster
09:13 Seth_Karlo joined #gluster
09:14 nishanth joined #gluster
09:49 poornima_ joined #gluster
09:56 gyadav joined #gluster
10:14 apinter joined #gluster
10:17 msvbhat joined #gluster
10:24 semiosis joined #gluster
10:24 semiosis joined #gluster
10:29 nooke4000 joined #gluster
10:34 ppai joined #gluster
10:38 nooke4000 Guys, a quick question for glusterfs-3.10 (CentOS SIG): Why the SHD is showing N/A when listing the status of a volume? Also when I run heal and after it heal info it gives an error as it can't get the volfiles
10:38 nooke4000 idea?
10:45 Champi joined #gluster
11:06 kettlewell joined #gluster
11:28 vinurs joined #gluster
11:44 ppai joined #gluster
11:53 apandey joined #gluster
11:55 apandey_ joined #gluster
11:55 apandey_ joined #gluster
11:57 ashiq joined #gluster
12:03 karthik_us|afk joined #gluster
12:13 DV joined #gluster
12:23 kpease joined #gluster
12:28 ira joined #gluster
12:32 kpease joined #gluster
12:33 baber joined #gluster
12:38 jtux joined #gluster
12:39 unclemarc joined #gluster
12:44 Skinny joined #gluster
12:45 atinm joined #gluster
12:56 jtux left #gluster
12:56 msvbhat joined #gluster
12:59 prasanth_ joined #gluster
13:00 shyam joined #gluster
13:00 shaunm joined #gluster
13:13 sona joined #gluster
13:20 plarsen joined #gluster
13:29 riyas joined #gluster
13:31 FuzzyVeg joined #gluster
13:38 FuzzyVeg joined #gluster
13:41 skylar joined #gluster
13:41 armyriad joined #gluster
13:50 armyriad joined #gluster
13:57 aardbolreiziger joined #gluster
13:58 skoduri joined #gluster
14:09 kdhananjay joined #gluster
14:21 haloween joined #gluster
14:21 haloween Hi there !
14:26 vinurs joined #gluster
14:27 sona joined #gluster
14:29 haloween i have a issue with 3.10, i'm unable to access the server, logfile said " no authentication module is interested in accepting remote-client (null)"
14:32 Karan joined #gluster
14:35 jtux joined #gluster
14:46 farhorizon joined #gluster
14:56 ghenry joined #gluster
15:00 msvbhat joined #gluster
15:02 wushudoin joined #gluster
15:04 wushudoin joined #gluster
15:14 bdurrow joined #gluster
15:29 farhorizon joined #gluster
15:40 ankush joined #gluster
15:44 gyadav joined #gluster
15:46 Seth_Kar_ joined #gluster
16:23 kramdoss_ joined #gluster
16:29 Gambit15 joined #gluster
17:10 Shu6h3ndu joined #gluster
17:26 atinm joined #gluster
17:28 quest`` joined #gluster
17:29 msvbhat joined #gluster
17:37 gem joined #gluster
17:42 guhcampos joined #gluster
17:44 quest`` Hey, was having some issues with the gluster 3.8.10 packages on the ubuntu trusty PPA
17:44 quest`` gluster crashes on launch... but if I build gluster from source, it works fine...
17:44 quest`` anyone else reported issues with the trusty gluster 3.8.10 packages?
17:45 quest`` Also, is the script used to build them available anywhere?
17:47 quest`` It seems that the package that crashes for me us glusterfs-common... as if I build from source, the other packages work fine, but the moment I install glusterfs-common it segfaults
17:50 JoeJulian quest``: Everything used to build the PPA is on launchpad.
17:50 quest`` yeah the assets are, but I don't see the actual script used to build the packages
17:50 JoeJulian That's launchpad itself.
17:51 quest`` ah so no way to easily try and build it locally?
17:52 quest`` I was hoping I could try and do that on a base ubuntu image, then figure out what was off in the glusterfs-common package so I could submit a fix upstream
17:52 JoeJulian None that I'm aware of. I'm a long way from being a .deb expert.
17:52 quest`` KK, maybe someone else here has some context =)
17:52 JoeJulian kkeithley, semiosis: ^
17:53 quest`` Odd thing is, compiling the source everything works great... but I use the packages and it segfaults on startup =(
17:54 quest`` Basically the moment I try to do a peer probe
17:54 jiffin joined #gluster
17:55 JoeJulian That does sound interesting.
17:55 JoeJulian Maybe a gcc bug?
17:56 quest`` Yeah, and 3.7.20 works fine.
17:56 quest`` JoeJulian: not sure, which is why I am here =)
17:56 JoeJulian I wonder if I just bump the build version if it would be any different from what's up there now.
17:58 quest`` I have tried a couple 3.8.x builds now, and all of them have the same issue. That is why I went back to 3.7.20
17:58 quest`` But now I want to use latest ganesha, and it requires > 3.8.4
17:59 quest`` and this is part of a packaged up install, so I would much rather use the ppa than build my own
17:59 quest`` as the installer already takes into consideration the 3 different packages, and if I build from source it is far easier to package as 1 deb, but then I need to rework the existing installer logic
17:59 quest`` which is a PITA
18:00 susant joined #gluster
18:01 arpu joined #gluster
18:02 bwerthmann joined #gluster
18:02 quest`` Hola Ben =)
18:02 bwerthmann Hi quest
18:21 kkeithley JoeJulian, quest: actually, everything used to build the PPA is in github, Specifically https://github.com/gluster/glusterfs-debian
18:21 glusterbot Title: GitHub - gluster/glusterfs-debian: Debian packaging of Gluster (at github.com)
18:22 bwerthmann kkeithley: I think quest is looking for the invocation of the debian packaging files
18:26 quest`` yep I have that repo, just wanted to know what commands are executed against it =)
18:27 kkeithley dbuild is used to create .dsc and .source_changes files. On debian I pass the .dsc file to pbuilder to build.
18:28 kkeithley I don't know debian packaging to know what takes a .source_changes file.
18:28 kkeithley s/dbuild/debuild/
18:28 glusterbot What kkeithley meant to say was: debuild is used to create .dsc and .source_changes files. On debian I pass the .dsc file to pbuilder to build.
18:30 kkeithley On Ubuntu I send the _source.changes file to the PPA with dput and the PPA uses that to build
18:32 quest`` ok I think I got it working =)
18:32 quest`` I see libglusterfs building =)
18:32 kkeithley I don't know what dput actually does. I guess it sends the .dsc and the other files
18:33 quest`` so what I did for clarity... 1) cloned the glsuterfs-debian repo. Copied over the following files:
18:33 quest`` glusterfs_3.8.10-ubuntu1~trusty1.debian.tar.xz (13.2 KiB)
18:33 quest`` glusterfs_3.8.10-ubuntu1~trusty1.dsc (2.2 KiB)
18:33 quest`` glusterfs_3.8.10.orig.tar.gz (8.1 MiB)
18:34 quest`` then ran sudo pbuilder --build glusterfs_3.8.10-ubuntu1~trusty1.dsc
18:34 quest`` and it appears to be doing the right thing
18:34 kkeithley yup
18:34 kettlewell left #gluster
18:35 kkeithley well, if you took the .dsc file you can just run pbuilder, you don't need the glusterfs-debian files
18:35 quest`` fair enough =)
18:36 quest`` at this point I just want to test if my compiled versions work, the figure out why the PPA glusterfs-common package is breaking.
18:36 kkeithley yep
18:37 quest`` ok it produced debs =)
18:37 quest`` glusterfs-client_3.8.10-ubuntu1~trusty1_amd64.deb
18:37 quest`` glusterfs-common_3.8.10-ubuntu1~trusty1_amd64.deb
18:37 quest`` glusterfs-dbg_3.8.10-ubuntu1~trusty1_amd64.deb
18:37 quest`` glusterfs-server_3.8.10-ubuntu1~trusty1_amd64.deb
18:38 quest`` ok I can simulate the exact failure I saw from the PPA now
18:38 quest`` =)
18:38 quest`` Installing new version of config file /etc/glusterfs/glusterd.vol ...
18:38 quest`` * Starting glusterd service glusterd                                    [fail]
18:38 quest`` update from 3.7.20 -> 3.8.10 from the PPA or built using those scripts fails
18:38 quest`` now to figure out why =P
18:40 kkeithley might be recent changes to not install stuff in /usr/libexec
18:40 quest`` yeah I noticed there was a commit changing that logic... but the change did look good from my brief look
18:42 quest`` you are refering to https://github.com/gluster/glusterfs-debian/commit/82e06c7f579239c65b66a6d2f3c43d685ac81ae8
18:42 glusterbot Title: glusterfs-3.8.10 GA (trusty) · gluster/glusterfs-debian@82e06c7 · GitHub (at github.com)
18:42 quest`` right?
18:43 kkeithley yes
18:45 quest`` removing the patch, to see if it works
18:45 quest`` btw really appreciate the help =)
18:45 quest`` I know helping randoms on irc isn't the most fun
18:47 kkeithley well, we (meaning me) want to get it right
18:47 major I dunno .. can be tons of fun .. so long as the problem is interesting :)
18:47 quest`` major: fair enough =)
18:48 JoeJulian quest``: Helping randoms on IRC is why most of us are here hanging out. :) This isn't #centos.
18:48 kkeithley ouch
18:48 quest`` BURN
18:49 quest`` Great to see irc communities still thriving. I miss the old days.
18:49 bwerthmann agreed.
18:50 quest`` k with patch removed it does still build as expected =)
18:50 JoeJulian Another shameless plug. If you're in the Seattle area and want to come to a gluster meetup: https://www.meetup.com/Seattle-Storage-Meetup/
18:50 glusterbot Title: Seattle Storage Meetup (Seattle, WA) | Meetup (at www.meetup.com)
18:51 bwerthmann quest``: is the failure a segfault?
18:52 major JoeJulian, yer killing me ;)
18:52 quest`` peer probe still causes glsuterd to dump
18:52 major s/shameless plug/taunting those who wont be in the area to make it/
18:52 quest`` =(
18:52 JoeJulian major: Hey, it's your fault.
18:52 glusterbot major: Error: I couldn't find a message matching that criteria in my history of 1000 messages.
18:52 quest`` I am going to grab lunch, will chat with you all later =)
18:53 JoeJulian major: You're away in mid April?
18:53 bwerthmann I should do the same.
18:53 major kinda yes/no?
18:53 bwerthmann (lunch)
18:53 major my train south-bound leaves at 1805 on Thursdays to go back to visit family :)
18:54 major meetup @ 1630 on thursday .. its like .. kick the pummel of the knife once or twice before twisting it for the 3rd time :(
18:55 JoeJulian Ugh.. Thursdays were the only day that was available.
18:55 major hehe
18:55 major maybe I can make an exception for that day of the month and head home on Friday
18:56 major though .. friday night is also my EMS duty night .. bleh the pain
18:56 JoeJulian We should just get amye to cover Uber. ;P
18:57 major lmao
18:57 major that would be one bloody expensive uber
18:57 amye ... why Uber.
18:57 JoeJulian I'm thinking it would be about $200.
18:57 major lol
18:57 major could lease a car for the month for the cost of that ride
18:57 JoeJulian He lives half way to Portland from here.
18:58 amye Ha
18:58 JoeJulian Oooh, Amye should come to the meetup. She could give you a ride home
18:58 amye .. depends on when it is?
18:58 JoeJulian https://www.meetup.com/Seattle-Storage-Meetup/
18:58 glusterbot Title: Seattle Storage Meetup (Seattle, WA) | Meetup (at www.meetup.com)
18:58 JoeJulian April 15, 6:30pm.
18:58 amye oooh maybe
18:58 major Apr 13th?
18:59 JoeJulian 13
18:59 major pretty certain the 15th is a saturday
18:59 JoeJulian yeah
18:59 amye april 13 is tight but :P
18:59 amye Let me hop off a plane and review schedules.
18:59 JoeJulian You jet setter, you.
18:59 amye God you're telling me. :P
19:00 amye Nah, I'm just attempting to juggle things for that week. No big deal.
19:04 major JoeJulian, should fix the description in that meetup :P
19:04 guhcampos joined #gluster
19:07 susant joined #gluster
19:07 major hold it .. there was an implication there that amye is south of my house..
19:08 amye Typically I am based out of Portland, OR.
19:08 amye This doesn't happen all that often due to $reasons.
19:10 ghenry joined #gluster
19:12 major well .. when you are you are closer to my house than work is by about .. ~30 miles or so I suppose ..
19:12 amye major, if you're interested, drop me an email at amye@redhat.com
19:12 amye Dropping up to Seattle for a new meetup is worth it. ;)
19:13 major yah, I will likely go, but I will already be in the area.  Its the getting home Thursday night that will be a problem for me.. last train south leaves out at 1805
19:14 major likely need to catch the train souh friday
19:14 amye understandable.
19:15 major I am normally here in the Seattle area Sunday evening (or Monday morning depending on how late the train is) until Thursday evening, then its back home to be with family and Fire/EMS duties.
19:16 annettec joined #gluster
19:17 amye Gotcha
19:21 susant left #gluster
19:23 Gambit15 Hey guys
19:25 Gambit15 Just had one of my files start giving an I/O error, so I checked the volume's heal info & noticed the file was listed once on both bricks 1 & 2, and listed twice on brick 3 (arbiter)
19:26 telius joined #gluster
19:26 Gambit15 I'm using dist-rep 2+1, and the files are only ever acessed by one client at a time (VMs)
19:27 Gambit15 File was listed as described for about 30 minutes, and is now down to showing just one entry on ethe arbiter
19:27 Gambit15 The file's still giving an I/O error though
19:30 Gambit15 For the most part, the only thing I'm getting in the bricks' logs are the repeated connections from the host trying to access the file.
19:30 JoeJulian Gambit15: Can you post the ,,(extended attributes) from that file or gfid on each server?
19:30 glusterbot Gambit15: (#1) To read the extended attributes on the server: getfattr -m .  -d -e hex {filename}, or (#2) For more information on how GlusterFS uses extended attributes, see this article: http://pl.atyp.us/hekafs.org/index.php/2011/04/glusterfs-extended-attributes/
19:31 Gambit15 JoeJulian, I forget, is that direct on the brick or on the gluster mount?
19:32 farhoriz_ joined #gluster
19:32 JoeJulian on the brick
19:38 Gambit15 JoeJulian: https://paste.fedoraproject.org/paste/Nz~d5LU6yhcu9eET~c83nV5M1UNdIGYhyRLivL9gydE=
19:38 glusterbot Title: Untitled - Modern Paste (at paste.fedoraproject.org)
19:38 Gambit15 I've also copied in the only message in the brick logs which isn't a connection negotiation
19:39 Gambit15 The volume is a cloned snapshot, however that'll be migrated to a full volume as soon as I'm able. Not sure it makes much difference, but I notice they're stored slightly differently
19:44 JoeJulian Gambit15: That getfattr is the same on every brick?
19:50 Chinorro joined #gluster
19:51 burn joined #gluster
20:00 Gambit15 JoeJulian: https://paste.fedoraproject.org/paste/mq3IdOutCyFdFFpOgO8oEF5M1UNdIGYhyRLivL9gydE=
20:00 glusterbot Title: Fork of Untitled - Modern Paste (at paste.fedoraproject.org)
20:01 Gambit15 brick1: trusted.afr.dirty=0x000000010000000000000000 ... brick2: trusted.afr.dirty=0x000000020000000000000000 ... brick3/arbiter: trusted.afr.dirty=0x000000010000000000000000
20:03 JoeJulian Yeah, but the trusted.afr.*-client don't show anything abnormal. Shows updated from v2 to v1 and v0.
20:03 JoeJulian s/updated/updates/
20:03 glusterbot What JoeJulian meant to say was: Yeah, but the trusted.afr.*-client don't show anything abnormal. Shows updates from v2 to v1 and v0.
20:04 Gambit15 Well the file's still showing in the heal list for the arbiter (but non of the other bricks), and it's still returning an I/O error when trying to access it over the mount
20:05 Gambit15 Running "file" on the file on the brick is returning the expected result
20:05 Gambit15 Any ideas?
20:08 Gambit15 Everything's running v3.8.8
20:10 JoeJulian Depending on how lazy I'm feeling I would either go into all the logs and look at the activity and try to find the problem, maybe make sure the file is readable on all the bricks, or, if I just want it working, I would start with a "gluster volume start $volname force" and see if the replaced shd clears it up.
20:12 jkroon joined #gluster
20:16 Gambit15 Would that reset the current client connections? I've got other VMs running on that volume that I'd rather not interrupt
20:16 Gambit15 The file must be readable on all the bricks, I just read all of their attributes!
20:20 baber joined #gluster
20:22 Gambit15 Huh, getting lots of RPC errors
20:22 Gambit15 E [rpcsvc.c:560:rpcsvc_check_and_reply_error] 0-rpcsvc: rpc actor failed to complete successfully
20:23 JoeJulian No, start force will not interrupt the clients. The attributes are stored in an inode. Only reading the file will tell you if the file is readable. I usually just dd it to /dev/null.
20:27 Gambit15 Looks like something keeps making glusterd disconnect on each of the servers, causing quorum to fail, and everything coming back up & regaining quorum 10 seconds later
20:28 Gambit15 Ugh, I still haven't been able to fully investigate that old, bizzare, disconnect issue we discussed a month ago :(
20:28 Gambit15 Been a right thorn in my side, but had no time so far...
20:31 farhorizon joined #gluster
20:35 Gambit15 The first 6GB of the file on each of the bricks read without error, although not keen on waiting for the full 300
20:35 quest`` So I have replicated the issue I am seeing in 3.8.20 with trusty. Even when I build the packages myself they have the same error as in the PPA... but when I compile directly from source it works as expected...
20:35 quest`` 3.8.10 rather
20:37 JoeJulian Gambit15: Yeah, I'm sure that's enough.
20:38 Gambit15 "gluster volume start $volname force" - is that required on just one of the peers, or all of them?
20:38 Gambit15 Tried it on two peers (one of which is the main client), but no dice, still giving I/O errors
20:46 quest`` also notice in the build log
20:46 quest`` dpkg-gencontrol: warning: Depends field of package glusterfs-client: unknown substitution variable ${shlibs:Depends}
20:46 quest`` so it looks like dependencies for shared libs are not setup right
20:55 bluenemo joined #gluster
21:06 farhorizon joined #gluster
21:11 shyam joined #gluster
21:16 jobewan joined #gluster
21:38 quest`` anyone want to help me debug the failures? can't seem to track down why the packages don't work, but a compilation manually works fine on the exact same box =\
21:40 BuBU29 joined #gluster
21:47 quest`` Noticing that when peer probe crashes... the UUID of the peer is totally wrong
21:47 quest`` 00000000-0000-0000-0000-000000000000
21:49 JoeJulian Gambit15: Was afk grabbing lunch. The only think I _know_ can cause that is split-brain but your files are clearly not split-brain.
21:50 JoeJulian Gambit15: If you make a new mount somewhere else, can you access the file there?
21:55 Wizek_ joined #gluster
21:58 farhorizon joined #gluster
21:58 cholcombe joined #gluster
22:27 Gambit15 JoeJulian, nope!
22:27 JoeJulian Post the whole new client file from the new mount.
22:27 JoeJulian s/client file/client log files/
22:27 glusterbot What JoeJulian meant to say was: Post the whole new client log files from the new mount.
22:28 JoeJulian where'd that 's' come from? Gah!
22:28 bwerthmann joined #gluster
22:32 quest`` yeah with or without the libexec patch I am getting segfaults in trusty with 3.8.10 =(
22:32 quest`` after the debs are built, do you guys actually test them on live environments?
22:33 quest`` it launches but crashes on first peer probe
22:33 quest`` uuid=00000000-0000-0000-0000-000000000000
22:33 quest`` =\
22:34 Gambit15 JoeJulian: https://paste.fedoraproject.org/paste/6faigPXBHQpQTMvXLqrwL15M1UNdIGYhyRLivL9gydE=
22:34 glusterbot Title: Untitled - Modern Paste (at paste.fedoraproject.org)
22:36 JoeJulian I know I don't test anything on Ubuntu. Does launchpad even have any test integration?
22:36 * quest`` hates ubuntu, just stuck supporting a product on trusty =(
22:36 quest`` but something seriously regressed from 3.7.x to 3.8.x
22:36 Vaelatern joined #gluster
22:37 quest`` I just don't see anything obvious
22:37 quest`` and it definitely isn't the latest patch you added for libexec handling
22:37 Gambit15 FYI, this "client" (v2/s2) hosts one of the bricks for the volume, however it wasn't mounting it.
22:37 quest`` as I removed that and rebuilt it all
22:37 quest`` on a happier note, at least I know how to build PPA's locally now ;)
22:38 JoeJulian Gambit15: Oh! ok.... it is detecting split-brain. I guess I would reset trusted.afr.dirty on v2 and see if that allows it to clean up.
22:38 quest`` What I found is that it installs ok, seems to be stable, but the moment you do the first peer probe, it segfaults, and corrupts /var/lib/glusterd to the point it wont start again
22:38 quest`` if you clear that dir, it starts fine
22:39 quest`` until you actually, you know, add a peer
22:39 Gambit15 I ran a manual "heal $vol" a while ago, but at no point has "heal $vol info split-brain" ever shown anything...
22:39 Gambit15 Is that not trustworthy?
22:40 Gambit15 I noticed the errors in the log though
22:42 quest`` is this related? https://bugzilla.redhat.com/show_bug.cgi?id=1387564
22:42 glusterbot Bug 1387564: high, unspecified, ---, bugs, CLOSED CURRENTRELEASE, [Eventing]: UUID is showing zeros in the event message for the peer probe operation.
22:42 JoeJulian Gambit15: I've never seen it to be not trustworthy. After we get you working again, maybe we can figure out if you've found a new bug.
22:43 quest`` and any chance you will get trusty packages up for 3.9 or 3.10?
22:43 Gambit15 Coincidentaly, the VM affected is the one hosting my documentation :( Could you pass me the command to reset the attribute?
22:44 JoeJulian quest``: As for "do you guys..." anything, I'm just hanging out here why I'm doing my $dayjob. I help where I can and do what I have time to do. There's always room for more community members to contribute.
22:44 JoeJulian s/why/while/
22:44 glusterbot What JoeJulian meant to say was: quest``: As for "do you guys..." anything, I'm just hanging out here while I'm doing my $dayjob. I help where I can and do what I have time to do. There's always room for more community members to contribute.
22:44 JoeJulian and, apparently, I need to go back to typing school.
22:44 * Gambit15 really needs to get around to mirroring his wiki VM
22:44 quest`` kk... well I figure the maintainers of the gluster ppa might be here =)
22:44 JoeJulian I'm one of them.
22:45 quest`` KK, any plans to add 3.9 or 3.10 packages for trusty? Or is there a way I can assist?
22:45 quest`` with the PPA I don't have your keys, can't sign them, etc
22:45 quest`` hehe
22:46 JoeJulian the ppa keys are launchpad's.
22:47 JoeJulian It's been a while since I looked at how to create a new build.
22:47 JoeJulian If you want to figure that out, I'm sure we can get you added to the members list.
22:48 Gambit15 JoeJulian: setfattr -n trusted.afr.dirty -v 0x000000000000000000000000 ?
22:49 JoeJulian Yes.
22:49 Gambit15 Done
22:49 Gambit15 Now?
22:50 JoeJulian probably
22:55 Gambit15 Should /var/log/glusterfs/glfsheal-data2.log show me the current heal status?
22:56 Gambit15 'cause I ain't getting anything useful in there!
22:57 JoeJulian Could be any of them that picks up the heal, or even a client. I see you have all the client-side self-heal options actively enabled.
23:03 Wizek_ joined #gluster
23:08 farhorizon joined #gluster
23:09 Gambit15 Right, cheers then. Heading home now, but will have a look see if anything's happened when I arrive
23:10 Gambit15 If there's no progress, what would be your next suggestion? (I'll check your replies when I get in, also)
23:10 Gambit15 I find it bizzare & frustrating that my arbiter repairs are *never* quick :(
23:12 JoeJulian Gambit15: I would zero out another (probably the arbiter).
23:13 vinurs joined #gluster
23:37 nishanth joined #gluster
23:42 quest`` just for fun I built the 3.7.20 packages from scratch, and they work fine. Going to see if any 3.8 builds work and get more details about where the problem was introduced
23:45 JoeJulian +1
23:45 quest`` nm, they don't archive the old dsc files =(
23:45 quest`` unless I am not looking in the right place
23:45 quest`` but I can't find the .dsc files for older 3.8 builds
23:46 JoeJulian Oh, right. I don't think launchpad does that.
23:46 quest`` I hate debs
23:46 JoeJulian +1
23:46 quest`` their packaging is overly complex and a complete pain in the ass
23:47 JoeJulian Wait... trusty, right?
23:47 quest`` yeah
23:47 JoeJulian Didn't you say that was missing for 3.8?
23:47 quest`` there is a dsc for 3.8.10
23:47 JoeJulian Oh, no, you said 3.10 I think.
23:47 quest`` but none of the earlier builds
23:48 quest`` no ppa at all for 3.9 and 3.10 for trusty
23:48 JoeJulian 3.9 is eol
23:48 quest`` yep
23:48 quest`` I was hoping to jump to 3.10
23:48 quest`` as it fixed a few uuid issues
23:48 quest`` and that is what I am seeing in 3.8.10
23:48 quest`` all UUIDs are being written as 0's
23:48 quest`` and then it segfaults
23:49 quest`` I just need to stop being lazy and just build 3.10 from scratch and build my own packages
23:49 quest`` I just really didn't want to add more complexity to my release process
23:50 quest`` but I guess I have no choice =\
23:50 JoeJulian I'm looking to see what I need to re-learn
23:50 quest`` every time I build debs I have to read the docs again
23:50 quest`` and of course they have like 10 tools that do similar things
23:50 quest`` so i feel your pain =P
23:54 JoeJulian quest``: If you click on the "166 successful" builds, you can get to the old ones.
23:54 quest`` ooh nice and the artifacts are there?
23:54 quest`` I just saw logs
23:54 quest`` no artifacts
23:55 quest`` for instance
23:55 quest`` https://launchpad.net/~gluster/+archive/ubuntu/glusterfs-3.7/+build/10688959
23:55 glusterbot Title: i386 build of glusterfs 3.7.15-ubuntu1~trusty1 : glusterfs-3.7 : “Gluster” team (at launchpad.net)
23:55 quest`` I see it list the ones it should have built, but the build artifacts are empty
23:56 JoeJulian hmm
23:56 quest`` Did I miss a link?
23:56 JoeJulian Not that I see. Looks like launchpad deletes legacy binaries.
23:57 quest`` kk... I am building 3.10 from source now
23:59 quest`` nope.... ugly compile errors =(
23:59 JoeJulian I imagine that's why there's no trusty build.

| Channels | #gluster index | Today | | Search | Google Search | Plain-Text | summary