Perl 6 - the future is here, just unevenly distributed

IRC log for #gluster, 2015-02-25

| Channels | #gluster index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:00 JustinClift elitecoder: You're welcome.  It was all JoeJulian's effort. :)
00:01 JoeJulian excellent
00:01 elitecoder JustinClift: How's their support? lol, that's pretty much what they focused on when my co-workers were in a call with them. AWS's pricing blows them out of the water, we don't really need sysadmins much
00:01 elitecoder we found a sysadmin company that deals exclusively with AWS for such a good price so
00:01 misc JustinClift: good, good so i can add them in salt ( who just add ssh keys and yum-cron ) without fear of breaking stuff
00:01 elitecoder we'z doin that
00:02 elitecoder Though, files are still missing so I'm a little ... curious about wtf happened there.
00:03 elitecoder But maybe in the future, I'll be able to fix issues a little quicker. Maybe they managed to disappear due to reboots without fixing them or who knows what combination of events
00:03 JustinClift elitecoder: Rackspace support is generally decent
00:03 JustinClift elitecoder: Most of the issues we've raised with them have had sensible people answer
00:03 misc yep, same here, there is a human on the line, and one who is not pressured to take 45 calls per minutes
00:04 JustinClift elitecoder: Sometimes when asking them tricky things (eg sysctl variables weirdness in our specific situation) it hasn't yet been resolved properly
00:04 JustinClift elitecoder: But that's unusual
00:04 elitecoder mm
00:04 misc JustinClift: to be fair, if they can't reproduce, not sure if we could do as well in house :)
00:04 JustinClift elitecoder: I've not heard anything positive about AWS support however
00:04 sprachgenerator joined #gluster
00:04 elitecoder We don't use their support
00:04 JustinClift ... even from people who pay for it
00:04 JustinClift Yeah
00:05 kminooie JoeJulian:  I replaced get_volfile with both api_get_volfile and cli_get_volfile  and got the same error in both cases which makes me believe that rpc file is not being included some how? maybe? my python is really rusty
00:05 elitecoder Just because we don't need it..
00:05 JustinClift Translate that to = I've heard some extremely negative things about AWS support, from people that pay for it
00:05 JustinClift Yeah
00:05 JustinClift ;)
00:06 PorcoAranha joined #gluster
00:06 PorcoAranha joined #gluster
00:06 JoeJulian kminooie: What commit are you on?
00:07 JustinClift misc: slave25 is back up, and has the sysctl adjustment commented out
00:07 T3 joined #gluster
00:07 JoeJulian Oh, wait...
00:07 kminooie 742353373a85ae4a5     "Just use whatever "env" shows up in the path"
00:07 JoeJulian You installed. I didn't.
00:07 JoeJulian I bet that's it.
00:07 * JustinClift considers renaming the "slave" nodes to "ozconvict" instead
00:07 kminooie i didn't install . just ran from repo
00:07 JustinClift Us being so PC after all :)
00:08 misc JustinClift: isn't 'convict' already included in 'oz' ?
00:08 JustinClift Mayb
00:10 misc so they are all 6.6 centos ?
00:12 elitecoder JustinClift: wow that's too bad -.- (aws support)
00:12 JustinClift misc: http://bash.org/?262417
00:13 misc JustinClift: ah ah
00:13 JustinClift :D
00:14 JustinClift For anyone listening in, I'm Australian ;)
00:15 JustinClift Gah
00:16 sprachgenerator joined #gluster
00:16 JustinClift Power keeps dropping out at my place
00:16 misc the charm of the countryside
00:16 kkeithley_ and you left Oz for Blighty?
00:17 misc next time, you are gonna complain that you have to walk 0.5 miles to get drinkable water
00:18 kkeithley_ JustinClift: do you know the magic for bypassing checkpatch.pl?
00:19 JustinClift kkeithley_: No :(
00:19 JustinClift I got hit with checkpatch problems one time, and never ended up resubmitting
00:19 JustinClift kkeithley_: Well, Blighty seemed like a good idea at the time :)
00:22 elitecoder JoeJulian: Remember on like friday we were talking about how a file couldn't just disappear?
00:22 gildub joined #gluster
00:23 JoeJulian yes
00:23 elitecoder I found an error message in my logs, sandwiched between a 200 http request, and a 404 http request
00:23 elitecoder [2015-02-03 23:54:29.418647] I [server-rpc-fops.c:727:server_removexattr_cbk] 0-files-server: 19451048: REMOVEXATTR /files/secure/uploads/vqy11sPHpfZ​REeYdHCxCKELiXg20AJD1AKkEkOWJ.png (4dbab086-81e4-43f5-a131-8b159f6fbf8e) of key security.ima ==> (No data available)
00:23 elitecoder two of these, slightly different, but same filename
00:24 misc mouahah half of the slave are now in salt
00:24 elitecoder Does that mean anything to you?
00:25 JoeJulian Means it couldn't remove the xattr. Probably because the file wasn't there?
00:26 elitecoder Yeah that's all I gathered from it too =\ was hoping you had more insight... maybe a bug related to when that shows up
00:27 JoeJulian I'm trying to figure out how a disk with a 4TB xfs partition, shows up with a 1TB ext4 partition after someone rebooted the machine that has it.
00:27 JoeJulian Not possible, I said.
00:28 elitecoder LOL
00:29 JoeJulian Yeah, the drive just happened to change size and somehow established a valid alternative? Must have been the most lucky coincidence of cosmic particle bombardment ever.
00:29 JustinClift JoeJulian: Is there a DVD in the attached drive?
00:29 * JustinClift is thinking some kind of auto-configure thingy
00:30 JoeJulian Heh, nope.
00:30 JustinClift Installer of some kind maybe
00:30 JustinClift k
00:30 JoeJulian pxe booted hosts with 30x4TB drives in knox trays.
00:30 JustinClift PXE booted hosts
00:30 JoeJulian (it's pretty awesome actually)
00:30 JustinClift So, look through the PXE boot logs and see what the box actaully booted
00:31 JoeJulian I can't wait to do a preso on this setup.
00:31 JustinClift Because it sounds like it picked up something wrong to boot
00:31 JoeJulian Nah, it has to be a human.
00:31 JustinClift If you say so :)
00:32 JoeJulian That's why I make the big bucks.
00:32 JustinClift :D
00:32 JustinClift I used to have PXE booting IB hosts
00:32 JustinClift Was kinda awesome in one regard, and a nightmare in another regards
00:32 JoeJulian I wish I could tell the whole thing, but I'm not sure I should have even said that much yet.
00:33 JustinClift Needed to specially rebuild kernels to include the right drivers every time there was a kernel update
00:33 JustinClift But it was *fast*
00:33 JustinClift (booting off IB over raided ssd)
00:33 JoeJulian Nice
00:34 JustinClift Hopefully it'll be less of a maintenance nightmare to get that working again soon
00:34 JustinClift Now that SCST is going into CentOS Storage SIG, it'll mean a real no-maintance source of the right kernels
00:34 JustinClift In theory ;)
00:40 krypton joined #gluster
00:41 kripper joined #gluster
00:47 JoeJulian kminooie: The only possible reason I can think of for you to get that error is if it's loading a different rpc module. I thought the python path should include the current directory first, though.
00:50 harish_ joined #gluster
00:55 elitecoder lllater guys
00:55 JoeJulian later
00:57 elitecoder left #gluster
01:01 kminooie so just for the heck of it I did install it and the error message that  I get is different  http://ur1.ca/jslky  line 55 is 2 line after the call to get_volfile  ( I guess something messed up with my env, thou I can't think of what would have the same effect on both fedora and debian ) the install was on fedora
01:01 kminooie JoeJulian: ^^^
01:05 kripper I'm getting "Transport endpoint is not connected" errors. Any hint to debug the problem?
01:06 twx joined #gluster
01:07 JoeJulian That would be a network issue.
01:07 JoeJulian Or a service not running, check "gluster volume status"
01:07 kripper JoeJulian: mount and unmount should fix it?
01:07 kripper https://bugzilla.redhat.co​m/show_bug.cgi?id=1058300
01:07 glusterbot Bug 1058300: unspecified, unspecified, ---, bugs, NEW , VMs do not resume after paused state and storage connection to a gluster domain (they will also fail to be manually resumed)
01:07 kripper it's causing problems to oVirt and libvirt
01:07 JoeJulian kripper: unmount and mount won't fix a broken network, if that's the problem.
01:08 kripper it's a local no-replica gluster mount
01:08 kripper gluster volume status report it as online
01:08 sprachgenerator joined #gluster
01:08 mkzero_ joined #gluster
01:08 tru_tru_ joined #gluster
01:08 kripper I can access the mount, but libvirt is receiving this message
01:12 kripper I think I need to find out what file is accessing libvirt
01:12 kripper the qemu process is open
01:12 kripper maybe it has an obsolete file-descriptor open
01:12 kripper since the problem occurs after stopping and restarting the gluser volume
01:12 kripper lsof reports:
01:12 kripper qemu-kvm 2205 qemu   11u  unknown                                       /b55d0462-0dec-41a3-b0fd-6225fc5cf248/im​ages/4d72e187-01d8-4b6a-b343-68226328c12​e/722d759b-6cdf-4692-91a7-9b0e4b5f89ed (stat: No such file or directory)
01:12 bala joined #gluster
01:13 JoeJulian qemu is using libgfapi, though, isn't it?
01:14 JoeJulian I don't know for sure.
01:16 ThatGraemeGuy joined #gluster
01:16 ThatGraemeGuy joined #gluster
01:18 kripper yes
01:18 kripper JoeJulian: I dicovered the bug
01:18 kripper it is not gluster related
01:18 kripper libvirt or oVirt are trying to access a wrong path
01:18 kripper maybe through libgfapi
01:18 kripper but it's not gluster's fault
01:21 kbyrne joined #gluster
01:24 kripper thanks!
01:24 JoeJulian Excellent find.
01:24 crashmag joined #gluster
01:24 harmw joined #gluster
01:24 mbelaninja joined #gluster
01:24 Arrfab joined #gluster
01:24 Peanut joined #gluster
01:24 crashmag joined #gluster
01:28 _Bryan_ joined #gluster
01:52 crashmag joined #gluster
01:52 Peanut joined #gluster
01:52 Arrfab joined #gluster
01:52 mbelaninja joined #gluster
01:52 harmw joined #gluster
01:52 bala joined #gluster
01:52 tru_tru_ joined #gluster
01:52 mkzero_ joined #gluster
01:52 sprachgenerator joined #gluster
01:52 twx joined #gluster
01:52 kripper joined #gluster
01:52 krypton joined #gluster
01:52 gildub joined #gluster
01:52 MugginsM joined #gluster
01:52 Rapture joined #gluster
01:52 DV joined #gluster
01:52 badone_ joined #gluster
01:52 wkf joined #gluster
01:52 jobewan joined #gluster
01:52 swebb joined #gluster
01:52 PaulCuzner joined #gluster
01:52 rwheeler joined #gluster
01:52 toti joined #gluster
01:52 spiette joined #gluster
01:52 fubada joined #gluster
01:52 sputnik13 joined #gluster
01:52 T0aD joined #gluster
01:52 victori joined #gluster
01:52 y4m4_ joined #gluster
01:52 jackdpeterson joined #gluster
01:52 bennyturns joined #gluster
01:52 ildefonso joined #gluster
01:52 bitpushr joined #gluster
01:52 wushudoin joined #gluster
01:52 Leildin joined #gluster
01:52 side_control joined #gluster
01:52 rotbeard joined #gluster
01:52 ira joined #gluster
01:52 the-me joined #gluster
01:52 huleboer joined #gluster
01:52 maveric_amitc_ joined #gluster
01:52 rp__ joined #gluster
01:52 brad[] joined #gluster
01:52 uebera|| joined #gluster
01:52 ron-slc_ joined #gluster
01:52 bfoster joined #gluster
01:52 snewpy joined #gluster
01:52 asku joined #gluster
01:52 shaunm joined #gluster
01:52 7YUAAFKZG joined #gluster
01:52 7GHAAYHNJ joined #gluster
01:52 R0ok_ joined #gluster
01:52 papamoose joined #gluster
01:52 stickyboy joined #gluster
01:52 nhayashi joined #gluster
01:52 codex joined #gluster
01:52 haomai___ joined #gluster
01:52 siel joined #gluster
01:52 lezo joined #gluster
01:52 owlbot joined #gluster
01:52 sadbox joined #gluster
01:52 al joined #gluster
01:52 johnnytran joined #gluster
01:52 ws2k3 joined #gluster
01:52 raz joined #gluster
01:52 glusterbot joined #gluster
01:52 Intensity joined #gluster
01:52 atrius joined #gluster
01:52 unixfg joined #gluster
01:52 weykent joined #gluster
01:52 NuxRo joined #gluster
01:52 VeggieMeat joined #gluster
01:52 a2 joined #gluster
01:52 cfeller joined #gluster
01:52 masterzen joined #gluster
01:52 Andreas-IPO joined #gluster
01:52 jvandewege joined #gluster
01:52 sage joined #gluster
01:52 vincent_vdk joined #gluster
01:52 Sjors joined #gluster
01:52 edong23 joined #gluster
01:52 abyss^ joined #gluster
01:52 edwardm61 joined #gluster
01:52 misc joined #gluster
01:52 telmich joined #gluster
01:52 purpleidea joined #gluster
01:52 necrogami joined #gluster
01:52 JonathanD joined #gluster
01:52 malevolent joined #gluster
01:52 tdasilva joined #gluster
01:52 ghenry joined #gluster
01:52 scuttlemonkey joined #gluster
01:52 yoavz joined #gluster
01:52 Slasheri_ joined #gluster
01:52 m0zes joined #gluster
01:52 pdrakeweb joined #gluster
01:52 xavih joined #gluster
01:52 mrEriksson joined #gluster
01:52 delhage joined #gluster
01:52 dblack joined #gluster
01:52 devilspgd joined #gluster
01:52 yosafbridge joined #gluster
01:52 xaeth_afk joined #gluster
01:52 _br_ joined #gluster
01:52 afics joined #gluster
01:52 atrius` joined #gluster
01:52 javi404 joined #gluster
01:52 cornus_ammonis joined #gluster
01:52 bjornar joined #gluster
01:52 msvbhat joined #gluster
01:52 khanku joined #gluster
01:52 edualbus joined #gluster
01:52 JordanHackworth joined #gluster
01:52 mdavidson joined #gluster
01:52 tessier joined #gluster
01:52 verdurin joined #gluster
01:52 cyberbootje joined #gluster
01:52 ckotil joined #gluster
01:52 michatotol joined #gluster
01:52 eryc_ joined #gluster
01:52 nixpanic joined #gluster
01:52 Bardack joined #gluster
01:52 DJClean joined #gluster
01:52 lanning joined #gluster
01:52 Dave2 joined #gluster
01:52 Guest52518 joined #gluster
01:52 RobertLaptop joined #gluster
01:52 mator joined #gluster
01:52 coreping joined #gluster
01:52 gothos joined #gluster
01:52 kke_ joined #gluster
01:52 JoeJulian joined #gluster
01:52 verboese|sleep joined #gluster
01:52 oxidane_ joined #gluster
01:52 Kins joined #gluster
01:52 Bosse joined #gluster
01:52 pedrocr joined #gluster
01:52 mikedep333 joined #gluster
01:52 [o__o] joined #gluster
01:52 tberchenbriter_ joined #gluster
01:52 Folken_ joined #gluster
01:52 PinkFreud joined #gluster
01:52 saltsa joined #gluster
01:52 johnmark joined #gluster
01:52 dockbram joined #gluster
01:52 johnbot joined #gluster
01:52 raatti joined #gluster
01:52 msciciel joined #gluster
01:52 fyxim joined #gluster
01:52 jessexoc joined #gluster
01:52 kkeithley joined #gluster
01:52 ndevos joined #gluster
01:52 marcoceppi joined #gluster
01:52 Ramereth joined #gluster
01:52 XpineX joined #gluster
01:52 Telsin joined #gluster
01:52 frankS2 joined #gluster
01:52 kalzz joined #gluster
01:52 Champi joined #gluster
01:52 kenansulayman joined #gluster
01:52 primusinterpares joined #gluster
01:52 Lee- joined #gluster
01:52 semiosis joined #gluster
01:52 obnox joined #gluster
01:52 mibby joined #gluster
01:52 partner joined #gluster
01:52 puiterwijk joined #gluster
01:52 Arminder joined #gluster
01:52 Rydekull joined #gluster
01:52 samsaffron___ joined #gluster
01:52 capri joined #gluster
01:52 klaas joined #gluster
01:52 lkoranda joined #gluster
01:52 Gugge joined #gluster
01:52 JamesG joined #gluster
01:52 ur__ joined #gluster
01:52 and` joined #gluster
01:52 sauce joined #gluster
01:52 dastar joined #gluster
01:52 suliba joined #gluster
01:52 _Bryan_ joined #gluster
02:04 sac`away joined #gluster
02:04 h4rry joined #gluster
02:12 harish_ joined #gluster
02:13 jmarley joined #gluster
02:16 sac`away joined #gluster
02:16 h4rry joined #gluster
02:16 harish_ joined #gluster
02:18 kminooie joined #gluster
02:18 kminooie what happened?
02:21 kripper JoeJulian: The issue is still not clear. I was confused because lsof reported wrong paths.
02:22 kripper JoeJulian: My guess is that qemu keeps filehandlers open pointing to the gluster storage
02:22 JoeJulian That does seem pretty likely.
02:22 kripper JoeJulian: when the volume is restarted, these file handlers are invalid
02:22 kripper JoeJulian: and give a "Transport endpoint is not connected"
02:23 JoeJulian Oh, you're restarting the whole volume, or just one server?
02:23 kripper JoeJulian: the whole volume
02:23 kripper JoeJulian: but there are probably other ways to reproduce
02:23 JoeJulian Ah, well then yes. The fd cannot be maintained if all the servers are stopped.
02:23 kripper JoeJulian: for example, a network problem
02:24 JoeJulian I wouldn't expect a network problem to cause that though.
02:24 kripper JoeJulian: hmm...then I'm testing the wrong way
02:24 kripper http://lists.ovirt.org/piperma​il/users/2014-June/025157.html
02:25 kripper A few cases this can be reproduced:  - switch dies - cable unplug - gluster volume stop - gluster brick dies
02:25 kripper same symptom, other causes
02:26 kripper JoeJulian: please explain me someting
02:26 JoeJulian Make sure to file a bug with gluster then. Perhaps there's a bug in libgfapi.
02:26 glusterbot https://bugzilla.redhat.com/en​ter_bug.cgi?product=GlusterFS
02:26 kripper JoeJulian: if a VM is moved from H1 to H2, on H2 it should connect to h2:/gluster instead of h1:/gluster?
02:27 kripper JoeJulian: I know both will work, but isn't it better to access the gluster server on the same host?
02:27 JoeJulian no
02:27 JoeJulian @mount host
02:27 JoeJulian @factoid mount host
02:27 JoeJulian hmm, must part of that network problem they mentioned.
02:29 kripper ?
02:29 kripper didn't get the answer
02:29 JoeJulian You only specify a mount host to retrieve the volume configuration. After that the client connect directly to the necessary hosts.
02:31 kripper if VM was connected to h1:/gluster, once moved to H2 it will use the URI "h1:/gluster" to get the volume configuration and work as if it was mounted as h2:/gluster?
02:32 kripper is there any difference if I mount h1:/gluster or h2:/gluster?
02:32 kripper or is it exactly the same?
02:32 JoeJulian exactly the same
02:32 kminooie I got booted. JoeJulian: did you see my last post about 'gluster system getspec' not working in 3.6.2  (or is it just me )?
02:32 kripper JoeJulian: nice
02:33 JoeJulian In fact, I use rrdns to map "gluster" to all of my servers so I can just mount gluster:myvol and it'll pick the first one that responds.
02:33 JoeJulian kminooie: it should be using the api. The api has that command since 3.6.0.
02:34 JoeJulian ok, I've got to go eat dinner with the family. Goodnight.
02:34 kminooie right but api_get_volfile function returns 0 ( number zero )
02:34 kminooie night
02:34 kripper JoeJulian: goodnight. thanks you very much!
02:38 kminooie I am out here as well. have a good night everyone ( or day, where ever everyone is )
02:48 ilbot3 joined #gluster
02:48 Topic for #gluster is now Gluster Community - http://gluster.org | Patches - http://review.gluster.org/ | Developers go to #gluster-dev | Channel Logs - https://botbot.me/freenode/gluster/ & http://irclog.perlgeek.de/gluster/
02:50 victori joined #gluster
02:52 MugginsM joined #gluster
02:56 aravindavk joined #gluster
03:10 Peanut joined #gluster
03:19 bharata-rao joined #gluster
03:20 rjoseph joined #gluster
03:27 atalur joined #gluster
03:28 dgandhi joined #gluster
03:43 geerlingguy joined #gluster
03:44 nishanth joined #gluster
03:47 atinmu joined #gluster
03:48 prasanth_ joined #gluster
03:49 itisravi joined #gluster
03:54 Scotch joined #gluster
04:02 shubhendu joined #gluster
04:06 jobewan joined #gluster
04:09 gem joined #gluster
04:17 soumya joined #gluster
04:29 RameshN joined #gluster
04:29 nbalacha joined #gluster
04:30 victori joined #gluster
04:35 anoopcs joined #gluster
04:36 jiffin joined #gluster
04:37 badone__ joined #gluster
04:41 schandra joined #gluster
04:41 deepakcs joined #gluster
04:44 aravindavk joined #gluster
04:44 ppai joined #gluster
04:48 ndarshan joined #gluster
04:48 sputnik13 joined #gluster
04:54 kripper joined #gluster
04:58 rafi joined #gluster
05:09 lalatenduM joined #gluster
05:12 dusmant joined #gluster
05:24 jobewan joined #gluster
05:24 rjoseph joined #gluster
05:28 PaulCuzner joined #gluster
05:30 kumar joined #gluster
05:32 Manikandan joined #gluster
05:32 Manikandan_ joined #gluster
05:38 kdhananjay joined #gluster
05:40 hagarth joined #gluster
05:41 anrao joined #gluster
05:47 spandit joined #gluster
05:49 side_control joined #gluster
05:50 kshlm joined #gluster
05:56 glusterbot News from newglusterbugs: [Bug 1196019] Any op on files in the root directory of the volume fails unless absolute path is specified. <https://bugzilla.redhat.co​m/show_bug.cgi?id=1196019>
05:58 overclk joined #gluster
06:00 vimal joined #gluster
06:01 ppai joined #gluster
06:09 atalur joined #gluster
06:10 dusmant joined #gluster
06:12 sprachgenerator joined #gluster
06:13 atalur_ joined #gluster
06:14 raghu joined #gluster
06:17 Guest15282 joined #gluster
06:26 nshaikh joined #gluster
06:26 glusterbot News from resolvedglusterbugs: [Bug 1194596] [geo-rep]: In a mountbroker setup, set_geo_rep_pem_keys.sh fails to copy keys to its slave peers. <https://bugzilla.redhat.co​m/show_bug.cgi?id=1194596>
06:27 Guest15282 joined #gluster
06:33 victori joined #gluster
06:35 kanagaraj joined #gluster
06:40 nangthang joined #gluster
06:43 nangthang joined #gluster
06:48 aravindavk joined #gluster
06:49 rjoseph joined #gluster
06:51 atinmu joined #gluster
06:55 mbukatov joined #gluster
06:56 glusterbot News from newglusterbugs: [Bug 1196027] Fix memory leak while using scandir <https://bugzilla.redhat.co​m/show_bug.cgi?id=1196027>
06:56 glusterbot News from newglusterbugs: [Bug 1196028] libgfapi: glfs_init() hangs on pthread_cond_wait() when user is non-root <https://bugzilla.redhat.co​m/show_bug.cgi?id=1196028>
07:00 lalatenduM joined #gluster
07:00 doekia joined #gluster
07:02 spandit joined #gluster
07:02 lalatenduM joined #gluster
07:04 fattaneh joined #gluster
07:09 prasanth_ joined #gluster
07:13 soumya joined #gluster
07:13 fattaneh left #gluster
07:24 atinmu joined #gluster
07:25 bala joined #gluster
07:34 rjoseph joined #gluster
07:40 soumya joined #gluster
07:45 tru_tru joined #gluster
07:46 spandit joined #gluster
07:48 LebedevRI joined #gluster
08:20 kovshenin joined #gluster
08:29 gildub joined #gluster
08:30 soumya_ joined #gluster
08:31 ctria joined #gluster
08:38 [Enrico] joined #gluster
08:38 prasanth_ joined #gluster
08:38 SOLDIERz joined #gluster
08:40 kripper joined #gluster
08:54 Slashman joined #gluster
09:00 liquidat joined #gluster
09:10 ctria joined #gluster
09:23 Norky joined #gluster
09:32 ctria joined #gluster
09:32 ppai joined #gluster
09:36 [Enrico] joined #gluster
09:36 Manikandan joined #gluster
09:37 soumya__ joined #gluster
09:37 schandra joined #gluster
09:48 social joined #gluster
10:11 SOLDIERz joined #gluster
10:14 dusmant joined #gluster
10:18 ricky-ticky joined #gluster
10:19 spandit joined #gluster
10:21 ricky-ticky1 joined #gluster
10:22 gildub joined #gluster
10:27 kovshenin joined #gluster
10:33 ndevos REMINDER: in about 90 minutes from now, the weekly Gluster Community meeting starts in #gluster-meeting
10:41 SOLDIERz_ joined #gluster
10:42 harish_ joined #gluster
10:45 kovshenin joined #gluster
10:50 itisravi joined #gluster
10:53 schandra joined #gluster
10:53 firemanxbr joined #gluster
10:54 T0aD joined #gluster
11:01 Pupeno joined #gluster
11:02 meghanam joined #gluster
11:10 kripper joined #gluster
11:12 spandit joined #gluster
11:19 bene2 joined #gluster
11:21 sputnik13 joined #gluster
11:22 hagarth joined #gluster
11:30 kripper left #gluster
11:32 kovshenin joined #gluster
11:39 itpings joined #gluster
11:39 itpings hi guys
11:39 itpings so my questions remains the same
11:40 itpings has my post been proof checked
11:52 hagarth joined #gluster
11:53 kovshenin joined #gluster
11:54 itisravi joined #gluster
11:58 soumya_ joined #gluster
11:58 enseven joined #gluster
12:00 ndevos REMINDER: Gluster Community Meeting starts now in #gluster-meeting
12:00 jdarcy joined #gluster
12:01 Apeksha joined #gluster
12:02 enseven joined #gluster
12:06 enseven Hi all! Is there a possibility to restart a brick i.e. the glusterfsd process without restarting the whole node?
12:09 ira joined #gluster
12:13 kovshenin joined #gluster
12:14 Debloper joined #gluster
12:19 kovshenin joined #gluster
12:22 ndevos enseven: kill the glusterfs process and 'gluster volume start $VOLNAME force' ?
12:27 SOLDIERz_ joined #gluster
12:27 glusterbot News from newglusterbugs: [Bug 1196161] libgfapi-python: Errno not populated properly by ctypes <https://bugzilla.redhat.co​m/show_bug.cgi?id=1196161>
12:36 enseven ndevos: Is this possible while a gluster file system ist mounted on the node? Wouldn't this unmount or break the gluster client and get processes accessing the gluster file system in real trouble? Btw.: I have 3 nodes, containing each one distributed brick of each gluster filesystem and having them mounted through a gluster client. So I'd like to restart the bricks on one of the nodes while continiously running the client and doing i/o on the g
12:36 enseven luster file systems, which then has to go to the other nodes only and then be healed.
12:43 ricky-ticky joined #gluster
12:52 JustinClift enseven: If there's no replication, that sounds like it's going to have a bad result
12:52 JustinClift :/
12:54 monotek1 joined #gluster
12:55 enseven JustinClift: Oh, sorry! They are not distributed, they are replicated. I have all my data on glusterfs tripple. ;-)
12:56 JustinClift In theory, the mount should pull data only from available nodes
12:57 JustinClift So, if a node goes down, the others deal with the requests
12:57 JustinClift However... it's been ages since I've been going GlusterFS stuff hands on, so there's going to be real-world side effects I'm not thinking of
12:57 JustinClift enseven: eg test it in a VM first or something :D
12:58 anoopcs joined #gluster
12:59 enseven Ok, that sounds a bit risky to me.
13:00 enseven Isn't ther a kind of gluster brick offline command?
13:02 Slashman_ joined #gluster
13:04 tigert joined #gluster
13:05 JustinClift enseven: Nope.  It's not a bad idea tho ;)
13:05 edwardm61 joined #gluster
13:06 o5k joined #gluster
13:08 enseven JustinClift: Where can I request this for a feature for future versions?
13:09 JustinClift enseven: https://bugzilla.redhat.com/en​ter_bug.cgi?product=GlusterFS
13:09 JustinClift For "Component", choose "cli" (since it's a command line enhancement)
13:10 JustinClift For "Version", choose "mainline" (eg the stuff being developed)
13:10 JustinClift For Summary and Description, that's all you. :)
13:11 enseven JustinClift: Thanks a lot. :)
13:13 rjoseph joined #gluster
13:14 JustinClift :)
13:18 hagarth joined #gluster
13:26 dooder123 joined #gluster
13:27 suliba joined #gluster
13:34 bennyturns joined #gluster
13:37 o5k hello, i installed gluster on two nodes, by following this guide http://www.gluster.org/documentation/quickstart/ , when i run the command to create the volume : " sudo gluster volume create testvol rep 2 transport tcp node01:/export/sdb1/brick node02:/export/sdb1/brick " it gives me this : "Failed to perform brick order check. Do you want to continue creating the volume?  (y/n)"
13:37 o5k what am I missing ?
14:00 crashmag joined #gluster
14:04 aravindavk joined #gluster
14:04 bene2 joined #gluster
14:08 nbalacha joined #gluster
14:09 toti joined #gluster
14:12 wkf joined #gluster
14:17 meghanam joined #gluster
14:18 o5k There is a Note: From node01 to the other nodes (do not peer probe the first node) , but here http://irclog.perlgeek.de/gluster/2012-10-22 they mentioned to peer probe in both directions
14:19 hagarth joined #gluster
14:19 vipulnayyar joined #gluster
14:19 dgandhi joined #gluster
14:19 nishanth joined #gluster
14:20 hybrid512 joined #gluster
14:23 ndevos ~hostnames | o5k
14:23 glusterbot o5k: Hostnames can be used instead of IPs for server (peer) addresses. To update an existing peer's address from IP to hostname, just probe it by name from any other peer. When creating a new pool, probe all other servers by name from the first, then probe the first by name from just one of the others.
14:24 ndevos o5k: the command you pasted from the quickstart looks good to me, what version are you using?
14:27 o5k ndevos, i'm using the last version 3.6.2
14:28 glusterbot News from newglusterbugs: [Bug 1075417] Spelling mistakes and typos in the glusterfs source <https://bugzilla.redhat.co​m/show_bug.cgi?id=1075417>
14:28 glusterbot News from newglusterbugs: [Bug 1196223] gluster brick offline/online (disable/enable) command that does not affect glusterfs client on the same node??? <https://bugzilla.redhat.co​m/show_bug.cgi?id=1196223>
14:28 glusterbot News from newglusterbugs: [Bug 1191006] Building argp-standalone breaks nightly builds on Fedora Rawhide <https://bugzilla.redhat.co​m/show_bug.cgi?id=1191006>
14:30 ndevos o5k: not sure what could be the issue then, have you checked if the hostnames resolve to the correct IP-address?
14:31 ndevos o5k: 'gluster pool list' should show your two systems
14:32 georgeh-LT2 joined #gluster
14:41 jmarley joined #gluster
14:45 magamo joined #gluster
14:45 magamo Hello everyone.
14:46 theron joined #gluster
14:46 magamo Quick question: Is there a way in which you can query to see what fuse clients are mounting a particular volume remotely?  (Similar to showmounts -a for NFS?)
14:47 Folken_ is there a way you can confirm your disperse volume is healthy? if I shutdown one host the volume fails to respond via it's mount point
14:48 T3 joined #gluster
14:49 awerner joined #gluster
14:50 o5k ndevos, gluster pool list shows different hostnames: node02 and localhost in the node01 and 192.168.50.51 and localhost in the node02
14:50 o5k ndevos, that doesn't cause a problem ?
14:51 o5k although i configured the /etc/hosts and set in it the right ip addresses for each node
14:52 o5k for the UUID provided after launching gluster pool list they are coherent
14:55 B21956 joined #gluster
14:56 ndevos o5k: how do you mean, both systems have the same UUID?
14:57 rafi joined #gluster
14:57 ndevos magamo: I guess you're looking for: gluster volume status $VOLUME clients
14:59 deepakcs joined #gluster
15:00 o5k ndevos, no, but I thought it should print out node01 and localhost, as the two hostnames, because when i peer probed them i used hostnames node01 and node02 and haven't mentioned any IPs
15:00 dgandhi1 joined #gluster
15:00 squizzi joined #gluster
15:01 mikemol joined #gluster
15:01 ndevos ~hostnames | o5k
15:01 glusterbot o5k: Hostnames can be used instead of IPs for server (peer) addresses. To update an existing peer's address from IP to hostname, just probe it by name from any other peer. When creating a new pool, probe all other servers by name from the first, then probe the first by name from just one of the others.
15:01 ndevos but, I do not think that should be an issue...
15:02 ricky-ticky joined #gluster
15:05 o5k ndevos, peer probe: success. Host node01 port 24007 already in peer list
15:05 ndevos o5k: but the hostnames should look nicer now?
15:06 o5k the same : 4a2ab1f7-9d1a-462b-a6f0-39db2​7d940e0192.168.50.51Connected
15:07 ndevos hmm, not sure what the problem could be
15:09 o5k it doesn't cause a problem, i succeded to create the volume using just hostnames :)
15:09 o5k volume create: gv0: success: please start the volume to access data
15:10 ndevos uh, okay, but I thought that you did that before and it failed?
15:11 o5k yes it failed, i deleted /var/lib/glusterd/* and restarted, and this time I used only hostnames in my commands after having configured them in the hosts file
15:14 soumya|afk joined #gluster
15:15 ckotil while doing performance testing on my 2 brick replicated mounts on 2 hosts, I notice that whenever I access a file for the first time I see an error 'remote operation failed: No data available.' is that normal?
15:15 ckotil even a find on a path will cause it. run the same find again, and that doesnt happen. i was thinking maybe gluster is setting some xattr or gfid for the first time. im on 3.6
15:18 JustinClift ckotil: That doesn't sound good.  Which OS and which version of GlusterFS?
15:20 ckotil centos6 and 3.6 from the gluster.org repo
15:20 ckotil if i attempt to access the file a second time or run find I dont see that error
15:22 ildefonso joined #gluster
15:28 glusterbot News from newglusterbugs: [Bug 1188196] Change order of translators in brick <https://bugzilla.redhat.co​m/show_bug.cgi?id=1188196>
15:29 georgeh-LT2 joined #gluster
15:33 ckotil JustinClift: think it's worth downgrading and see if my results vary ?
15:34 JustinClift ckotil: I'm not actually sure.  It's been ages since I've done anything in depth with Gluster strangely enough :/
15:35 * JustinClift is more fighting with keeping our Jenkins infrastructure running recently :/
15:35 ckotil even though the rpm says 3.6 and glusterd --version reports 3.6, i see this in the logs [client-handshake.c:1413:sele​ct_server_supported_programs] 0-domains-client-1: Using Program GlusterFS 3.3, Num (1298437), Version (330)
15:35 hagarth ckotil: are you creating any data on the bricks directly?
15:35 ckotil yea there are writes. seems to work fine
15:36 hagarth ckotil: all writes for gluster need to happen from a mount
15:36 hagarth ckotil: ignore the 3.3 bit, it is a rpc program version which is different from the glusterfs version.
15:36 ckotil all my writes happen in a mount
15:36 ckotil and writes produce zero log messages, so that all seems OK
15:37 ckotil thanks for the clarification on the 3.3 bit
15:37 ckotil yeah writes are solid.
15:37 hagarth ckotil: do you see 'No data available' in the logs or on the mount?
15:38 ckotil it's in data-<volume name>.log
15:38 victori joined #gluster
15:38 ckotil actually that's pry just <volume name>.log
15:38 ckotil in the gluster log dir
15:39 hagarth ckotil: looks like a benign error that can be ignored as long as no problems are observed on mounts
15:39 dbruhn joined #gluster
15:40 ckotil seems benign enough. i just wonder if that's not soemthing i can bootstrap. like run a find on the entire branch. im pretty sure gluster is doing something. the first find I run is slow, and produces the log events. second find is much quicker and no log events
15:40 ckotil i went looking for a command to 'sync' the bricks, thinking that would resolve this, but it doesnt do what i thought it would do.
15:44 JustinClift Ahhh, yeah, I think you're noticing the behaviour of gluster when it does a stat of files
15:44 ckotil ah that could be it!
15:45 JustinClift Where it then looks across every node to get the most up-to-date info on the file being stat'd
15:45 ckotil nice. so if I stat everything before I do my perf testing, I should get better results
15:46 JustinClift Depends on your definition of "better" ;)
15:46 JustinClift But yeah, as in quicker latency for some stuff I guess
15:46 JustinClift bennyturns: ^ Thoughts?
15:46 * bennyturns catches up
15:46 ckotil :) ive seen the performance hit's already with gluster. but ive got some other things working too. proxying between multiple nodes, which is great
15:47 kshlm joined #gluster
15:48 harish_ joined #gluster
15:48 bennyturns ckotil, there is a xarrt cache that makes find fast the second time
15:48 ckotil excellent. thanks for working through that with me
15:48 bennyturns ckotil, what kind of perf tests you running?
15:49 ckotil well im using gluster to replicate files for a CMS. so im just using ab right now to benchmark apache
15:49 victori joined #gluster
15:49 bennyturns ahh cool
15:49 bennyturns ckotil, usually before I do systems test I benchmark each layer
15:49 magamo Thanks guys.
15:50 magamo left #gluster
15:50 ckotil comparing with native ext4 and one apache instance. gluster on ext4 with one instance. and then 2 gluster nodes and apache with a proxy in front
15:50 bennyturns awesome sounds like you have your bases covered :)
15:50 ckotil and with the proxy, and 2 gluster nodes, the performance is comparable to the native ext4 w/ one apache :)
15:50 bennyturns nice!
15:50 ckotil and huge wins, w.r.t availability
15:50 ckotil oh all this in a VPLS subnet separated by about 50 miles
15:51 ckotil sharing an IP w/ carp
15:52 JustinClift Ahhh cool.  Do I hear you volunteering to be a GlusterFS Case Study in a few months when it's all in production and stuff?
15:52 bennyturns sounds like an interesting setup, lmk if you run into anything, HTH!
15:53 ckotil I probably could JustinClift
15:54 ckotil yah, thanks everyone
15:54 corretico joined #gluster
15:55 plarsen joined #gluster
15:56 deniszh joined #gluster
16:00 JustinClift ckotil: Cool. :)
16:06 cmorandin joined #gluster
16:07 JustinClift ckotil: Email me, so I can ping youu in a few months?
16:07 JustinClift ckotil: justin@gluster.org, if that helps :)
16:08 [Enrico] joined #gluster
16:12 Gill joined #gluster
16:13 ckotil done
16:16 JustinClift ckotil: Got it.  Will do. :)
16:20 o5k ndevos, after having a volume gv0 created by this command : sudo gluster volume create gv0 rep 2 transport tcp node01:/export/sdb1/brick node02:/export/sdb1/brick and started it via sudo gluster volume start gv0
16:20 o5k i tried to make the last step in the guide which is testing the created volume, but the command sudo mkdir /mnt/gluster; sudo mount -t glusterfs node01:/gv0; sudo cp -r /var/log /mnt/gluster  gives me " mount: can't find node01:/gv0 in /etc/fstab "
16:22 kkeithley1 joined #gluster
16:23 bernux sudo mount -t glusterfs node01:/gv0 should be sudo mount -t glusterfs node01:/gv0 /mnt/gluster
16:26 vipulnayyar joined #gluster
16:31 victori joined #gluster
16:33 o5k bernux, thanks, it's ok now :)
16:36 bennyturns joined #gluster
16:44 Scotch joined #gluster
16:45 jobewan joined #gluster
16:46 kumar joined #gluster
16:49 gem joined #gluster
16:52 corretico joined #gluster
16:59 ildefonso joined #gluster
17:03 jackdpeterson @purpleidea -- I'm running into an issue where the rror I'm getting is, "initialization of volume 'management' failed, review your volfile again"
17:05 semiosis jackdpeterson: does /etc/glusterfs/glusterd.vol exist?  is it valid?  you could pastie.org it
17:06 jackdpeterson http://pastie.org/9981709
17:07 semiosis weird
17:07 telmich joined #gluster
17:09 JoeJulian kminooie: I installed 3.6.2 on a machine at home and tested splitmount against it. It worked perfectly and used the api. Can you uncomment api.glfs_set_logging in rpc.py, try again and paste the results?
17:13 jackdpeterson @purpleidea -- rm -rf /var/lib/glusterfs/* and re-executing puppet brings gluster back up .. but that's concerning to me
17:18 T3 joined #gluster
17:24 jbrooks joined #gluster
17:31 victori joined #gluster
17:35 valopee joined #gluster
17:37 sputnik13 joined #gluster
17:37 valopee left #gluster
17:37 jackdpeterson hmm, it looks like it keeps removing my peer definitions -- I don't have UUID defined for each peer but I do have the IP set
17:37 jackdpeterson going to test to see if manually assigning the UUID is the fix
17:38 Scotch JoeJulian: I figured out my 3.3.2 "weirdness" yesterday but now have other questions (and thanks again for the help)
17:39 Scotch issue was pretty straight forward...huge log files filled "/", bricks were "half" mounted meaning system apparently mounted volumes BUT you couldn't see via 'df' or other commands...however, remove-brick (which is an async process, no?) completed, tried to "stop" which generated failure status visible under "remove-brick" and "rebalance" status checks
17:40 lalatenduM joined #gluster
17:41 Scotch do I need to commit the remove-brick even though it only moved 3.4 of 19TB? to get past failure status'?
17:50 sputnik13 joined #gluster
17:53 Scotch anyone know if I can commit a remove-brick whose completion is now suspect due to an attempted "stop" after it reported a successful complete status (but only moved 3.4 of 19TB)?
17:54 Scotch v3.3.2
17:55 Scotch thinking it'll work and still preserve data on brick but had odd issues yesterday that have been resolved
17:56 vipulnayyar joined #gluster
17:58 JoeJulian It would preserve the data on the brick, yes.
17:59 JoeJulian That data wont, of course, be part of the volume any longer so you would need to copy it.
18:00 Scotch of course...thought I may be able to whack xattr and reset UUID for server (it's the last brick) to readd/try again...or is that asking for trouble?
18:01 JoeJulian I wouldn't.
18:01 JoeJulian Just mount the client and copy it back on.
18:01 Scotch ok...thx again.  Will do.  You rock :)
18:02 JoeJulian Hehe, thanks. :D
18:05 _Bryan_ joined #gluster
18:08 jmarley joined #gluster
18:12 Rapture joined #gluster
18:14 deepakcs joined #gluster
18:15 purpleidea jackdpeterson: i'll need more info before i can comment... what's concerning to you? (i think you mean disconcerning?)
18:16 victori joined #gluster
18:20 valopee joined #gluster
18:24 valopee Hi
18:24 glusterbot valopee: Despite the fact that friendly greetings are nice, please ask your question. Carefully identify your problem in such a way that when a volunteer has a few minutes, they can offer you a potential solution. These are volunteers, so be patient. Answers may come in a few minutes, or may take hours. If you're still in the channel, someone will eventually offer an answer.
18:25 JustinClift Hi :)
18:25 valopee Can someone tell me if it is save to delete all files in the .glusterfs/indices/xattrop directory on every brick
18:26 valopee I have this duplicated files issue on a production system
18:26 valopee now a heal is hanging on a certain file
18:27 hagarth joined #gluster
18:27 valopee It seems to me that the heal is hanging because there is a reference in .glusterfs/indices/xattrop to a file which does indeed not exist
18:28 valopee is the function of all files in .glusterfs/indices/xattrop only to speed up heal?
18:28 valopee if yes, than it should be safe to remove them, no?
18:30 JustinClift JoeJulian: ^ ?
18:30 valopee in /var/log/glusterfs/glustershd.log I see:
18:30 valopee [2015-02-25 18:30:02.077300] W [client-rpc-fops.c:2766:client3_3_lookup_cbk] 0-sr_vol01-client-30: remote operation failed: No such file or directory. Path: <gfid:625b6bc8-8fcd-4580-a659-0c6ebffb8d6e> (625b6bc8-8fcd-4580-a659-0c6ebffb8d6e)
18:30 valopee already more than 24 hours..
18:31 valopee every second the same message
18:32 valopee I did a find /export/brick*/.glusterfs/ -name 625b6bc8
18:32 valopee And found this in .glusterfs/indices/xattrop
18:33 valopee the logical replica pair of brick 31
18:33 valopee .. on brick 32
18:36 valopee OK, thanks Justin, I’ll try to get hold of Joe via the mailing list
18:36 valopee I just though I’d try here
18:37 wushudoin| joined #gluster
18:48 PeterA joined #gluster
18:50 sputnik13 joined #gluster
18:50 JoeJulian valopee: Yes, but I would start a full heal crawl if you do, ie. "gluster volume heal $vol full"
18:51 JoeJulian ... and here is a much more reliable way to reach me than the ML.
18:52 valopee Hi Joe. I was just buzzy writing a mail to the forum :-)
18:52 valopee I find this file here:
18:52 valopee [root@gluster02 xattrop]# find /export/brick*/.glusterfs/ -name 625b6bc8* -exec ls -al {} \;
18:52 valopee ----------. 2 root root 0 Feb 18 00:58 /export/brick12gfs02/.glusterfs/indices/xa​ttrop/625b6bc8-8fcd-4580-a659-0c6ebffb8d6e
18:52 valopee ---------T. 2 502 users 20725 Feb 20 23:13 /export/brick12gfs02/.glusterfs/62/5b​/625b6bc8-8fcd-4580-a659-0c6ebffb8d6e
18:52 glusterbot valopee: --------'s karma is now -2
18:52 glusterbot valopee: -------'s karma is now -3
18:53 valopee oeps glusterbot does not like me anymore
18:53 valopee I’ll send a mail
18:53 valopee Thank!
18:55 Rapture joined #gluster
19:08 T0aD joined #gluster
19:18 sputnik13 joined #gluster
19:22 JoeJulian valopee: 0 length mode 1000 files are DHT link files. They have ,,(extended attributes) that point to which brick the file is actually on.
19:22 glusterbot valopee: (#1) To read the extended attributes on the server: getfattr -m .  -d -e hex {filename}, or (#2) For more information on how GlusterFS uses extended attributes, see this article: http://pl.atyp.us/hekafs.org/index.php/​2011/04/glusterfs-extended-attributes/
19:25 valopee Thank you! I deleted the first of the 2 files, not the sticky bit file. Now the heal finally seems to be able to complete
19:29 sputnik13 joined #gluster
19:30 deniszh joined #gluster
19:30 JoeJulian excellent
19:33 bene3 joined #gluster
19:33 jackdpeterson @purpleidea -- the concerning part was that gluster kept auto-removing peers and requiring restart of the service. AFter manually specifying the UUID things were resolved. so something about the dynamic UUID generation (at least in my environment) wasn't working as I was expecting it to.
19:38 sputnik13 joined #gluster
19:39 coredump joined #gluster
19:45 bene2 joined #gluster
19:47 T3 joined #gluster
19:54 Philambdo joined #gluster
20:09 jackdpeterson I'm in the process of transitioning data from my previous gluster cluster (failed out to a single node due to epic split brain issues). Currently migrating data to a new gluste cluster with replica 3 and client-side quorum configured. I'm getting client-logs spammed with  W [client-rpc-fops.c:1210:client3_3_removexattr_cbk] 0-pod1-client-1: remote operation failed: No data available.
20:09 jackdpeterson What's the cause of this?
20:09 Pupeno joined #gluster
20:12 jackdpeterson method of copy is an rsync from local disk (a backup of the current-state cluster) to the new cluster w/ fuse mount -- mount -t glusterfs new_server:/pod1 /mnt/gluster_new
20:13 jackdpeterson (within --inplace to prevent the doubling of operations
20:24 chirino joined #gluster
20:25 rotbeard joined #gluster
20:28 SOLDIERz_ joined #gluster
20:29 DV joined #gluster
20:36 jbrooks joined #gluster
20:42 ndevos jackdpeterson: that would be fixed in the next 3.6 and 3.5 update: http://review.gluster.org/9524
20:43 jackdpeterson ah okay, cheers
20:47 PaulCuzner joined #gluster
20:47 valopee left #gluster
20:50 kripper joined #gluster
20:58 MugginsM joined #gluster
20:59 lalatenduM joined #gluster
21:02 Pupeno_ joined #gluster
21:07 badone_ joined #gluster
21:08 sputnik13 joined #gluster
21:19 deniszh joined #gluster
21:23 sputnik13 joined #gluster
21:32 sputnik13 joined #gluster
21:34 sputnik13 joined #gluster
21:45 klaas joined #gluster
21:48 zwevans joined #gluster
21:48 zwevans left #gluster
22:07 nmbr_ joined #gluster
22:18 social joined #gluster
22:21 gildub joined #gluster
22:30 purpleidea jackdpeterson: sounds like something fishy with your setup... can you fpaste your configs?
22:30 purpleidea jackdpeterson: and gluster/os/puppet versions, etc...
22:38 wkf joined #gluster
22:40 Slashman joined #gluster
22:43 DV joined #gluster
22:55 bennyturns joined #gluster
23:06 diegows joined #gluster
23:08 kripper Hi, I have been talking with people from libivirt and ovirt about an issue probably related with QEMU's file descriptors being invalidated when the VM image is running on a gluster-volume
23:09 kripper I was able to reproduce it by restarting the volume
23:10 kripper (which could be avoided in a production environment)
23:10 kripper but, according to some reports, I'm guessing that file descriptors could also be invalidated by other issues
23:11 kripper (not sure about network problems, though it was reported by some user)
23:12 kripper is there a chance that gluster's file descriptors could get into a bad state?
23:12 al joined #gluster
23:12 kripper BZ: https://bugzilla.redhat.co​m/show_bug.cgi?id=1058300
23:12 glusterbot Bug 1058300: unspecified, unspecified, ---, bugs, NEW , VMs do not resume after paused state and storage connection to a gluster domain (they will also fail to be manually resumed)
23:14 kripper One of the symptoms is the error "Transport endpoint is not connected (107)"
23:14 kripper Transport endpoint is not connected"Transport endpoint is not connected" is an error from FUSE that indicates that glusterfs was unable to serve a request for a file. Glusterfs provides no mechanism for persistent retries of failed operations (as at version 2.0). A common cause of this error is a network interruption.http://www.gluster.com/community/d​ocumentation/index.php/User_Guide indicates to fix this, umount then mount the failed files
23:14 kripper (Ref: http://www.gluster.org/community/documentation/​index.php/Transport_endpoint_is_not_connected)
23:18 kripper Some one from redhat told me that you can't just stop a filesystem while some process has an open file handle (in this case, the process is qemu, not libvirt), and still expect the handle to come back to life when the filesystem is present again, unless the filesystem is something like NFS that has super-long timeouts built into it to survive temporary outages while still keeping the same handle alive.
23:18 kripper And that gluster apparently doesn't have the ability to revive the handle when the connection comes back online.
23:18 kripper Should gluster do it? is it feasible?
23:21 JustinClift kripper: Timing wise, there's probably not many people on IRC at the moment
23:21 JustinClift They'll start to arrive in um... about 6 hours I think. :/
23:22 JustinClift Maybe ask on the gluster-users mailing list?  You won't get an answer any faster (or generally wont), but at least you don't have to hang around waiting and wondering. :)
23:35 kripper ok
23:35 kripper JustinClift: thanks
23:42 JoeJulian kripper: How long are these outages? More than 42 seconds?
23:51 deniszh1 joined #gluster
23:53 kripper JoeJulian: I'm not sure, I just saw the VM puased on my installation sometimes and only reproduced it by restarting the volume
23:53 kripper JoeJulian: can I simulate an outage by shutting down a node?
23:54 al joined #gluster
23:54 kripper JoeJulian: should file descriptors be kept alive?
23:58 al joined #gluster
23:59 JoeJulian kripper: doubtful, especially if you exceed ping-timeout.
23:59 JoeJulian Oh, which you will because ovirt suspends the vm.

| Channels | #gluster index | Today | | Search | Google Search | Plain-Text | summary