Perl 6 - the future is here, just unevenly distributed

IRC log for #gluster-dev, 2016-08-11

| Channels | #gluster-dev index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:04 decay joined #gluster-dev
00:05 shyam joined #gluster-dev
01:04 gem_ joined #gluster-dev
01:40 penguinRaider joined #gluster-dev
01:47 ilbot3 joined #gluster-dev
01:47 Topic for #gluster-dev is now Gluster Development Channel - http://gluster.org | For general chat go to #gluster | Patches - http://review.gluster.org/ | Channel Logs - https://botbot.me/freenode/gluster-dev/ & http://irclog.perlgeek.de/gluster-dev/
01:54 nishanth joined #gluster-dev
03:26 julim joined #gluster-dev
03:34 magrawal joined #gluster-dev
03:35 shubhendu_ joined #gluster-dev
03:46 itisravi joined #gluster-dev
03:48 nishanth joined #gluster-dev
04:04 spalai joined #gluster-dev
04:06 atinm joined #gluster-dev
04:21 sanoj joined #gluster-dev
04:31 nbalacha joined #gluster-dev
04:32 poornimag joined #gluster-dev
04:33 itisravi joined #gluster-dev
04:34 jiffin joined #gluster-dev
04:38 aravindavk joined #gluster-dev
04:52 spalai left #gluster-dev
04:56 aravindavk joined #gluster-dev
05:00 atalur joined #gluster-dev
05:15 ppai joined #gluster-dev
05:15 skoduri joined #gluster-dev
05:18 karthik_ joined #gluster-dev
05:19 penguinRaider joined #gluster-dev
05:22 atinm nigelb++
05:22 glusterbot atinm: nigelb's karma is now 23
05:23 nigelb :D
05:24 hgowtham joined #gluster-dev
05:28 ndarshan joined #gluster-dev
05:29 Manikandan joined #gluster-dev
05:34 rafi joined #gluster-dev
05:38 Bhaskarakiran joined #gluster-dev
05:39 Bhaskarakiran joined #gluster-dev
05:39 aravindavk joined #gluster-dev
05:41 kdhananjay joined #gluster-dev
05:44 ramky joined #gluster-dev
05:47 ankitraj joined #gluster-dev
05:47 devyani7_ joined #gluster-dev
05:48 ueberall joined #gluster-dev
05:48 ueberall joined #gluster-dev
05:49 ashiq joined #gluster-dev
05:49 atinm joined #gluster-dev
05:58 mchangir joined #gluster-dev
05:59 ramky joined #gluster-dev
06:03 aravindavk joined #gluster-dev
06:07 itisravi_ joined #gluster-dev
06:08 pur joined #gluster-dev
06:12 Muthu_ joined #gluster-dev
06:14 asengupt joined #gluster-dev
06:17 kshlm joined #gluster-dev
06:19 msvbhat joined #gluster-dev
06:23 aspandey joined #gluster-dev
06:25 kshlm aravindavk, pranithk, You guys have an AI from yesterday's meeting.
06:26 kshlm "pranithk/aravindavk/dblack to send out a reminder about the feature  deadline for 3.9"
06:27 prasanth joined #gluster-dev
06:28 nigelb kshlm: What was our logic behind making regressions run after Verified +1?
06:28 aravindavk kshlm: sure. Thanks
06:29 nigelb So that reviews can be in draft state before that?
06:29 nigelb We're running into a small edge case now - (bug 1366106)
06:29 glusterbot Bug https://bugzilla.redhat.com:443/show_bug.cgi?id=1366106 unspecified, unspecified, ---, nigelb, ASSIGNED , Regression jobs vote on the exact patchset
06:29 kshlm To avoid launching unnecessary jobs that took up resources.
06:30 atalur joined #gluster-dev
06:30 nigelb I'm wondering if we can use the "draft" state on Gerrit instead.
06:30 kshlm Ideally (IMO atleast) regressions should be the final step before a patch should get merged.
06:31 kshlm We could, but that would require the ./rfc.sh script to be modified.
06:31 nigelb ah, so ideally we'd want something like zuul doing the merging
06:31 kshlm Ideally. Yes.
06:32 nigelb I'm happy to modify the rfc.sh script.
06:32 nigelb But then, again, that doesn't solve the problem in an elegant way.
06:33 kshlm I'd love if we are able to use git-review instead of the script. That has support for draft already.
06:33 nigelb why don't we?
06:33 kshlm I think we could wrap git-review with the rfc.sh script.
06:34 kshlm Because git-review doesn't have support for doing out 'BUG'id check before push.
06:34 nigelb If it's a matter of documenting, I'm happy to do it.
06:34 kshlm s/out/our/
06:34 nigelb that can be a hook I suspect.
06:34 kshlm git-review has hooks?
06:34 nigelb git has hooks
06:34 kshlm It didn't the last time I saw it.
06:34 nigelb rfc.sh already uses a hook
06:35 kotreshhr joined #gluster-dev
06:35 nigelb (to add change-id)
06:36 kshlm That's automatic. But forcing devs to add a bug-id each time they commit would be a bit too much.
06:36 kshlm At least for me.
06:36 nigelb can be a pre-push
06:37 nigelb Anyway, let me try a few things and see if I can make life better.
06:37 kshlm If it works, I'm all for it.
06:37 kshlm Cool.
06:38 itisravi joined #gluster-dev
06:38 Muthu_ joined #gluster-dev
06:40 rafi joined #gluster-dev
06:44 Saravanakmr joined #gluster-dev
07:02 spalai joined #gluster-dev
07:02 prasanth joined #gluster-dev
07:04 devyani7 joined #gluster-dev
07:04 devyani7 joined #gluster-dev
07:13 skoduri joined #gluster-dev
07:22 rastar joined #gluster-dev
07:27 ndevos kshlm++ thanks for the CI <-> GitHub reporting fix
07:27 glusterbot ndevos: kshlm's karma is now 101
07:28 ppai joined #gluster-dev
07:29 karthik_ joined #gluster-dev
07:32 sanoj_ joined #gluster-dev
07:51 ppai joined #gluster-dev
07:57 prasanth joined #gluster-dev
08:13 ramky joined #gluster-dev
08:21 ramky joined #gluster-dev
08:27 post-factum nigelb: could you please take a look at https://build.gluster.org/job/rackspace-netbsd7-regression-triggered/18638/console ?
08:27 post-factum nigelb: why it stalled?
08:28 * misc think nigelb prefer a ticket
08:29 post-factum misc: should i do that via bz?
08:29 nigelb yes, please.
08:29 nigelb I bet that machine had a disconnect.
08:30 post-factum nigelb: under "tests" component?
08:30 nigelb project-infrastructure
08:31 nigelb yep. The machine needs a reboot.
08:31 nigelb post-factum: Can I cancel and retrigger?
08:31 post-factum nigelb: feel free to do that
08:31 nigelb I'll use the bug to add notes about what went wrong and how to avoid it.
08:32 post-factum nigelb: i'm filling bz now
08:32 nigelb cheers
08:32 nigelb Retriggered from Jenkins
08:34 Muthu_ joined #gluster-dev
08:36 poornimag rjoseph++
08:36 glusterbot poornimag: rjoseph's karma is now 4
08:37 post-factum nigelb: https://bugzilla.redhat.com/show_bug.cgi?id=1366168
08:37 glusterbot Bug 1366168: high, unspecified, ---, bugs, NEW , NetBSD regression test #18638 times out for review #14966
08:40 shaunm joined #gluster-dev
08:47 lpabon joined #gluster-dev
09:05 ndevos poornimag, rjoseph: I asked a question in http://review.gluster.org/14722/ , could you please leave a comment in the review?
09:07 ndevos rjoseph: also, there is no announcement for 3.8.2 yet, please refrain from merging changes until there is a confirmation the version is ok
09:19 ppai joined #gluster-dev
09:24 shubhendu__ joined #gluster-dev
09:44 rjoseph ndevos: Ah! my bad. I saw "glusterfs-3.8.2 released" mail so thought the branch is open for merge.
09:44 prasanth joined #gluster-dev
09:45 rjoseph ndevos: poornimag will add the comment on http://review.gluster.org/14722/ regarding the new bug detail
09:45 atalur joined #gluster-dev
09:51 karthik_ joined #gluster-dev
09:53 nigelb Oh fun.
09:53 nigelb this machine just refuses to shutdown cleanly.
09:57 devyani7 joined #gluster-dev
09:57 devyani7 joined #gluster-dev
10:00 post-factum nigelb: netbsd one?
10:02 nigelb post-factum: yeah.
10:03 nigelb we're either doing something really bad in our tests
10:03 nigelb or something has gone really bad with this machine.
10:03 nigelb post-factum: and yeah, the quota tests have timing issues on netbsd
10:04 post-factum nigelb: hope something could be done about that
10:05 nigelb jdarcy had some ideas. I don't know how to implement them. I'll try to catch him when he's back from vacation next week.
10:07 atalur joined #gluster-dev
10:08 nigelb misc: If something is blocking a shutdown, where do I look to find out what's blocking it?
10:09 nigelb I've looked at /var/messages
10:09 nigelb for extra fun, it's a bsd machine.
10:09 nigelb I know I can do a hard reboot
10:09 nigelb I'm trying to figure out what's blocking the soft reboot
10:12 msvbhat joined #gluster-dev
10:14 penguinRaider joined #gluster-dev
10:15 misc nigelb: what server ?
10:15 misc I would use some fancy strace command, or local equivalent and check process
10:15 nigelb nbslave7h
10:17 misc mhhh
10:17 misc so reboot is D, so that's a issue in the kernel I think
10:17 prasanth joined #gluster-dev
10:17 misc since 'mount' block too
10:18 misc so IMHO, bug in netbsd :)
10:29 nigelb Yeah, I got a trace and a dump of it, just in case Emmanuel wants to debug further.
10:37 noobs first we need to do peer probing or we need to create volume in gluster
10:39 jiffin noobs: after starting glusterd on all required node, then you need to create storage pool using peer probe
10:39 jiffin after that create required volume
10:40 noobs Thanks
10:43 nigelb misc: meh, gave up and did a hard restart eventually.
10:44 nigelb there's only so much digging around I can do :)
10:44 noobs jiffin: m getting error peer probe: failed: Probe returned with Transport endpoint is not connected
10:46 jiffin noobs:may be related to iptables, firewall
10:46 jiffin just flush iptables or stop firewall service
10:47 noobs jiffin: After stopping firewall services, it worked. Thanks
10:58 post-factum nigelb: should i re-trigger the test?
11:00 ppai joined #gluster-dev
11:10 nigelb post-factum: I already did. Let me reconfirm.
11:11 nigelb post-factum: oh, you had a failure.
11:11 nigelb yeah, retrigger.
11:11 post-factum okay
11:20 spalai left #gluster-dev
11:28 nigelb post-factum: sorry it's such a mess. I want to figure out how to make netbsd fail less.
11:28 post-factum nigelb: i'd wonder, how netbsd became a platform for regression tests, and why not freebsd or openbsd
11:29 misc mostly because netbsd people were interested into helping us
11:31 kshlm And also because netbsd was the first one to have fuse in its kernel.
11:31 ndevos nigelb: shutting down systems when an nfs-mount is active can cause interesting problems, it is either re-try until the nfs-server comes back, or loose data
11:31 kshlm IIRC emmanuel who helped get glusterfs on netbsd, aslo worked on getting that done.
11:32 ndevos nigelb: for our tests, if there is a nfs-mount, and gluster/nfs gets killed, the nfs-mount may hang indefinitely, in the hope the server comes back
11:33 nigelb ndevos: ah.
11:33 nigelb how does one shut it down in this scenario
11:33 nigelb even `kill -9` wasn't working.
11:34 ndevos nigelb: "reboot -f" or something like that
11:34 ndevos nigelb: dont try to flush the data in the filesystems :)
11:36 ndevos nigelb: processes with outstanding (or doing) I/O are in D-state, and you can not kill them on Linux, not sure if that is different on NetBSD
11:43 mchangir joined #gluster-dev
11:48 post-factum ndevos: i believe this is impossible on non-microkernels
11:51 ndevos post-factum: well, its software, anything is possible, its just an agreement if it is a good solution, and how much time you want to spend on it
11:52 post-factum ndevos: i believe it would imply some form of well-define state-machine, which is really complex approach for blk
11:52 ndevos post-factum: I didnt say it was easy :)
11:53 ndevos post-factum: and, I think most users do want their data to land on the nfs-server at one point...
11:54 post-factum ndevos: regarding nfs, shouldn't "intr" option do the trick for client?
11:54 ndevos post-factum: I guess so, not sure if that exists on NetBSD
11:54 post-factum ndevos: or "soft". whatever
11:56 misc but do we want to test only soft, and not what happen with hard ?
11:58 ppai joined #gluster-dev
11:59 nigelb ndevos: so I'm guessing reboot -n (http://netbsd.gw.com/cgi-bin/man-cgi?reboot+8+NetBSD-6.0+i386)
12:10 kdhananjay joined #gluster-dev
12:14 post-factum nigelb: unfortunately, ./tests/basic/quota.t still fails
12:14 post-factum nigelb: https://build.gluster.org/job/rackspace-netbsd7-regression-triggered/18650/console
12:14 post-factum nigelb: what could be done to overcome that?
12:16 nigelb I wonder if we've actually broken something on release-3.7.
12:22 post-factum nigelb: could coredumps or exact crash reason be extracted from regression tests?
12:29 prasanth joined #gluster-dev
12:40 msvbhat joined #gluster-dev
12:43 ndevos hi tdasilva and ppai, do you have some simple-fix bugs/improvements for swift? I might have someone interested in sending some small patches for openstack
12:45 tdasilva ndevos: hi, sure we have bugs ;)
12:46 tdasilva here's a list of 'low-hanging fruit' bugs: https://bugs.launchpad.net/swift/+bugs?field.tag=low-hanging-fruit
12:47 tdasilva ndevos: is that what you were looking for, or would you like something more specific?
12:47 ndevos tdasilva: that looks good, thanks!
12:48 ndevos tdasilva++ :)
12:48 glusterbot ndevos: tdasilva's karma is now 3
12:48 tdasilva ndevos: let me know if I can help with anything
12:51 ndevos tdasilva: will do, I'm not sure when she has time to write some code, I'll keep encouraging :)
12:55 msvbhat loadtheacc: ping
12:55 glusterbot msvbhat: Please don't naked ping. http://blogs.gnome.org/markmc/2014/02/20/naked-pings/
13:00 ppai joined #gluster-dev
13:04 Manikandan joined #gluster-dev
13:10 dlambrig joined #gluster-dev
13:12 raghug joined #gluster-dev
13:17 decay joined #gluster-dev
13:21 nbalacha joined #gluster-dev
13:21 ramky joined #gluster-dev
13:22 julim joined #gluster-dev
13:27 nigelb post-factum: If there are coredumps, you should see it in the logs.
13:27 nigelb sec, let me check your job's logs
13:35 lpabon joined #gluster-dev
13:36 jiffin1 joined #gluster-dev
13:47 nigelb post-factum: no luck, no cores for those failures.
13:48 nigelb I'm now running a check to see how the failures look across branches.
13:48 post-factum nigelb: and no visible reason of failure?
13:48 Manikandan joined #gluster-dev
13:52 nigelb post-factum: I'm reading the test code to see what's going on.
13:54 lpabon joined #gluster-dev
13:55 nigelb post-factum: you're not going to believe this.
13:55 misc "this weird trick who prevent your build from working"
13:55 nigelb post-factum: we've marked that test as bad on 3.8 and master.
13:56 post-factum but not on 3.7
13:56 post-factum ?
13:56 nigelb so the only people running into it are people merging onto 3.7
13:56 nigelb exactly.
13:56 post-factum saint cookies
13:56 nigelb I think it's time to disable on 3.7 as well.
13:56 nigelb sorry I didn't check this earlier.
14:00 post-factum nigelb: that, nevertheless, does not explain netbsd processes in D state
14:01 nigelb post-factum: that's probably an artifact of the hung tests
14:01 nigelb they're aborted and sometimes we have a new test run started
14:02 nigelb and they might hang too.
14:02 nigelb and so on.
14:02 nigelb until someone notices something has gone wrong.
14:02 nigelb I'm aware this happens, so I'm trying to setup a trigger to restart the machine when there's an aborted job. Or at the very least email me.
14:03 shyam joined #gluster-dev
14:03 post-factum nigelb: so we are not at the point of 1 temporary VM per 1 test?
14:03 nigelb post-factum: not yet.
14:03 nigelb working towards it.
14:03 post-factum good luck, should be very handy thing :)
14:03 nigelb For netbsd, we'll have to wait until the community cage is functional.
14:04 kkeithley wait, the community cage isn't functional yet? I thought it was.
14:05 nigelb Not as functional as I'd like it to be.
14:05 nigelb There's some machines that need to move into the final cage, etc.
14:06 shyam joined #gluster-dev
14:07 shyam left #gluster-dev
14:08 shyam joined #gluster-dev
14:09 kkeithley pj
14:09 kkeithley oh
14:10 hagarth joined #gluster-dev
14:16 post-factum https://build.gluster.org/job/rackspace-regression-2GB-triggered/22847/consoleFull
14:16 post-factum what changed in 3.8 comparing to 3.7 regarding tests?
14:16 post-factum ./tests/basic/gfapi/gfapi-trunc.c:5:32: error: glusterfs/api/glfs.h: No such file or directory
14:16 post-factum (smoke for 3.8 passes ok, 3.7 fails with the error above)
14:17 post-factum s/smoke/centos-regression
14:39 shubhendu_ joined #gluster-dev
14:41 jiffin joined #gluster-dev
14:42 ashiq joined #gluster-dev
14:51 hagarth joined #gluster-dev
14:52 msvbhat joined #gluster-dev
14:56 penguinRaider joined #gluster-dev
15:01 atalur_ joined #gluster-dev
15:04 wushudoin joined #gluster-dev
15:06 nbalacha joined #gluster-dev
15:10 ndevos laptop batteries are a fine thing, working on mine the whole day (forgot charger), and it now says it has only 53 HOURS left
15:12 amye oh there is a fun fedora bug for that
15:13 ndevos ah, I'm running Fedora!
15:13 shyam post-factum: I guess you renamed it from .t to .sh
15:14 shyam For the gfapi based test that is failing, that should do it
15:14 amye ndevos: I watched this happen on rbowen's machine in Brno.
15:16 ndevos amye: you can see it on mine too! http://devos.fedorapeople.org/tmp/lenovo-x240-battery.png
15:16 amye Awesome.
15:16 ndevos amye: happen to have a link to the bug?
15:21 anoopcs obnox++
15:21 glusterbot anoopcs: obnox's karma is now 6
15:22 amye ndevos: not off the top of my search :/
15:23 ndevos amye: hmm, ok, thanks for checking
15:23 obnox 53 hours is actually not too bad.
15:24 obnox ;-)
15:24 ndevos yeah, I'm impressed too!
15:24 obnox imagine how long it can last if the battery is full
15:24 ndevos add another 6 hours or so, I've been using it for that long already :D
15:25 * ndevos might suddenly become unresponsive on irc...
15:25 obnox 42 days if scaling linearly... ;-)
15:25 obnox there's some wonderful ultimate truth to that
15:25 misc ah ah
15:39 rafi1 joined #gluster-dev
15:50 hagarth joined #gluster-dev
15:59 spalai joined #gluster-dev
16:47 skoduri joined #gluster-dev
16:48 shyam joined #gluster-dev
16:49 kkeithley ndevos: Debian pkgs are up on d.g.o.  LATEST symlinks have been updated.  Announce away. (And I'm on PTO tomorrow.)
17:09 shubhendu__ joined #gluster-dev
17:18 post-factum shyam: will try to fix that and bring it to the state of other gfapi tests
17:20 ankitraj joined #gluster-dev
17:40 shyam joined #gluster-dev
17:57 rafi joined #gluster-dev
18:04 julim joined #gluster-dev
18:11 shubhendu__ joined #gluster-dev
18:13 kkeithley @later tell justinclift I have a question about that mac mini we were using here
18:13 glusterbot kkeithley: The operation succeeded.
18:15 lpabon_ joined #gluster-dev
18:20 ndevos ok, thanks kkeithley!
18:30 post-factum nigelb: have you really disabled netbsd quota test for 3.7? i still get it failed
18:31 hagarth I also wonder if we need netbsd tests for backports
18:33 post-factum why not?
18:33 hagarth since the same tests or more would have passed on master for the same patch
18:34 hagarth most backports seem to be blocked by spurious netbsd failures
18:34 post-factum then, we really need only smoke test for backports
18:34 hagarth post-factum: ideally yes
18:35 hagarth post-factum: though I could also argue that more tests should be run for backports ;)
18:36 Manikandan joined #gluster-dev
18:36 post-factum sure, if they do not fail spuriously
18:36 hagarth post-factum: right
18:37 hagarth hmm, we need more post-commit tests for backports
18:37 hagarth pre-commit can happen largely on master
18:44 hagarth worth a discussion on -devel
19:08 gem joined #gluster-dev
19:23 msvbhat joined #gluster-dev
19:27 shaunm joined #gluster-dev
19:51 rafi ndevos: ping
19:51 glusterbot rafi: Please don't naked ping. http://blogs.gnome.org/markmc/2014/02/20/naked-pings/
19:51 ndevos rafi: are you pinging naked again?
19:52 * ndevos notes its 21:52 where he is, and its *much* later for rafi
19:52 rafi ndevos: I'm too lazy, didn't thought about hardworking glusterbot
19:53 rafi ndevos: had an issue with nfs
19:53 ndevos rafi: glusterbot is always up for a good chat
19:53 ndevos rafi: "had"? that means you fixed it?
19:53 jiffin joined #gluster-dev
19:53 ndevos rafi: there is your nfs expert! I wonder why you didnt ask him
19:54 * ndevos points to jiffin
19:54 ndevos (with both hands)
19:54 * jiffin points expert's expert ->
19:54 jiffin ndevos:
19:55 rafi ndevos: We had a chat
19:55 ndevos rafi: I hope jiffin had better answers than glusterbot
19:56 ndevos is "we had a chat" similar to "we have to talk'?
19:57 rafi ndevos: jiffin has much big brain than the glusterbot
19:57 jiffin ndevos has better answer than jiffin
19:57 hagarth joined #gluster-dev
19:58 ndevos rafi: oh, good to know that jiffin is smarter than glusterbot
19:58 ndevos just dont tell the bot, he'll feel sad
19:58 post-factum rafi: do you still hope to get some answers?
19:58 jiffin :)
19:59 jiffin rafi didn't ask the question yet :P?
19:59 rafi post-factum: we need to finish off the discussion about jiffin :p
19:59 post-factum ah oh please proceed
19:59 * ndevos is hungry, so the answer is most likely "YES, IT IS INTENTIONAL"
19:59 * jiffin thinks how can he get answers without a question
19:59 rafi post-factum: jiffin >>>> question
20:00 ndevos wait, jiffin is going to ask rafi's question to me?
20:00 rafi ndevos, post-factum: basically I was testing nfs ganesh and meta
20:01 ndevos then I think post-factum should answer it :D
20:01 rafi ndevos: it works in rhel-6.4 client but not in fedora 22 and greater
20:01 * post-factum slaps his knee
20:01 rafi ndevos: getting ESTALE from nfs-client
20:02 post-factum citing ndevos, YES, IT IS INTENTIONAL
20:02 ndevos lol!
20:02 * jiffin smells good from ndevos kitchen
20:02 ndevos jiffin: I have not even started to cook something yet...
20:02 jiffin lol
20:03 ndevos rafi: thats your question you want an answer to?
20:04 rafi ndevos, basically meta is not giving any ESTALE error
20:04 post-factum rafi: is that somehow related to upcall?
20:04 ndevos rafi: sure, its probably something that causes the filehandle to change
20:04 rafi ndevos: so mostly it could be a problem where meta fakes the dentry iatt
20:04 rafi post-factum: nop
20:05 ndevos rafi: does the gfid from dentries sometimes change?
20:06 rafi ndevos: if there is a linked inode, then meta always gives the inode from its inode ctx
20:06 jiffin post-factum: upcall was not enabled
20:06 ndevos rafi: I *think* rhel-6.4 uses NFSv3 by default, whereas Fedora (and rhel-7) use NFSv4
20:06 post-factum then, rafi, how did you mount it?
20:07 ndevos post-factum: upcall is (atm) only needed/useful for multiple nfs-ganesha servers accessing the same volume
20:07 * rafi is testing with vers 4 in rhel6.4
20:08 rafi ndevos: i tried with nfsv4 and it works in rhel6.4
20:08 ndevos rafi: so only Fedora 22+ fails?
20:09 rafi ndevos: yes
20:09 ndevos rafi: I cant think of anything atm... so, get me the following:
20:10 ndevos 1. unmount & stop nfs-ganesha
20:10 ndevos 2. start a tcpdump on the nfs-ganesha server, capture all tcp traffic (without port 22)
20:10 ndevos 3. start nfs-ganesha
20:10 rafi ndevos: cool
20:11 ndevos 4. do a mount on the client, and whatever ls/cd/cat triggers the ESTALE
20:11 ndevos 5. stop the tcpdump process & gzip the generated file
20:11 ndevos 6. repeat from step 1 in a different setup
20:12 ndevos (where "different setup" probably just is a different client)
20:12 jiffin post-factum: we can manually configure ganesha for gluster volumes without using gluster cli commands
20:12 ndevos and you should capture the tcpdump with something like: tcpdump -i any -s 0 -w /var/tmp/yes-or-no-ESTALE.pcap tcp and not port 22
20:13 ndevos and you execute that tcpdump ON THE NFS-GANESHA SYSTEM
20:13 ndevos the tcpdump will contain the NFS-traffic, and GlusterFS procedures, we can compare that
20:13 post-factum jiffin: i know. i've already tried ganesha for mailboxes, but with no luck. got dovecot in D state for unknown reason (looked like ESTALE as well), and now i follow this discussion carefully
20:14 ndevos rafi: also, please use a gluster volume that consists out of a single brick, otherwise it is much more packets to go through
20:14 rafi ndevos: I'm testing with one brick only :)
20:15 ndevos rafi: good! no 6x2 + 4x2 tiered volumes for things like this :)
20:16 hagarth rafi: please test with a sharded distributed replicated volume ;)
20:16 * rafi people who uses such configurations are sadist :)
20:16 rafi just kidding
20:16 hagarth :D
20:17 ndevos hagarth: btw, you are co-owner of deprecating the stripe xlator, do you feel like sending a patch for the cli?
20:17 hagarth ndevos: just the cli?
20:17 ndevos hagarth: at first, yes, just warn + error-out that stripe is being deprecated in favour of sharding
20:18 ndevos hagarth: there is a bug for it too already, it's blocking the glusterfs-3.9.0 tracker
20:18 hagarth ndevos: maybe we should await sharding to work well with all use cases..
20:19 post-factum tiered sharded distributed replicated volume with georeplication enabled
20:19 ndevos hagarth: I do not think so, stripe also does not work for all use-cases?
20:19 hagarth post-factum: you missed quota and bitrot
20:19 post-factum hagarth: :(
20:19 hagarth post-factum: possibly arbiter too
20:19 post-factum rafi: you've got the point
20:20 hagarth ndevos: will consult kruthika & pranith about their opinion on sharding
20:20 ndevos I think sharded comes before tiered too
20:20 ndevos hagarth: I thought I sent an email to the -devel list about this, you should probably reply to that
20:21 hagarth ndevos: you indeed did
20:21 ndevos hah!
20:21 hagarth ndevos: and yes, will check on that thread
20:21 jiffin hagarth: do u know someone that uses  wide possibility of gluster?
20:21 ndevos wide?
20:21 rafi ndevos, post-factum: yes,
20:21 hagarth jiffin: as in all features supported by gluster?
20:22 jiffin hagarth: Yup
20:22 rafi ndevos, post-factum: may be we can include uss, quota, etc
20:22 * jiffin wonders how the volume create look like
20:22 ndevos rafi: step 7 in the above list is: pass the two tcpdumps to jiffin for analysis
20:23 * ndevos does really not care what type of volume is used for it then
20:23 hagarth jiffin: not quite .. distributed-replicated / distributed-dispersed are the more popular configurations
20:23 jiffin rafi, ndevos: gudnyt
20:23 hagarth jiffin: geo-replication does get used in some of those setups..
20:23 ndevos jiffin: you too, we'll talk tomorrow again
20:23 hagarth jiffin: that's about it. other features are enabled on a need basis
20:23 hagarth jiffin: good night!
20:24 ndevos thats the way ^^ to spell it, jiffin
20:24 post-factum jiffin: bye
20:24 jiffin hagarth:  gud nyt
20:24 jiffin post-factum: gud nyt
20:24 ndevos you're really saving your keyboard :)
20:27 hagarth ndevos: and thereby the world too ;)
20:28 ndevos cya hagarth, I'll be off and finally getting something for dinner now :)
20:29 hagarth ndevos: bon appetit!
20:30 post-factum ndevos: oh, that night eating
20:31 * ndevos decided on making a quick soup - bye!
20:42 shyam joined #gluster-dev
20:51 * rafi doesn't seen any error's in nfs server tcpdump file
20:51 * rafi suspect any change in behavior with nfs-client, particularly based on response of a readdirp
20:51 rafi ndevos: ^
20:54 ndevos rafi: that is possible, we can check the NFS filehandles (in the format of <magic><volume-id><gfid>) and the iatt/gfid - maybe something is wrong there
20:54 ndevos rafi: you can also enable "rpcdebug -... " something, to get more debugging from the nfs-client, including the NFS procedures and filehandles
20:55 rafi ndevos: okey
20:55 ndevos rafi: put the files somewhere so that I can access them, and send me an email (with gluster-devel on cc) and I'll look at it tomorrow
20:56 ndevos rafi: people.redhat.com would do, if you have an account there, or maybe <username>.fedorapeople.org or something like that
20:56 rafi ndevos: great, I will try people.redhat.com ,
20:57 rafi ndevos: and then fedorapeople
20:58 * rafi worries about any other confidential information in packet trace when it publicly share
20:58 ndevos rafi: create a public_html/tmp directory in your home-dir, and place the files there, you should be able to see them when like on http://devos.fedorapeople.org/tmp
20:59 rafi ndevos:  that is nice
20:59 * rafi finds userspace-rcu :)
20:59 ndevos rafi: or, you can do that on Red Hat internal file servers, but in that case you should now pass the link in the email to gluster-devel :)
21:00 rafi ndevos: sure
21:00 ndevos well, tmp is not *that* temporary, its more of a dumping ground :)
21:00 rafi ndevos: I will try to debug it further,
21:00 rafi ndevos: :)
21:01 ndevos rafi: if I can not see anything in the tcpdumps, the rpcdebug command (+ new tcpdump) may help
21:02 rafi ndevos: okey
21:02 ndevos rafi: I dont think you need to fix this by tomorrow, right? THEN GO TO SLEEP!
21:02 rafi ndevos: I'm interested to debug nfs problems
21:02 rafi ndevos: that is right
21:03 ndevos rafi: get those details, and I'll explain what I did to debug in an email
21:03 rafi ndevos: great
21:03 rafi ndevos: I will get those information tomorrow
21:03 * ndevos used to do ascii tshark outputs with arrows and explanations
21:04 ndevos rafi: sure, whenever you have time for it
21:04 rafi ndevos: that would be nicer,
21:05 rafi ndevos: i never used tshark, though not a bug fan of wireshark
21:05 rafi *big
21:05 ndevos rafi: I used tshark/wireshark *a* *lot*, it is one of the few options to debug production environments (there definitely is no gdb)
21:06 rafi ndevos: indeed,
21:06 rafi ndevos: tshark or wireshark, which one do you prefer ?
21:06 ndevos it is also one of the few tools that can capture things from multiple systems at the same time, so it is easier for people to gather stuff
21:07 misc does wireshark support gluster protocol ?
21:07 * rafi is always interested to learn from best
21:07 ndevos rafi: just for browsing and checking I use wireshark, then I explain with tshark outputs what the details are, tshark is then used instead of screenshots
21:07 rafi ndevos: great
21:08 ndevos misc: yes, it does, I needed that in order to support it for Red Hat customers, I have no idea how people did support (or development) without it
21:09 rafi ndevos:
21:09 rafi ndevos++ it was great help
21:09 glusterbot rafi: ndevos's karma is now 301
21:09 ndevos rafi++ no problem, good night!
21:09 glusterbot ndevos: rafi's karma is now 52
21:10 rafi ndevos: gud night
21:10 rafi ndevos: enjoy your dinner
21:10 ndevos heh, yes, thanks :)
21:11 rafi joined #gluster-dev
22:17 lpabon_ joined #gluster-dev
22:20 shyam joined #gluster-dev
22:58 lpabon_ joined #gluster-dev
23:12 lpabon_ joined #gluster-dev
23:24 shyam joined #gluster-dev

| Channels | #gluster-dev index | Today | | Search | Google Search | Plain-Text | summary