Perl 6 - the future is here, just unevenly distributed

IRC log for #gluster-dev, 2015-04-07

| Channels | #gluster-dev index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:09 shyam joined #gluster-dev
00:13 purpleidea joined #gluster-dev
00:22 dlambrig_ joined #gluster-dev
01:16 _Bryan_ joined #gluster-dev
01:51 ilbot3 joined #gluster-dev
01:51 Topic for #gluster-dev is now Gluster Development Channel - http://gluster.org | For general chat go to #gluster | Patches - http://review.gluster.org/ | Channel Logs - https://botbot.me/freenode/gluster-dev/ & http://irclog.perlgeek.de/gluster-dev/
02:30 vimal joined #gluster-dev
02:34 purpleidea joined #gluster-dev
02:47 dlambrig_ joined #gluster-dev
03:33 kkeithley1 joined #gluster-dev
03:33 kshlm joined #gluster-dev
03:38 ira joined #gluster-dev
03:39 hgowtham joined #gluster-dev
03:40 ashiq joined #gluster-dev
03:44 ashiq- joined #gluster-dev
03:46 itisravi joined #gluster-dev
03:52 kkeithley1 joined #gluster-dev
03:59 atinmu joined #gluster-dev
04:00 hgowtham joined #gluster-dev
04:02 dlambrig_ joined #gluster-dev
04:12 overclk joined #gluster-dev
04:13 kanagaraj joined #gluster-dev
04:21 anoopcs joined #gluster-dev
04:23 jiffin joined #gluster-dev
04:26 rafi joined #gluster-dev
04:26 ashiq- joined #gluster-dev
04:28 Manikandan joined #gluster-dev
04:37 hagarth joined #gluster-dev
04:42 purpleidea joined #gluster-dev
05:00 ndarshan joined #gluster-dev
05:01 soumya_ joined #gluster-dev
05:03 spandit joined #gluster-dev
05:05 hgowtham joined #gluster-dev
05:11 vimal joined #gluster-dev
05:14 vimal joined #gluster-dev
05:17 dlambrig_ joined #gluster-dev
05:23 kkeithley1 joined #gluster-dev
05:24 gem joined #gluster-dev
05:27 lalatenduM joined #gluster-dev
05:36 ppai joined #gluster-dev
05:40 kdhananjay joined #gluster-dev
05:40 ashiq- joined #gluster-dev
05:42 hchiramm joined #gluster-dev
05:44 nishanth joined #gluster-dev
05:45 deepakcs joined #gluster-dev
05:51 jiffin joined #gluster-dev
05:53 ira_ joined #gluster-dev
05:53 jiffin joined #gluster-dev
05:56 aravindavk joined #gluster-dev
05:57 raghu joined #gluster-dev
05:59 ashiq- joined #gluster-dev
06:07 nkhare joined #gluster-dev
06:19 hchiramm tigert++
06:19 glusterbot hchiramm: tigert's karma is now 1
06:20 tigert ha!
06:20 ndevos wohoo!
06:20 * tigert is not zero anymore o/
06:22 ashiq- joined #gluster-dev
06:26 anrao joined #gluster-dev
06:27 DV_ joined #gluster-dev
06:32 hagarth kshlm: any thoughts on http://review.gluster.org/#/c/10054/ ?
06:46 kshlm hagarth, LGTM
06:46 hagarth kshlm: cool, merging that one now.
06:47 pranithk joined #gluster-dev
06:47 pranithk left #gluster-dev
06:47 asengupt joined #gluster-dev
06:53 Manikandan joined #gluster-dev
07:16 DV joined #gluster-dev
07:16 Manikandan joined #gluster-dev
07:22 pranithk joined #gluster-dev
07:44 DV joined #gluster-dev
07:45 ashiq joined #gluster-dev
07:45 hgowtham joined #gluster-dev
08:00 pranithk joined #gluster-dev
08:04 aravindavk joined #gluster-dev
08:07 DV_ joined #gluster-dev
08:16 ashiq joined #gluster-dev
08:19 ernetas http://review.gluster.org/10126/ - could someone review this one?
08:30 DV_ joined #gluster-dev
08:30 anoopcs joined #gluster-dev
08:33 anrao joined #gluster-dev
08:38 dlambrig__ joined #gluster-dev
08:49 hagarth joined #gluster-dev
09:02 ndevos thanks ernetas, merged!
09:02 ernetas Thanks! :)
09:12 bala1 joined #gluster-dev
09:13 anrao joined #gluster-dev
09:22 soumya_ joined #gluster-dev
09:42 dlambrig_ joined #gluster-dev
10:04 pranithk joined #gluster-dev
10:31 pranithk joined #gluster-dev
10:47 anrao joined #gluster-dev
10:59 DV__ joined #gluster-dev
11:02 itisravi joined #gluster-dev
11:17 itisravi joined #gluster-dev
11:18 itisravi_ joined #gluster-dev
11:39 sac joined #gluster-dev
11:45 ndevos REMINDER: Gluster Bug Triage Meeting starts in 15 minutes in #gluster-meeting
11:54 Gaurav_ joined #gluster-dev
11:56 soumya_ joined #gluster-dev
12:00 hchiramm joined #gluster-dev
12:00 ndevos REMINDER: Gluster Bug Triage Meeting starting now in #gluster-devel
12:01 ndevos no, *not* here! -> #gluster-meeting
12:07 rafi atinmu++
12:07 glusterbot rafi: atinmu's karma is now 9
12:12 hagarth joined #gluster-dev
12:13 anoopcs joined #gluster-dev
12:14 ira joined #gluster-dev
12:19 Gaurav_ joined #gluster-dev
12:39 kanagaraj joined #gluster-dev
12:55 kanagaraj_ joined #gluster-dev
12:58 DV joined #gluster-dev
13:03 purpleidea joined #gluster-dev
13:03 purpleidea joined #gluster-dev
13:04 shyam2 joined #gluster-dev
13:08 ira joined #gluster-dev
13:09 ppai joined #gluster-dev
13:26 owlbot joined #gluster-dev
14:04 __test joined #gluster-dev
14:05 overclk joined #gluster-dev
14:10 kshlm joined #gluster-dev
14:11 overclk hagarth, ping. netbsd smoke failure: http://build.gluster.org/job/netbsd6-smoke/4712/console
14:11 overclk hagarth, any idea?
14:12 ndevos overclk: I think the git service is a little overloaded or something, checkout is sometimes very slow (or not working)
14:13 _Bryan_ joined #gluster-dev
14:13 hagarth overclk: no idea, let me check logs there
14:14 JustinClift overclk: Retrigger it
14:15 JustinClift Yeah, I suspect the git service might be getting hit too hard
14:15 JustinClift hagarth: Do the logs show a different problem?
14:15 * JustinClift hasnt looked
14:16 JustinClift hagarth: We could shut down Gerrit for a few minutes, and run git gc as the review user on the repo
14:16 overclk JustinClift, retriggered and within minutes the same result.
14:17 overclk ndevos, thanks! then probably it should get resolved soon..
14:18 ndevos overclk: yes, certainly now JustinClift and hagarth are both checking it :D
14:20 shyam joined #gluster-dev
14:20 hagarth JustinClift: / is 96% full
14:22 ndevos you post too many patches!
14:24 jobewan joined #gluster-dev
14:36 hagarth JustinClift: would need to clean up some space in /
14:36 wushudoin joined #gluster-dev
14:37 JustinClift hagarth: Looking now
14:38 wushudoin joined #gluster-dev
14:39 JustinClift hagarth: /home/amar is 33GB
14:39 JustinClift Hmmmm
14:39 hagarth JustinClift: but that doesn't add to /
14:39 JustinClift Gah
14:40 JustinClift Hmmm root partition is only 7.2GB on the server.  That's pretty small. :/
14:41 JustinClift hagarth: The /git/ directory for glusterfs is in /git/ partion, not on /
14:41 JustinClift hagarth: That has 211GB free
14:41 hagarth JustinClift: but logs etc. from applications will hit /
14:41 hagarth and might get a ENOSPC
14:41 lpabon joined #gluster-dev
14:41 hagarth JustinClift: we probably need to have /var as a separate partition at some point in time
14:42 JustinClift hagarth: the /review partition is on /
14:42 JustinClift s/parition/dir/
14:42 JustinClift We could shut down Gerrit, move that to [some other partition], and soft link it
14:42 hagarth JustinClift: yep!
14:43 JustinClift That would make abt 1GB free on /
14:43 hagarth and maybe git-daemon too
14:43 JustinClift Yeah
14:43 JustinClift Sound workable?  If so, I'll do it right now. ;)
14:43 hagarth JustinClift: go ahead, that's the only option atm
14:44 hagarth rather looks like the only option ;)
14:45 JustinClift Sure
14:45 JustinClift Email sent to gluster-devel + gluster-infra warning ppl
14:45 dlambrig_ joined #gluster-dev
14:47 hagarth JustinClift: need to move /var/mail away from / too
14:48 JustinClift hagarth: k
14:48 JustinClift Services shut down atm.
14:48 JustinClift Moving /review into /opt/review now
14:49 hagarth JustinClift: ok
14:49 JustinClift Hopefully the new NFS based home for the /review dir doesn't cause weirdness
14:50 JustinClift We have nightly backup though, just in case it all goes bad
14:50 JustinClift :/
14:51 hagarth JustinClift: haha, let us see. If it works well, we might replace NFS with gluster NFS ;)
14:51 JustinClift I hear 3.6.3 might be stable :p
14:52 JustinClift Lets be daring and put it on 3.7-master? ;D
14:53 JustinClift hagarth: Any preferences for where to put /var/mail ?
14:53 hagarth JustinClift: I would not mind that. eating our own dogfood is the best way to mature any software.
14:53 hagarth JustinClift: move to /share ?
14:54 JustinClift Sure
14:54 hagarth ndevos: how do you look the idea of replacing kernel nfs with gluster nfs in r.g.o deployment?
14:57 JustinClift hagarth: Do we have a standard set of git gc options we use?
14:57 JustinClift eg --auto or --aggressive or ?
14:58 JustinClift If no, lets just go with "git gc" and see what happens. ;)
14:58 hagarth vanilla git gc
14:58 JustinClift k
14:59 JustinClift I don't do git gc enough to remember the difference.  Apart from the prune option. ;)
14:59 JustinClift It's running now.
14:59 pranithk joined #gluster-dev
15:02 JustinClift hagarth: Do we be super daring and yum upate + reboot it?
15:02 JustinClift ... we do have working backups ...
15:02 hagarth JustinClift: go ahead
15:08 JustinClift Yum is updating now
15:15 JustinClift Rebooting
15:16 misc rebooting which ?
15:17 JustinClift review.gluster.org
15:17 JustinClift misc: Yes, one of the ones we've been pretty careful of
15:18 JustinClift ... and it's not coming back. :/
15:18 overclk joined #gluster-dev
15:18 JustinClift Dammiot
15:18 JustinClift Dammit
15:18 misc JustinClift: fsck
15:19 JustinClift Could be
15:19 JustinClift misc: I've just logged into engg (the VM host for it)
15:19 JustinClift I have no clue with Xen though :/
15:20 JustinClift misc: virsh is showing 3 domains running
15:20 JustinClift 1 is Domain-0, so I'm getting that's the Xen hypervisor
15:21 JustinClift And there's shell.gluster.org (review.gluster.org?) and build.gluster.org.
15:24 JustinClift misc: Do you have time to look into why review.gluster.org isn't coming back?
15:25 misc JustinClift: as soon as I finish uploading iso to racksapce
15:25 misc according to the web interface, it should be done in -120 seconds
15:25 misc so I expect a new breaktrought in physics to understand that and then i can look at xen :)
15:26 misc JustinClift: so virsh on what server ?
15:26 JustinClift misc: http://fpaste.org/208022/84203891/
15:27 JustinClift misc: That's the virsh in engg.gluster.com
15:28 JustinClift misc: If it goes pear shaped, we can probably get last night's backup functional in a new VM in Rackspace in under 30 mins
15:28 JustinClift We'll need to generate new ssh keys for the clients... but well, tough. ;)
15:29 misc we will save it
15:29 JustinClift Yeah, I suspect so
15:29 misc mhh, how do we access engg.gluster ? bounce from build.gluster ?
15:29 JustinClift Hoping it actually is a fsck or something
15:29 JustinClift misc: I just ssh there directly
15:30 misc JustinClift: ssh: Could not resolve hostname engg.gluster.com: Name or service not known
15:30 misc mhh
15:30 misc my dns is broken
15:30 JustinClift 184.107.76.9
15:30 JustinClift ^ engg.gluster.com
15:32 misc nope, I am supposed to have the password ? ( refuse my ssh keys )
15:32 * JustinClift looks
15:32 misc I know I connected in the past
15:33 JustinClift I connected directly as root
15:33 JustinClift Guessing you tried a username?  Try root?
15:33 shyam joined #gluster-dev
15:33 JustinClift misc: Oh wow, the authorized_key file doesn't have your ssh keys in it
15:33 JustinClift misc: And yeah, I'm really sure you've been on there before
15:34 JustinClift misc: I'll add you now
15:34 misc JustinClift: I think I did jump from builder or something
15:34 hagarth JustinClift: can I get on there? I have managed xen
15:35 misc well, virsh console is the way to get the console access, no ?
15:35 JustinClift hagarth: Your keys are on it
15:35 JustinClift misc: I've added yours too
15:36 JustinClift hagarth: Your shot first I guess :)
15:36 hagarth JustinClift: on there, booting up r.g.o now
15:37 misc yep, it was just stopped
15:37 * misc watch console
15:37 misc and it is back
15:37 JustinClift k, briefly, what should have I done instead? :)
15:37 hagarth JustinClift: xm create /path/to/r.g.o
15:38 JustinClift Ahhh
15:38 misc mhh, why isn't the VM in libvirt ?
15:39 JustinClift hagarth: We don't have it set to auto-start via libvirt then.  Gotcha.
15:39 JustinClift hagarth: Thanks
15:39 * JustinClift is starting Gerrit now
15:39 * hagarth bbl, dinner + bed time for kid etc. :D
15:40 JustinClift hagarth: Cool.  Thanks for the assist. :)
15:40 JustinClift overclk: You around?
15:43 JustinClift misc: review.gluster.org has been updated to 5.11 via yum now btw.
15:43 JustinClift misc: It's package set is now current
15:47 kdhananjay joined #gluster-dev
15:50 JustinClift misc: Looking at the maillog on review.gluster.org, it doesn't seem happy? http://fpaste.org/208029/42842179/
15:52 JustinClift hagarth: getting reports in private IRC that logging into GerritUI is now failing
15:52 JustinClift Just tried it, and same here
15:52 JustinClift hagarth: I'm going to shut down Gerrit, move it back to the / partition, and see if that fixes it
15:56 misc JustinClift: indeed
15:56 misc JustinClift: let me finish to upload iso ( again) and I cna look
16:06 JustinClift Well
16:06 JustinClift Gerrit is back into it's old spot
16:06 JustinClift And has been restarted
16:06 JustinClift I can't sign in though :(
16:12 JustinClift Ahhh.
16:12 JustinClift Possibly clock skew related
16:14 hagarth JustinClift: seems to be the case - auth.openid.OpenIdServiceImpl : OpenID failure: Nonce verification failed.  Likely caused by clock skew on this server, install/configure NTP.
16:14 JustinClift misc: With the mail problem, I do remember the yum update created an /etc/postfix/main.cf.rpmnew
16:14 JustinClift misc: I did a diff of the main.cf and .rpmnew one and didn't see anything super obvious to me
16:14 JustinClift misc: But, there might be something obviously wrong in the diff to you
16:14 Debloper joined #gluster-dev
16:15 JustinClift hagarth: Yeah, I'm shutting down Gerrit again, then will fix the skew and start it up
16:15 * JustinClift doesn't like screwing with clock skew with open databases
16:16 hagarth JustinClift: ok, move it back to /opt/review ?
16:19 hagarth JustinClift: my r.g.o filter in thunderbird is broken after the reboot .. wonder why
16:20 hagarth JustinClift: from has changed - review at d.g.o to review at d.g.com
16:20 JustinClift hagarth: I can move the dir back into /opt, yeah
16:20 JustinClift Oh, name change too?
16:20 JustinClift Interesting
16:21 JustinClift hagarth: /etc/sysconfig/network has "HOSTNAME=dev.gluster.com"
16:21 JustinClift hagarth: should that be .org?>
16:21 hagarth yeah
16:21 JustinClift Can change it right now.  We should reboot it again for good measure though, just to be sure.
16:22 JustinClift Gimme a sec btw, as ntpdate isn't forcing the date change and I'm not sure why
16:24 JustinClift Moving /review dir now
16:28 JustinClift So, it seems like engg.gluster.com needs the time changed
16:29 JustinClift I'm pretty sure that'll screw with Jenkins, but no real choice :(
16:29 JustinClift I'll shut down Jenkins temporarily
16:29 * JustinClift sighs
16:39 JustinClift k, rebooted r.g.o, and xm created it
16:43 JustinClift hagarth misc: I can't seem to affect the time in review.gluster.org VM
16:44 JustinClift I've run ntpdate in the VM --> does nothing.  Run ntpdate in the host box --> time changed -> Rebooted r.g.o VM --> time unchanged
16:44 JustinClift Nothing I'm doing affects the damn time
16:44 JustinClift Any idas
16:44 JustinClift Any ideas?
16:47 jobewan joined #gluster-dev
16:48 Manikandan joined #gluster-dev
17:04 hagarth JustinClift: hang on, let me look around
17:05 JustinClift hagarth: Hold on
17:05 hagarth JustinClift: ok
17:05 JustinClift rjones in RH IRC is helping out
17:06 JustinClift The issue seems to be the "clocksource" for the VM
17:06 JustinClift The one for build.gluster.org is fine
17:06 JustinClift The one for review.gluster.org isn't
17:07 hagarth JustinClift: I see, bbiab
17:08 JustinClift hagarth: k
17:09 Manikandan joined #gluster-dev
17:12 foster joined #gluster-dev
17:15 JustinClift Restarting the VM again
17:15 JustinClift Lets see if the sysctl variable + ntpd keep it in shape now...
17:17 JustinClift No, it doesn't
17:20 JustinClift Rebooting again...
17:26 anrao joined #gluster-dev
17:27 JustinClift For some unknown reason, after rebooting it's _always_ out by -1420 seconds
17:27 JustinClift I can't see where it's getting that figure from
17:28 misc ntpdate in rc.d ?
17:30 JustinClift Nope
17:30 JustinClift Anway, it's manually fixed atm
17:31 JustinClift I've shut down the services in the domain (eg MySQL, Gerrit, etc), ran ntpdate to get the clock accurate, then started them back up again
17:31 JustinClift I can log into Gerrit UI now, so that bits working
17:31 JustinClift If we intend to reboot this box again, we'll have to remember to do this :/
17:31 JustinClift Now, to fix the jobs on Jenkins which barfed when I stopped that... :/
17:36 hchiramm_ joined #gluster-dev
17:40 hagarth JustinClift: is everything ok now?
17:42 JustinClift hagarth: I think so
17:42 JustinClift hagarth: I've just retriggered all of the Jenkins jobs that aborted when I restarted Jenkins too
17:43 JustinClift hagarth: They seem happy with git cloning and stuff, which is positive
17:43 JustinClift hagarth: Can you login to Gerrit ok?
17:43 hagarth JustinClift: thanks for the re-trigger. I think we should evolve a script some time to retrigger Jenkins jobs that get aborted
17:44 * hagarth logs out of gerrit
17:44 JustinClift Cool
17:44 JustinClift hagarth: Hmmm, yeah, useful idea :)
17:44 * hagarth is able to login back
17:44 JustinClift Cool
17:44 JustinClift I think that's sorted then
17:44 hagarth seems like we're doing good.
17:44 JustinClift Well, at least we updated the damn Gerrit box ;)
17:45 JustinClift Less chance of remote exploit :/
17:45 hagarth unscheduled outage FTW! ;)
17:46 hagarth my mail filter for gerrit is also working fine. so all seems to be well.
17:47 dlambrig_ joined #gluster-dev
17:48 JustinClift hagarth: Cool.  I'm going to collapse in a chair for a while do not look at a computer terminal for 20 mins. ;)
17:49 hagarth JustinClift: me too will crash soon. enjoy the rest of your evening!
17:50 JustinClift Sleep well. :)
17:54 dlambrig_ left #gluster-dev
18:18 lalatenduM joined #gluster-dev
18:31 anrao joined #gluster-dev
18:32 hagarth joined #gluster-dev
18:35 _Bryan_ joined #gluster-dev
18:37 lpabon joined #gluster-dev
19:29 _Bryan_ joined #gluster-dev
19:31 shaunm joined #gluster-dev
19:57 shyam joined #gluster-dev
19:57 hchiramm_ joined #gluster-dev
20:17 DV joined #gluster-dev
20:19 hchiramm_ joined #gluster-dev
20:27 purpleidea joined #gluster-dev
20:27 purpleidea joined #gluster-dev
20:44 purpleidea joined #gluster-dev
20:44 purpleidea joined #gluster-dev
21:16 badone_ joined #gluster-dev
21:32 badone__ joined #gluster-dev
22:12 hchiramm joined #gluster-dev
22:44 hchiramm joined #gluster-dev

| Channels | #gluster-dev index | Today | | Search | Google Search | Plain-Text | summary