Perl 6 - the future is here, just unevenly distributed

IRC log for #gluster-dev, 2015-09-24

| Channels | #gluster-dev index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:05 suliba joined #gluster-dev
00:52 zhangjn joined #gluster-dev
01:48 pranithk joined #gluster-dev
02:04 vmallika joined #gluster-dev
02:53 htrmeira joined #gluster-dev
03:01 overclk joined #gluster-dev
03:03 vmallika joined #gluster-dev
03:04 skoduri joined #gluster-dev
03:22 vmallika joined #gluster-dev
03:25 skoduri joined #gluster-dev
03:44 nbalacha joined #gluster-dev
03:51 shubhendu__ joined #gluster-dev
03:57 aspandey joined #gluster-dev
04:02 itisravi joined #gluster-dev
04:08 kanagaraj joined #gluster-dev
04:13 sakshi joined #gluster-dev
04:34 kdhananjay joined #gluster-dev
04:41 pppp joined #gluster-dev
04:43 overclk joined #gluster-dev
04:47 ppai joined #gluster-dev
04:48 gem joined #gluster-dev
05:06 ndarshan joined #gluster-dev
05:18 vimal joined #gluster-dev
05:19 Bhaskarakiran joined #gluster-dev
05:21 raghu joined #gluster-dev
05:25 kdhananjay joined #gluster-dev
05:29 kotreshhr joined #gluster-dev
05:32 Manikandan joined #gluster-dev
05:32 pranithk joined #gluster-dev
05:33 nishanth joined #gluster-dev
05:38 kdhananjay joined #gluster-dev
05:39 jiffin joined #gluster-dev
05:49 hchiramm joined #gluster-dev
05:49 hchiramm_ joined #gluster-dev
05:53 anekkunt joined #gluster-dev
05:55 hgowtham joined #gluster-dev
06:12 pppp joined #gluster-dev
06:18 overclk_ joined #gluster-dev
06:20 vimal joined #gluster-dev
06:23 ggarg joined #gluster-dev
06:28 rgustafs joined #gluster-dev
06:33 aspandey joined #gluster-dev
06:34 atalur joined #gluster-dev
06:35 Backer_ joined #gluster-dev
06:36 * Backer_ slaps pranithk around a bit with a large fishbot
06:37 pranithk Backer_: hey!
06:37 Backer_ Hi Pranith
06:37 pranithk Backer_: tell me dude, what happened on the setup that lead to EIO?
06:39 Backer_ There is no workload. we stopped all upload and download operations due fixlaypout directory display issue and started fixlayout. It went smoothly up to last evening.
06:39 rjoseph joined #gluster-dev
06:40 pranithk Backer_: and?
06:41 Backer_ Now we the fixlayout is hanging with following message..[2015-09-24 05:34:59.055132] I [MSGID: 109028] [dht-rebalance.c:3042:gf_defrag_status_get] 0-glusterfs: Files migrated: 0, size: 0, lookups: 0, failures: 0, skipped: 0
06:41 pranithk Backer_: I saw quite a few EIOs from ec subvolumes that is why I am asking what happened.
06:43 Backer_ I cant understand why  log shows "some subvolumes are unavailable"
06:43 Backer_ All bricks are running and cluster stat is good
06:43 pranithk Backer_: How many files do you have on the mount?
06:45 spalai joined #gluster-dev
06:46 deepakcs joined #gluster-dev
06:46 Backer_ there is some millions files and folders. Nearly about 2 PB data.
06:46 Backer_ glustershd log shows ..... [2015-09-24 06:39:26.001589] W [MSGID: 122053] [ec-common.c:166:ec_check_status] 0-glustertestdr-disperse-77: Operation failed on some subvolumes (up=3FF, mask=3FF, remaining=0, good=3FB, bad=4) [2015-09-24 06:39:26.001626] W [MSGID: 122002] [ec-common.c:122:ec_heal_report] 0-glustertestdr-disperse-77: Heal failed [Input/output error] The message "W [MSGID: 122053] [ec-common.c:166:ec_check_status] 0-g
06:46 spalai left #gluster-dev
06:47 pranithk Backer_: Could you send me logs in either a mail or create a bz and attach them to it?
06:47 pranithk Backer_: I need to analyze them
06:48 Backer_ which log do you need ? can i send any one node full logs
06:49 pranithk Backer_: you have 30 nodes right?
06:49 pranithk Backer_: first node was doing rebalance everytime?
06:50 pranithk Backer_: I need logs of 1st 11th and 21st node logs
06:50 Backer_ currently we are doing fixlayout. so it is running on all nodes
06:50 pranithk Backer_: oh
06:50 pranithk Backer_: but you said fix layout stopped?
06:51 Backer_ No it is not stopped. Still it is showing running on rebalance status command.
06:51 Backer_ No it is not stopped. Still it is showing "running" on rebalance status command.
06:52 Backer_ Are you using teamviewer?
06:53 Backer_ you can take remote of my machine.
06:55 pranithk Backer_: I am working now. I get off at around 5 or 6. I can take a look at the machines then? send me the mail with machine details. I would love it if you give me read-only access.
06:55 Backer_ pranith?
06:56 pranithk Backer_: If I get off early I can take a look at them even before that...
06:56 pranithk Backer_: there?
06:57 pranithk Backer_: There is lunch break from 1PM-2PM. I can quickly finish lunch and spend 30 minutes but not sure if I can find RC in that time...
06:58 Backer_ I am in home town and celebrating bakrid festival.
06:58 Backer_ can i send logs to your mail id?
06:59 pranithk Backer_: ayyo! totally.
06:59 pranithk Backer_: How much time do I have for debugging the issues?
07:01 Backer_ if you want i can give the remote through teamviewer.You can work as your wish. But i have to be in online for giving remote.
07:01 pranithk Backer_: I am working on 2 customer issues in office. So can't take a look at your setup till ~6:00PM
07:02 Backer_ ok can you take 6:30 PM?
07:02 pranithk Backer_: you send me the logs. If I don't find anything I will ask you for the machine details and we can take it up based on your availability. I am not available during the weekend unfortunately. SO I prefer to find anything today itself
07:02 pranithk Backer_: sure!
07:03 Backer_ ok i will be available on 6:30PM for remote. If you need log, please let me know. i will send it now.
07:04 pranithk Backer_: send the logs right away. I will try to find some time during lunch
07:04 Backer_ which node logs do you need ?
07:04 Backer_ super...
07:04 pranithk Backer_: wherever the rebalance stopped.
07:06 Backer_ rebalance is not stopped on any node. only all the node rebalance logs are showing the message..[2015-09-24 05:34:59.055132] I [MSGID: 109028] [dht-rebalance.c:3042:gf_defrag_status_get] 0-glusterfs: Files migrated: 0, size: 0, lookups: 0, failures: 0, skipped: 0
07:07 Backer_ No other activities
07:08 pranithk Backer_: oh, so you are saying fixlayout stopped?
07:08 pranithk Backer_: Are you just worried because of this log?
07:10 nbalacha joined #gluster-dev
07:10 Backer_ No. fixlayout is not happening on some folders due to this fixlayout hangs and those folder is not showing during dir display. I want to run the fixlayout on all the folders.
07:11 pranithk Backer_: Okay, I just spoke to dht folks. They are asking for rebalance logs from all machines and xattrs of the directories where you are observing that the fix-layout is not happening
07:12 Backer_ if i stop and restart the fixlayout, it will start from scratch and again i need to wait four more days to complete the fixlayout.
07:12 ppai joined #gluster-dev
07:12 pranithk Backer_: good news. nbalacha wants to take a look at the setup
07:12 pranithk Backer_: she asks you not to stop
07:12 pranithk Backer_: she wants to take a look
07:13 pranithk Backer_: nbalacha is going to contact you. All the best!
07:13 nbalacha Backer_, hi. I'll try to help you out here
07:13 Backer_ can she join with you on your teamviewer session?
07:14 pranithk Backer_: We are complete linux shop. Could you setup 'screen' session?
07:14 pranithk Backer_: or send her ssh access
07:14 pranithk Backer_: not on the chat log :-D
07:15 pranithk Backer_: send her email. nbalacha@redhat.com
07:16 Backer_ will check about screen session...
07:16 Backer_ what does she need through her mail?
07:17 Backer_ logs?
07:17 pranithk Backer_: ssh access to the machine so that she can look at the logs, run commands etc
07:18 nbalacha Backer_, logs would be great as well
07:18 Backer_ Cluster is running on private network. So i can provide remote to any windows which she can ssh in to cluster
07:18 nbalacha Backer_, if you do not want to provide access, I will let you know what else I need
07:19 nbalacha Backer_, you can run the commands and send me the output
07:19 Backer_ ok what command and which nodes logs do you need
07:21 pranithk Backer_: try preceding your question with the person whom you are asking this question, otherwise it can get confusing... so something like "nbalacha: ok what command and which nodes logs do you need" It changes the color of IRC of the person you asked the question. So it is more helpful that way for us
07:22 Backer_ Pranithk:ok will do
07:22 pranithk Backer_: there you go!
07:23 pranithk Backer_: apparently nbalacha has sent you some messages in a private window. Could you check once?
07:23 Backer_ nbalacha: Could you send the list of command and node details which you need logs?
07:24 nbalacha Backer_, I have opened up a chat session in a private window with you
07:24 nbalacha Backer_, are you able to see that?
07:50 overclk joined #gluster-dev
07:53 ppai joined #gluster-dev
08:15 overclk joined #gluster-dev
08:34 overclk joined #gluster-dev
08:38 zhangjn joined #gluster-dev
08:43 maveric_amitc_ joined #gluster-dev
08:44 jiffin ndevos: ping , soumya has put comments on http://review.gluster.org/#/c/12219/ and http://review.gluster.org/#/c/12159/ , can you please confirm which one is right to go?
08:47 ndevos jiffin: we can include the change in the master branch too, I'm just not sure if all changes to those scripts were included in master, or some only in 3.7
08:49 ndevos jiffin: ok, a "git diff" shows that both scripts in the branches match, thats a good thing, I think
08:51 ndevos jiffin: can you un-abandon http://review.gluster.org/12159 somehow, we probably should merge that then
08:51 jiffin ndevos: sure
08:51 jiffin ndevos: done
08:53 ndevos jiffin: merged both :)
08:55 jiffin ndevos: thankss
08:55 jiffin ndevos++ for merging the patches
08:55 glusterbot jiffin: ndevos's karma is now 205
08:55 ndevos jiffin++ thanks too!
08:55 glusterbot ndevos: jiffin's karma is now 13
09:00 ppai joined #gluster-dev
09:04 Bhaskarakiran joined #gluster-dev
09:25 kotreshhr joined #gluster-dev
09:33 kdhananjay csim/others: Does anyone know why the netbsd regression run is always getting stuck at https://build.gluster.org/job/rackspace-netbsd7-regression-triggered/10473/console for hours together?
09:38 csim kdhananjay: emmanuel would know, ask on gluster-infra
09:39 overclk joined #gluster-dev
09:54 kdhananjay csim: OK, thanks.
10:03 sakshi joined #gluster-dev
10:05 Bhaskarakiran joined #gluster-dev
10:05 spalai joined #gluster-dev
10:06 ppai joined #gluster-dev
10:08 kotreshhr joined #gluster-dev
10:11 kkeithley1 joined #gluster-dev
10:23 kdhananjay joined #gluster-dev
10:34 EinstCrazy joined #gluster-dev
10:34 ppai joined #gluster-dev
10:44 kotreshhr joined #gluster-dev
10:57 Bhaskarakiran joined #gluster-dev
10:58 Bhaskarakiran joined #gluster-dev
11:02 overclk joined #gluster-dev
11:07 overclk_ joined #gluster-dev
11:09 overclk__ joined #gluster-dev
11:13 kdhananjay joined #gluster-dev
11:14 overclk joined #gluster-dev
11:14 kotreshhr joined #gluster-dev
11:27 zhangjn joined #gluster-dev
11:45 jiffin1 joined #gluster-dev
11:54 ppai joined #gluster-dev
12:05 overclk joined #gluster-dev
12:05 kotreshhr left #gluster-dev
12:12 ndk joined #gluster-dev
12:17 ppai joined #gluster-dev
12:32 jrm16020 joined #gluster-dev
12:36 overclk joined #gluster-dev
12:37 suliba joined #gluster-dev
12:40 spalai left #gluster-dev
12:42 ppp joined #gluster-dev
12:46 prasanth_ joined #gluster-dev
12:52 kotreshhr joined #gluster-dev
12:53 hagarth joined #gluster-dev
12:57 ppai joined #gluster-dev
12:57 shaunm joined #gluster-dev
12:57 jiffin joined #gluster-dev
13:00 sakshi joined #gluster-dev
13:06 kotreshhr left #gluster-dev
13:12 EinstCrazy joined #gluster-dev
13:20 maveric_amitc_ joined #gluster-dev
13:24 firemanxbr joined #gluster-dev
13:28 asengupt joined #gluster-dev
13:36 Bhaskarakiran joined #gluster-dev
13:42 pranithk left #gluster-dev
13:44 dlambrig_ joined #gluster-dev
13:45 zhangjn joined #gluster-dev
13:47 zhangjn joined #gluster-dev
13:48 zhangjn joined #gluster-dev
13:49 shyam joined #gluster-dev
13:59 zhangjn joined #gluster-dev
14:11 kdhananjay joined #gluster-dev
14:33 maveric_amitc_ joined #gluster-dev
14:40 EinstCrazy joined #gluster-dev
14:40 zhangjn joined #gluster-dev
14:41 zhangjn joined #gluster-dev
14:42 zhangjn joined #gluster-dev
14:42 zhangjn joined #gluster-dev
14:44 zhangjn joined #gluster-dev
14:51 shyam joined #gluster-dev
15:01 poornimag joined #gluster-dev
15:01 Bhaskarakiran joined #gluster-dev
15:02 cholcombe joined #gluster-dev
15:03 lpabon joined #gluster-dev
15:05 shaunm joined #gluster-dev
15:09 zhangjn_ joined #gluster-dev
15:20 Bhaskarakiran joined #gluster-dev
15:26 zhangjn joined #gluster-dev
15:28 zhangjn joined #gluster-dev
15:28 zhangjn joined #gluster-dev
15:35 zhangjn_ joined #gluster-dev
15:46 jiffin joined #gluster-dev
15:46 zhangjn joined #gluster-dev
15:52 jiffin joined #gluster-dev
15:57 jiffin1 joined #gluster-dev
16:05 jiffin joined #gluster-dev
16:06 kkeithley_ everyone needs to remember to use the libglusterfs/syscalls wrappers.  A lot of unwrapped syscalls have crept back into the source since the last time I cleaned them up. :-(
16:13 jiffin1 joined #gluster-dev
16:19 skoduri joined #gluster-dev
16:24 wolsen joined #gluster-dev
16:24 kbyrne joined #gluster-dev
16:25 kkeithley joined #gluster-dev
16:27 ndk joined #gluster-dev
16:30 poornimag joined #gluster-dev
16:34 lpabon joined #gluster-dev
16:37 overclk joined #gluster-dev
16:57 asengupt joined #gluster-dev
16:59 zhangjn_ joined #gluster-dev
17:06 kotreshhr joined #gluster-dev
17:06 kotreshhr left #gluster-dev
17:08 zhangjn joined #gluster-dev
17:15 RedW joined #gluster-dev
17:21 maveric_amitc_ joined #gluster-dev
17:34 suliba joined #gluster-dev
17:35 poornimag joined #gluster-dev
17:37 overclk joined #gluster-dev
17:40 overclk joined #gluster-dev
17:49 jiffin joined #gluster-dev
17:52 shyam joined #gluster-dev
18:00 amye joined #gluster-dev
18:25 jobewan joined #gluster-dev
18:25 jrm16020 joined #gluster-dev
18:29 amye joined #gluster-dev
18:42 jiffin1 joined #gluster-dev
18:49 ggarg joined #gluster-dev
18:54 jiffin1 joined #gluster-dev
19:37 shaunm joined #gluster-dev
20:24 htrmeira joined #gluster-dev
20:43 poornimag joined #gluster-dev
20:47 skoduri joined #gluster-dev
21:45 poornimag joined #gluster-dev
22:09 zhangjn joined #gluster-dev
22:12 zhangjn joined #gluster-dev
22:13 zhangjn joined #gluster-dev
22:36 zhangjn joined #gluster-dev
22:38 zhangjn joined #gluster-dev
22:57 zhangjn_ joined #gluster-dev
23:29 poornimag joined #gluster-dev
23:41 htrmeira joined #gluster-dev

| Channels | #gluster-dev index | Today | | Search | Google Search | Plain-Text | summary