Perl 6 - the future is here, just unevenly distributed

IRC log for #gluster, 2017-02-16

| Channels | #gluster index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
01:03 shdeng joined #gluster
01:44 daMaestro joined #gluster
01:49 plarsen joined #gluster
02:02 Gambit15 joined #gluster
02:09 sbulage joined #gluster
02:26 pjrebollo joined #gluster
02:36 f0rpaxe joined #gluster
02:46 skoduri joined #gluster
03:03 magrawal joined #gluster
03:29 Jacob843 joined #gluster
03:33 nbalacha joined #gluster
03:42 RameshN joined #gluster
03:43 Shu6h3ndu joined #gluster
03:55 nishanth joined #gluster
04:04 pjrebollo joined #gluster
04:06 itisravi joined #gluster
04:10 atinm joined #gluster
04:19 skumar joined #gluster
04:31 msvbhat joined #gluster
04:37 buvanesh_kumar joined #gluster
04:45 sbulage joined #gluster
04:57 Prasad joined #gluster
05:00 rafi joined #gluster
05:11 apandey joined #gluster
05:16 k4n0 joined #gluster
05:18 kramdoss_ joined #gluster
05:21 ndarshan joined #gluster
05:23 sbulage joined #gluster
05:24 ankitr joined #gluster
05:29 sanoj joined #gluster
05:30 msvbhat joined #gluster
05:37 buvanesh_kumar joined #gluster
05:38 kotreshhr joined #gluster
05:39 skoduri joined #gluster
05:42 karthik_us joined #gluster
05:45 kdhananjay joined #gluster
05:47 sbulage joined #gluster
05:48 riyas joined #gluster
05:52 pjrebollo joined #gluster
05:53 sbulage joined #gluster
05:59 rjoseph joined #gluster
06:00 ppai joined #gluster
06:00 Prasad_ joined #gluster
06:01 ashiq joined #gluster
06:03 sanoj joined #gluster
06:04 Prasad joined #gluster
06:06 Prasad joined #gluster
06:06 daMaestro joined #gluster
06:07 vbellur left #gluster
06:07 ankitr joined #gluster
06:09 mb_ joined #gluster
06:14 rafi joined #gluster
06:16 Karan joined #gluster
06:22 sbulage joined #gluster
06:23 susant joined #gluster
06:36 rafi joined #gluster
06:42 susant joined #gluster
06:45 nishanth joined #gluster
06:48 msvbhat joined #gluster
06:49 jkroon joined #gluster
06:50 apandey joined #gluster
07:00 poornima joined #gluster
07:02 R0ok_ joined #gluster
07:05 jiffin joined #gluster
07:09 ItsMe` joined #gluster
07:20 msvbhat joined #gluster
07:25 jtux joined #gluster
07:27 mhulsman joined #gluster
07:28 mhulsman joined #gluster
07:29 sbulage joined #gluster
07:35 unlaudable joined #gluster
07:38 shutupsquare joined #gluster
07:41 ivan_rossi joined #gluster
07:42 ashiq joined #gluster
07:48 Prasad_ joined #gluster
07:49 sbulage joined #gluster
07:51 [diablo] joined #gluster
07:58 k4n0 joined #gluster
08:05 mbukatov joined #gluster
08:08 arpu joined #gluster
08:08 jtux joined #gluster
08:13 mhulsman joined #gluster
08:16 fsimonce joined #gluster
08:25 pulli joined #gluster
08:25 derjohn_mob joined #gluster
08:32 sbulage joined #gluster
08:33 kotreshhr left #gluster
08:42 saali joined #gluster
08:44 pulli joined #gluster
08:46 k4n0 joined #gluster
08:54 Prasad__ joined #gluster
08:54 mhulsman joined #gluster
08:56 atinm joined #gluster
09:00 ahino joined #gluster
09:05 rafi joined #gluster
09:08 foggy joined #gluster
09:10 kotreshhr joined #gluster
09:10 mhulsman joined #gluster
09:12 Prasad_ joined #gluster
09:19 Wizek_ joined #gluster
09:20 derjohn_mob joined #gluster
09:27 apandey joined #gluster
09:35 shutupsquare joined #gluster
09:48 Jacob843 joined #gluster
10:08 pjrebollo joined #gluster
10:10 Philambdo joined #gluster
10:13 jwd joined #gluster
10:15 Seth_Karlo joined #gluster
10:16 Seth_Karlo joined #gluster
10:26 Seth_Karlo joined #gluster
10:33 pjrebollo joined #gluster
10:37 pjrebollo joined #gluster
11:00 Arrakaij joined #gluster
11:00 atinm joined #gluster
11:00 kotreshhr joined #gluster
11:02 Arrakaij hello, i have some trouble here with gluster i might need some help to solve
11:02 Arrakaij i have a test setup to get a feeling for it, which is 11 bricks with distributed volume
11:03 Arrakaij it basically works fine, speeds are good, however, after half a day pushing files to the volume, the speed suddenly dropped dramatically from 110mb/s to 2 or 3 mb/s
11:04 Arrakaij now in that state, it doesnt recover itself, only if i restart the service. then after 5 minutes it drops again
11:05 Arrakaij the disks are fine, i can write on them through the fs with normal speeds if that happens
11:06 Arrakaij i just don't know where to look to see whats wrong.
11:14 sbulage joined #gluster
11:30 tallmocha joined #gluster
11:31 Arrakaij_ joined #gluster
11:33 ppai joined #gluster
11:36 Philambdo joined #gluster
11:40 jtux joined #gluster
11:42 hybrid512 joined #gluster
11:46 Karan joined #gluster
11:46 rjoseph joined #gluster
11:46 jiffin joined #gluster
11:46 atinm joined #gluster
11:46 ankitr joined #gluster
11:46 itisravi joined #gluster
11:46 ndarshan joined #gluster
11:46 rafi joined #gluster
11:46 RameshN joined #gluster
11:46 shruti joined #gluster
11:46 susant joined #gluster
11:46 kotreshhr joined #gluster
11:46 skumar joined #gluster
11:46 skoduri joined #gluster
11:46 sanoj joined #gluster
11:46 nbalacha joined #gluster
11:46 ashiq joined #gluster
11:46 kdhananjay joined #gluster
11:46 Humble joined #gluster
11:46 buvanesh_kumar joined #gluster
11:47 lalatenduM joined #gluster
11:48 apandey joined #gluster
11:48 pjrebollo joined #gluster
11:50 ndarshan joined #gluster
12:03 nh2 joined #gluster
12:07 Arrakaij g2g, i try again later
12:09 k4n0 joined #gluster
12:16 msvbhat joined #gluster
12:21 atinm joined #gluster
12:23 jtux joined #gluster
12:25 kotreshhr joined #gluster
12:43 humblec joined #gluster
12:46 shutupsquare joined #gluster
12:49 sac joined #gluster
13:04 Seth_Karlo joined #gluster
13:05 unclemarc joined #gluster
13:05 Arrakaij_ joined #gluster
13:05 Seth_Karlo joined #gluster
13:07 msvbhat joined #gluster
13:10 petan joined #gluster
13:12 derjohn_mob joined #gluster
13:16 k4n0 joined #gluster
13:18 ira joined #gluster
13:21 kotreshhr joined #gluster
13:23 baber joined #gluster
13:24 karthik_us joined #gluster
13:28 pjrebollo joined #gluster
13:43 k4n0 joined #gluster
13:43 msvbhat joined #gluster
13:51 msvbhat joined #gluster
13:52 ankitr joined #gluster
14:09 Arrakaij hello, is someone online who could help me with analysing an performance issue with gluster?
14:16 nbalacha joined #gluster
14:20 freepe joined #gluster
14:20 d-fence joined #gluster
14:23 buvanesh_kumar joined #gluster
14:24 ivan_rossi joined #gluster
14:25 tallmocha joined #gluster
14:27 kotreshhr left #gluster
14:34 skylar joined #gluster
14:36 susant left #gluster
14:39 gem joined #gluster
14:43 shyam joined #gluster
14:48 vbellur joined #gluster
14:55 buvanesh_kumar joined #gluster
14:57 jeffspeff joined #gluster
15:01 gem joined #gluster
15:04 nbalacha joined #gluster
15:13 ankitr joined #gluster
15:14 nishanth joined #gluster
15:22 Gambit15 joined #gluster
15:45 farhorizon joined #gluster
15:53 squizzi joined #gluster
16:07 wushudoin joined #gluster
16:07 jiffin joined #gluster
16:39 mpiet_cloud joined #gluster
16:40 vbellur joined #gluster
16:44 farhorizon joined #gluster
16:45 hybrid512 joined #gluster
16:46 jdossey joined #gluster
17:01 rejy joined #gluster
17:09 hybrid5121 joined #gluster
17:10 hybrid512 joined #gluster
17:16 jkroon joined #gluster
17:16 kpease joined #gluster
17:16 farhorizon joined #gluster
17:19 vbellur joined #gluster
17:20 nishanth joined #gluster
17:21 ivan_rossi left #gluster
17:43 tallmocha joined #gluster
17:44 jiffin joined #gluster
17:59 tallmocha joined #gluster
18:01 jwd joined #gluster
18:02 skylar joined #gluster
18:05 farhorizon joined #gluster
18:10 farhorizon joined #gluster
18:15 tallmocha joined #gluster
18:16 ira joined #gluster
18:17 vbellur joined #gluster
18:43 shutupsquare joined #gluster
18:49 f0rpaxe joined #gluster
18:53 shyam joined #gluster
18:54 vbellur joined #gluster
18:59 squizzi joined #gluster
19:05 oajs joined #gluster
19:14 Humble joined #gluster
19:16 baber joined #gluster
19:22 tallmocha joined #gluster
19:23 ahino joined #gluster
19:39 Gambit15 Hey, anyone around? I'm having an issue deleting a volume which went awry during creation & now only appears on a single peer
19:39 Gambit15 http://lists.gluster.org/pipermail/gl​uster-users/2017-February/029977.html
19:39 glusterbot Title: [Gluster-users] Failed snapshot clone leaving undeletable orphaned volume on a single peer (at lists.gluster.org)
19:40 Gambit15 0-glusterd: Staging failed on s1. Error: Volume data-teste does not exist
19:40 Gambit15 etc
19:40 Gambit15 JoeJulian, you in the vicinity?
19:40 JoeJulian mostly
19:40 Gambit15 Huzzah!
19:42 JoeJulian Gambit15: If you just want to force the volume to be gone, you can delete its directory from /var/lib/glusterd/vols/$diediedie
19:43 Gambit15 So simply deleting that directory on the misbehaving server & restarting glusterd should fix the issue?
19:44 Gambit15 There are no other files or such which'll need updating?
19:44 JoeJulian right
19:45 Gambit15 Should I stop glusterd beforehand, is restarting it after the act sufficient?
19:45 Gambit15 *or
19:45 JoeJulian For that one, after should be sufficient.
19:48 msvbhat joined #gluster
19:48 Gambit15 rm -rf /var/lib/glusterd/vols/data-teste/; systemctl restart glusterd; gluster volume list
19:48 Gambit15 It's still showing data-teste
19:49 Gambit15 And the dir's returned
19:49 Gambit15 Will try stopping it first
19:50 shyam joined #gluster
19:51 Gambit15 JoeJulian, nope, the dir was recreated again
19:51 jdossey_ joined #gluster
19:51 JoeJulian That means it's grabbing the configuration from another server when it starts.
19:52 JoeJulian If you don't have server quorum enabled, you should be able to stop all your glusterd, delete it, then start them again.
19:53 Gambit15 And nothing in the logs
19:53 JoeJulian brb... heading downstairs to grab some lunch.
19:53 Gambit15 But gluster's complaint is that the other peers *don't* know about the volume :S
19:54 i64 Hello, in a tiered gluster volume, what happens when the hot tier fails?
19:55 Gambit15 Yup, /var/lib/glusterd/vols/data-teste only exists on s0 (of s0-s3)
20:00 vbellur joined #gluster
20:07 skylar joined #gluster
20:22 ashiq joined #gluster
20:32 shutupsquare Does anyone know what: no active sinks for performing self-heal on file <gfid> means?
20:32 shutupsquare I have checked the file in question on both bricks and checksum is the same
20:35 JoeJulian shutupsquare: It means the ,,(extended attributes) on that file (from the brick viewpoint) show a change pending for *another* brick. The brick that should receive that change is not active (according to the client that produced that message).
20:35 glusterbot shutupsquare: (#1) To read the extended attributes on the server: getfattr -m .  -d -e hex {filename}, or (#2) For more information on how GlusterFS uses extended attributes, see this article: http://pl.atyp.us/hekafs.org/index.php/​2011/04/glusterfs-extended-attributes/
20:35 JoeJulian Gambit15: Huh, I never looked to see if a sync would pull from memory or disk. Sounds like it's from memory.
20:36 tallmocha joined #gluster
20:40 shutupsquare JoeJulian: Hey thanks, ill give it a read now
20:42 Arrakaij Hello, i have an performance issue here i need help with analysing.
20:42 derjohn_mob joined #gluster
20:43 Arrakaij i have setup a testsystem with 11 bricks which works fine in the beginning, then the glusterfsd process drops speed drastically after some time until i restart the glusterd service.
20:43 Arrakaij after that it works good for a while, then dropping again.
20:44 Arrakaij i dont know where to look into why that is
20:45 JoeJulian My first two guesses: you have files that are being healed or your servers are going in to swap.
20:46 Arrakaij the memory on the servers is 50% used while that happens
20:46 Gambit15 JoeJulian: So a reboot'll be in order then?
20:46 Arrakaij so i'd rule that out, how to see if files are getting healed
20:47 Gambit15 Arrakaij: gluster volume heal <vol> info
20:48 Arrakaij is not of type replica or disperse
20:48 Arrakaij distributed volume
20:52 Arrakaij i also guessed the fs itself is the cause, but when that happens i am able to write with normal speeds onto the filesystem affected
20:52 Arrakaij using zfs behind gluster
20:53 Seth_Karlo joined #gluster
20:54 Seth_Karlo joined #gluster
20:59 Arrakaij i also dropped the io-thread count to 1 to rule out that the disk performance screws things up.
21:00 Arrakaij i push a mix of large and small files to the server with 1gbit, and the speed is around 80mb/s which is fine, but then drops to 2-3mb/s, normally on the large (40-100gb) files
21:11 plarsen joined #gluster
21:25 baber joined #gluster
21:25 vbellur joined #gluster
21:35 Philambdo joined #gluster
21:40 mhulsman joined #gluster
21:57 pjrebollo joined #gluster
22:03 JoeJulian Arrakaij: zfs makes me consider, even more, looking at memory usage.
22:03 JoeJulian You're not using dedup, are you?
22:04 Arrakaij nope
22:04 Arrakaij but memory usage is at around 4gb only
22:04 Arrakaij the testserver just has 8 but it seems okay to me
22:06 JoeJulian That was exactly the problem I was having the one time I tried zfs, so maybe I'm the wrong person to ask.
22:06 JoeJulian There's a lot of people in here that love zfs. Any of you want to chime in?
22:07 Arrakaij just tweaked zfs a bit tho and give it a shot, especially txg_timeout
22:08 Arrakaij whats too less memory in terms of glusterfs tho?
22:09 Arrakaij is it just bad if theres no memory left or is there more to it?
22:10 JoeJulian There's a bunch of caching that gluster will try to do, both on the server and on the client. If you hit swap, things are going to slow down. You're not hitting swap, though, so I'm at a loss for what else it could be.
22:11 Arrakaij the server doesnt even has swap :)
22:11 Arrakaij and the client is going though nfs
22:12 JoeJulian Oh, well nfs is slower for large files. To get full throughput you want to use the fuse client or, better yet, libgfapi.
22:13 Arrakaij the performance is okay for 1gbit connection, 80-100mb/s
22:13 Arrakaij i try to switch to fuse tho, server just came back up
22:16 Arrakaij just started a copy process with fuse and tuned zfs, lets see how it works out
22:17 Arrakaij can let it run over night
22:19 Arrakaij tho fuse doesnt seem faster at all, its around 1/3 of the nfs performance
22:26 PaulCuzner joined #gluster
22:26 PaulCuzner left #gluster
22:26 shyam joined #gluster
22:40 Arrakaij thanks for the hints, i'll let it run over night and report back
22:42 cholcombe can the gluster fuse mount now do deep mounts?
23:04 pjrebollo joined #gluster
23:08 cholcombe is there a PPA for heketi?
23:54 pjrebollo joined #gluster
23:57 Jacob843 joined #gluster

| Channels | #gluster index | Today | | Search | Google Search | Plain-Text | summary