Perl 6 - the future is here, just unevenly distributed

IRC log for #gluster, 2016-01-30

| Channels | #gluster index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:01 haomaiwa_ joined #gluster
00:02 hagarth CyrilPeponnet: would the qcow image be updated on the master before the symlink gets synced to the slave?
00:02 CyrilPeponnet everything is done to the master
00:03 CyrilPeponnet update the qcow then update the symlink
00:04 CyrilPeponnet actually it's the same thing as copy file1 then file2, if I set sync-jobs to 1 is the file1 be geo-replicated before file2 get transfered
00:04 CyrilPeponnet (I guess this is the same thing. I'm testing this right now)
00:06 JoeJulian hagarth: The gv-cinder-replicate-4:self-heal inode lock has an owner=5c62767b9c7f0000. How can I convert that number into a client?
00:06 CyrilPeponnet but yes @hagarth that's the question
00:18 CyrilPeponnet well it doesnt work...
00:18 CyrilPeponnet the latest symlink is updated but the qcow2 is still transfering
00:20 CyrilPeponnet it sucks :/
00:22 baojg joined #gluster
00:24 JoeJulian bummer
00:30 hagarth JoeJulian: are you able to observe transport or connection_id details with owner?
00:30 CyrilPeponnet I guess I will have to tweak gsyncd
00:33 JoeJulian transport's also a number. No connection_ids.
00:35 JoeJulian I just need these heals to finish (#$%%#! 20 Tb images! I told them!)
00:36 hagarth JoeJulian: are you seeing these details from statedump?
00:36 JoeJulian yes
00:37 JoeJulian That's the only place I know to find those details. If there's something better, I'm all ears.
00:37 hagarth JoeJulian: I would look at statedump too
00:38 JoeJulian Plus, the state dump tells me how far along the heal is. Useful when the admins want to know how much longer.
00:38 CyrilPeponnet well actually the geo-rep didn't transfert the full file... it stopped a 3GB...
00:40 CyrilPeponnet wow
00:40 CyrilPeponnet in fact it stats transfered file1 then pause it and transfert file2 then resume on file 1
00:40 CyrilPeponnet How fun it is...
00:42 hagarth CyrilPeponnet: drop a note on -users.. Aravinda or Venky might have some useful input for this
00:42 CyrilPeponnet I will
00:42 CyrilPeponnet thks
00:55 msciciel2 joined #gluster
00:55 luizcpg_ joined #gluster
00:56 CyrilPeponnet the funny this, if you set sync-jobs to 0 then it accept it but no thing is georeplicated anymore
00:58 edong23_ joined #gluster
00:58 chirino_ joined #gluster
00:59 samppah joined #gluster
00:59 yosafbridge` joined #gluster
00:59 Ramereth|home joined #gluster
00:59 julim_ joined #gluster
00:59 lezo_ joined #gluster
00:59 csim_ joined #gluster
00:59 frankS2_ joined #gluster
01:00 Chr1st1an_ joined #gluster
01:00 fyxim_ joined #gluster
01:01 EinstCrazy joined #gluster
01:01 haomaiwa_ joined #gluster
01:02 sghatty__ joined #gluster
01:02 Larsen__ joined #gluster
01:02 mpingu_ joined #gluster
01:03 semiosis_ joined #gluster
01:03 DJCl34n joined #gluster
01:03 ashka` joined #gluster
01:04 semiosis joined #gluster
01:05 stopbyte joined #gluster
01:05 ndevos joined #gluster
01:05 ndevos joined #gluster
01:05 Akee joined #gluster
01:09 dataio joined #gluster
01:09 ackjewt joined #gluster
01:11 DJClean joined #gluster
01:21 jmarley joined #gluster
01:23 ctria joined #gluster
01:32 harish joined #gluster
01:33 Lee1092 joined #gluster
01:39 vmallika joined #gluster
01:50 bennyturns joined #gluster
02:01 haomaiwa_ joined #gluster
02:07 nangthang joined #gluster
02:16 atinm joined #gluster
02:17 klaxa joined #gluster
02:21 vmallika joined #gluster
02:22 jeek joined #gluster
02:31 nangthang joined #gluster
02:40 dlambrig_ joined #gluster
02:48 sakshi joined #gluster
03:01 haomaiwa_ joined #gluster
03:10 jeek joined #gluster
03:36 shubhendu joined #gluster
04:01 haomaiwa_ joined #gluster
04:02 nishanth joined #gluster
04:06 anoopcs joined #gluster
04:10 farhoriz_ joined #gluster
04:17 atinm joined #gluster
04:19 shubhendu_ joined #gluster
04:20 calavera joined #gluster
04:21 tswartz i have an rsync process that is syncing to gfs every couple of hours, but it takes a while to complete even though not much has changed and I get some pretty high cpu and ram usage while the rsync happens
04:21 shubhendu_ joined #gluster
04:22 baojg joined #gluster
04:22 tswartz any ideas on what the issue could be?
04:57 MACscr|lappy_ joined #gluster
05:01 haomaiwang joined #gluster
05:05 sakshi joined #gluster
05:09 gem joined #gluster
05:23 vmallika joined #gluster
05:23 ppai joined #gluster
05:53 rafi joined #gluster
06:05 gem joined #gluster
06:12 calavera joined #gluster
06:16 haomaiwa_ joined #gluster
06:20 atinm joined #gluster
06:51 kovshenin joined #gluster
07:01 haomaiwa_ joined #gluster
07:12 nehar joined #gluster
07:15 swebb joined #gluster
07:39 sakshi joined #gluster
08:01 haomaiwa_ joined #gluster
08:04 luizcpg joined #gluster
08:21 skoduri joined #gluster
08:35 kovshenin joined #gluster
08:44 post-factum tswartz: how many files do you have?
08:54 kovshenin joined #gluster
09:01 17WABOSYC joined #gluster
09:03 kshlm joined #gluster
09:27 gem joined #gluster
09:35 rafi joined #gluster
10:00 DJClean joined #gluster
10:01 ashka joined #gluster
10:01 7JTAADNZJ joined #gluster
10:14 skoduri joined #gluster
10:33 gem joined #gluster
10:49 baojg joined #gluster
10:55 EinstCrazy joined #gluster
10:56 baojg joined #gluster
11:08 haomaiwa_ joined #gluster
11:09 kshlm joined #gluster
11:16 skoduri joined #gluster
11:27 kshlm joined #gluster
11:50 Pupeno joined #gluster
11:50 Pupeno joined #gluster
12:01 haomaiwa_ joined #gluster
12:04 jmarley joined #gluster
12:16 nehar joined #gluster
12:23 Pupeno joined #gluster
12:38 Bhaskarakiran joined #gluster
12:38 Bhaskarakiran joined #gluster
13:01 haomaiwa_ joined #gluster
13:02 Pupeno joined #gluster
13:03 deniszh joined #gluster
13:10 jeek joined #gluster
13:33 tswartz post-factum, 1,818,759 in 51GB
14:01 haomaiwang joined #gluster
14:03 kovshenin joined #gluster
14:19 kanagaraj joined #gluster
14:21 kovshenin joined #gluster
14:44 post-factum tswartz: that's a lot. gluster is slow enough in traversing lots of files, especially if there are lots of them per 1 folder
14:44 tswartz i see
14:44 post-factum tswartz: what is the source for rsync?
14:45 post-factum ordinary FS?
14:45 tswartz yup
14:45 post-factum then i suggest you using lsyncd
14:45 tswartz probably not a bad idea, but really the rsync is temporary until i switch networks completely
14:46 post-factum it keeps tracking changes on FS with inotify, rsyncing only what actually was changed
14:46 tswartz yeah, i use lsyncd to keep some configs synced between machines
14:46 tswartz maybe this will encourage me to write a de-duping system for our stuff
14:46 post-factum also, there is clsync: https://github.com/xaionaro/clsync
14:46 glusterbot Title: xaionaro/clsync: file live sync daemon base... - GitHub (at github.com)
14:47 post-factum what is the data you store actually?
14:47 post-factum mail?
14:48 tswartz the vast majority is images. it's about 1,200 wordpress sites
14:48 tswartz crap tons are duplicates
14:48 post-factum are they arranged by subfolders?
14:48 tswartz yes
14:48 post-factum dedup is planned for 4.0, AFAIK
14:49 post-factum correct me if i'm wrong
14:49 post-factum you could store hashes somewhere in DB, and write new image if only no hash found in DB
14:49 tswartz so it says
14:50 post-factum or rename each file according to its hash and use no DB at all
14:50 tswartz so 1,200 wp sites go out each day and pick up an article from a remote server, the same article by the way
14:50 tswartz each time this article is pulled in, it brings in the featured image and stores it per blog
14:51 tswartz along with each size of that image that gets used in the theme
14:51 tswartz so it really is just a lot of the same files repeated, daily, for 4 years
14:51 post-factum sounds like huge waste of space :_
14:52 tswartz it truly is. and it really wasn't an issue not long ago, but this thing grows exponentially
14:54 tswartz good advise using lsyncd to do this for the time being though, hadn't really considered it
14:56 shaunm joined #gluster
15:01 7GHAB6I7G joined #gluster
15:07 wushudoin joined #gluster
15:08 chirino joined #gluster
15:12 Bhaskarakiran joined #gluster
15:15 chirino joined #gluster
15:15 skoduri joined #gluster
15:17 bennyturns joined #gluster
15:41 bluenemo joined #gluster
15:43 nishanth joined #gluster
15:44 EinstCrazy joined #gluster
15:57 wushudoin joined #gluster
16:01 haomaiwang joined #gluster
16:07 klaas_ joined #gluster
16:24 EinstCrazy joined #gluster
16:26 EinstCrazy joined #gluster
16:31 EinstCrazy joined #gluster
16:34 dlambrig_ joined #gluster
16:36 mowntan joined #gluster
16:39 caitnop joined #gluster
16:48 ctria joined #gluster
16:49 chirino joined #gluster
16:57 ahino joined #gluster
17:01 haomaiwang joined #gluster
17:15 tswartz joined #gluster
17:17 nehar joined #gluster
18:01 haomaiwa_ joined #gluster
18:42 plarsen joined #gluster
18:57 tom[] joined #gluster
19:01 haomaiwa_ joined #gluster
19:03 moss_ joined #gluster
19:08 calavera joined #gluster
19:14 dlambrig_ joined #gluster
19:17 sloop joined #gluster
19:42 ahino joined #gluster
19:46 tom[] joined #gluster
19:51 tswartz joined #gluster
19:52 dries joined #gluster
20:01 17WABOX4Z joined #gluster
20:09 jrm16020 joined #gluster
20:12 dries I didn't try the streaming. I hope they could capture all the videos?
20:12 dries sorry that was meant for #fosdem :-/
20:39 moss JoeJulian: Just got my Girl Scout cookies... yum :D
20:39 rafi joined #gluster
20:42 calavera joined #gluster
20:48 JoeJulian Nice! I haven't even gotten mine yet. :)
20:48 tom[] joined #gluster
21:01 haomaiwa_ joined #gluster
21:20 post-factum cookies... have to make some tea
21:33 ahino joined #gluster
21:33 baojg joined #gluster
21:36 dries the fosdem app also indicates it is cancelled..
21:44 chirino joined #gluster
21:45 mhulsman joined #gluster
21:48 mhulsman joined #gluster
21:50 Wizek joined #gluster
22:01 haomaiwa_ joined #gluster
22:37 baojg joined #gluster
22:50 mowntan joined #gluster
22:59 calavera joined #gluster
23:01 haomaiwang joined #gluster
23:07 Ramereth joined #gluster
23:13 sghatty_ joined #gluster
23:17 harish joined #gluster
23:17 Wizek joined #gluster
23:39 baojg joined #gluster
23:44 dlambrig_ joined #gluster
23:57 luizcpg joined #gluster
23:58 EinstCrazy joined #gluster

| Channels | #gluster index | Today | | Search | Google Search | Plain-Text | summary