Perl 6 - the future is here, just unevenly distributed

IRC log for #gluster, 2015-05-26

| Channels | #gluster index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:10 akay1 joined #gluster
00:12 aaronott joined #gluster
00:19 plarsen joined #gluster
01:09 JoeJulian minfig404: ,,(path or prefix)
01:09 glusterbot minfig404: http://joejulian.name/blog/glusterfs-path-or-a-prefix-of-it-is-already-part-of-a-volume/
01:09 JoeJulian ... or... I think you're now supposed to be able to add "force" to the create in 3.7.
01:15 harish joined #gluster
01:21 julim joined #gluster
01:39 gildub joined #gluster
01:39 gildub joined #gluster
01:48 ilbot3 joined #gluster
01:48 Topic for #gluster is now Gluster Community - http://gluster.org | Patches - http://review.gluster.org/ | Developers go to #gluster-dev | Channel Logs - https://botbot.me/freenode/gluster/ & http://irclog.perlgeek.de/gluster/
02:11 nangthang joined #gluster
02:55 rjoseph joined #gluster
03:22 TheSeven joined #gluster
03:23 overclk joined #gluster
03:23 sripathi joined #gluster
03:24 autoditac_ joined #gluster
03:32 minfig404 JoeJulian: Thanks for the pointer. There's something else goofy going on here, but I can't put my finger on it. It looks to me like the two machines can't coordinate a lock, and that causes the transction to never start, or even properly logged as a failure.
03:43 kanagaraj joined #gluster
03:44 sakshi joined #gluster
03:54 itisravi joined #gluster
03:54 atinmu joined #gluster
04:03 shubhendu joined #gluster
04:05 RameshN joined #gluster
04:07 bharata-rao joined #gluster
04:10 yazhini joined #gluster
04:17 nangthang joined #gluster
04:18 glusterbot News from newglusterbugs: [Bug 1224857] DHT - rebalance - when any brick/sub-vol is down and rebalance is not performing any action(fixing lay-out or migrating data) it should not say 'Starting rebalance on volume <vol-name> has been successful' . <https://bugzilla.redhat.com/show_bug.cgi?id=1224857>
04:28 anrao joined #gluster
04:35 hchiramm joined #gluster
04:40 deepakcs joined #gluster
04:47 shubhendu joined #gluster
04:48 gem joined #gluster
04:49 ndarshan joined #gluster
04:49 ramteid joined #gluster
04:49 kshlm joined #gluster
04:53 schandra joined #gluster
04:55 soumya joined #gluster
05:03 meghanam joined #gluster
05:07 rjoseph joined #gluster
05:10 ashiq joined #gluster
05:11 Manikandan joined #gluster
05:14 pppp joined #gluster
05:17 vikumar joined #gluster
05:17 kdhananjay joined #gluster
05:19 Apeksha joined #gluster
05:19 hgowtham joined #gluster
05:20 spandit joined #gluster
05:21 arao joined #gluster
05:22 rafi joined #gluster
05:22 Philambdo joined #gluster
05:23 smohan joined #gluster
05:29 karnan joined #gluster
05:30 anil joined #gluster
05:32 ashiq joined #gluster
05:33 Manikandan joined #gluster
05:47 Anjana joined #gluster
05:48 jayunit1000 joined #gluster
05:50 Bhaskarakiran joined #gluster
05:50 hagarth joined #gluster
05:52 R0ok_ joined #gluster
06:06 soumya joined #gluster
06:13 rgustafs joined #gluster
06:16 mike25de joined #gluster
06:19 kdhananjay joined #gluster
06:23 raghu joined #gluster
06:23 shubhendu joined #gluster
06:26 poornimag joined #gluster
06:29 jtux joined #gluster
06:30 atalur joined #gluster
06:35 mkzero joined #gluster
06:39 Anjana joined #gluster
06:39 lexi2 joined #gluster
06:49 DV_ joined #gluster
06:50 nangthang joined #gluster
06:55 wowaslansin joined #gluster
07:03 atinmu joined #gluster
07:04 deniszh joined #gluster
07:08 elico joined #gluster
07:09 jtux joined #gluster
07:13 ppai joined #gluster
07:13 LebedevRI joined #gluster
07:15 Jitendra joined #gluster
07:16 arao joined #gluster
07:18 soumya joined #gluster
07:18 liquidat joined #gluster
07:19 glusterbot News from newglusterbugs: [Bug 1223213] gluster volume status fails with locking failed error message <https://bugzilla.redhat.com/show_bug.cgi?id=1223213>
07:19 glusterbot News from newglusterbugs: [Bug 1223215] gluster volume status fails with locking failed error message <https://bugzilla.redhat.com/show_bug.cgi?id=1223215>
07:21 dusmant joined #gluster
07:24 atinmu joined #gluster
07:32 DV_ joined #gluster
07:35 anrao joined #gluster
07:36 davidself joined #gluster
07:41 dusmant joined #gluster
07:44 fsimonce joined #gluster
07:50 soumya joined #gluster
07:51 arao joined #gluster
07:52 [Enrico] joined #gluster
08:00 dusmant joined #gluster
08:00 arao joined #gluster
08:01 saurabh_ joined #gluster
08:09 nsoffer joined #gluster
08:12 atinmu joined #gluster
08:17 ctria joined #gluster
08:22 ekman joined #gluster
08:25 gildub joined #gluster
08:31 kshlm joined #gluster
08:33 TvL2386 joined #gluster
08:40 harish joined #gluster
08:47 RameshN joined #gluster
08:54 saurabh_ joined #gluster
09:10 jiffin joined #gluster
09:25 Slashman joined #gluster
09:26 saurabh_ joined #gluster
09:27 anrao joined #gluster
09:42 badone_ joined #gluster
09:46 s19n joined #gluster
09:54 atalur joined #gluster
10:00 dusmant joined #gluster
10:06 haomaiwa_ joined #gluster
10:36 atinmu REMINDER: Gluster Community Bug Triage meeting today at 12:00 UTC at #gluster-meeting
10:39 ira joined #gluster
10:41 vishvendra joined #gluster
10:43 Leildin joined #gluster
10:48 lalatenduM hchiramm_: searched body of gluster-users (using thunderbird) , for centos got 1188 entries and for fedora 745. Searched subject lines in gluster-users , for centos got 277 , for fedora 40
10:49 soumya joined #gluster
10:49 lalatenduM hchiramm_: so does it tell us we have more centos users than fedora users :)
10:50 lalatenduM ndevos: ^^
10:52 ira lalatenduM: Or that the CentOS ones had more reason to post ;)
10:53 lalatenduM ira: haha, but gluster issues should not be linked with platform, is it?
10:54 ira lalatenduM: http://www.tylervigen.com/spurious-correlations
10:54 ira If you've never seen it... it makes my point quite clearly.
10:54 lalatenduM ira: haha I have seen it before :)
10:54 ira lalatenduM: Actually, they may.  Depends.  If we left a bad version in CentOS. :)
10:55 lalatenduM ira: yeah, there are verious variables
10:56 ira lala: Indeed.  Correlation just implies correlation.  :)   You need to find out why the variables are acting together.  (If they are and it isn't a strange concidence ;) )
10:57 lalatenduM ira: agree , it is not that simple
10:57 ira Ok.. </rant> on my side ;)
10:57 lalatenduM ira: however it shows that we have active users using CentOS+gluster
10:57 lalatenduM and we should care about it
10:57 lalatenduM :)
10:58 ira lalatenduM: That I'd agree with.
10:58 ira Or at least one really loud one ;)
10:58 ira </troll>
10:58 lalatenduM ira: :)
11:02 jcastill1 joined #gluster
11:07 jcastillo joined #gluster
11:07 Manikandan joined #gluster
11:07 Manikandan_ joined #gluster
11:13 kshlm joined #gluster
11:16 [Enrico] joined #gluster
11:19 glusterbot News from newglusterbugs: [Bug 1217722] Tracker bug for Logging framework expansion. <https://bugzilla.redhat.com/show_bug.cgi?id=1217722>
11:23 aravindavk joined #gluster
11:24 dusmant joined #gluster
11:25 Anjana joined #gluster
11:28 atalur joined #gluster
11:30 autoditac joined #gluster
11:32 surabhi joined #gluster
11:32 maveric_amitc_ joined #gluster
11:33 gem joined #gluster
11:47 ppai joined #gluster
11:48 ndarshan joined #gluster
11:53 gem joined #gluster
11:54 rgustafs joined #gluster
12:00 soumya joined #gluster
12:00 meghanam joined #gluster
12:01 rafi1 joined #gluster
12:03 surabhi_ joined #gluster
12:05 rjoseph joined #gluster
12:06 uxbod joined #gluster
12:06 uxbod Good day all
12:07 uxbod Have created a geo-replication link but wish to ask how one can synchronise all existing files please ?
12:07 uxbod if I create a new file in syncs fine, but all the existing ones do not
12:14 vishvendra joined #gluster
12:14 poornimag joined #gluster
12:20 glusterbot News from newglusterbugs: [Bug 1218479] Gluster NFS Mount Permission Denied Error (Occur Intermittent) <https://bugzilla.redhat.com/show_bug.cgi?id=1218479>
12:20 glusterbot News from newglusterbugs: [Bug 1225018] Scripts/Binaries are not installed with +x bit <https://bugzilla.redhat.com/show_bug.cgi?id=1225018>
12:20 itisravi joined #gluster
12:20 rafi joined #gluster
12:20 tom[] joined #gluster
12:21 pppp joined #gluster
12:24 kanagaraj joined #gluster
12:26 jayunit1000 joined #gluster
12:27 kanagaraj_ joined #gluster
12:30 B21956 joined #gluster
12:34 chirino joined #gluster
12:35 hagarth joined #gluster
12:39 rjoseph joined #gluster
12:47 xiu joined #gluster
12:50 delhage joined #gluster
12:52 wkf joined #gluster
12:53 Slashman hello, I'm using glusterfs 3.6.3, is it possible to set posix ACL on a glusterfs fuse mount point? seems like the acl mount option is not supported for fuse.glusterfs :/
13:00 glusterbot News from resolvedglusterbugs: [Bug 885424] File operations occur as root regardless of original user on 32-bit nfs client <https://bugzilla.redhat.com/show_bug.cgi?id=885424>
13:00 glusterbot News from resolvedglusterbugs: [Bug 974886] timestamps of brick1 and brick2 is not the same. <https://bugzilla.redhat.com/show_bug.cgi?id=974886>
13:00 glusterbot News from resolvedglusterbugs: [Bug 990220] Group permission with high GID Number (200090480) is not being honored by Gluster <https://bugzilla.redhat.com/show_bug.cgi?id=990220>
13:00 glusterbot News from resolvedglusterbugs: [Bug 1021998] nfs mount via symbolic link does not work <https://bugzilla.redhat.com/show_bug.cgi?id=1021998>
13:00 glusterbot News from resolvedglusterbugs: [Bug 1077406] Striped volume does not work with VMware esxi v4.1, 5.1 or 5.5 <https://bugzilla.redhat.com/show_bug.cgi?id=1077406>
13:00 glusterbot News from resolvedglusterbugs: [Bug 1109613] gluster volume create fails with ambiguous error <https://bugzilla.redhat.com/show_bug.cgi?id=1109613>
13:00 glusterbot News from resolvedglusterbugs: [Bug 1115850] libgfapi-python fails on discard() and fallocate() due to undefined symbol <https://bugzilla.redhat.com/show_bug.cgi?id=1115850>
13:00 glusterbot News from resolvedglusterbugs: [Bug 1175758] [USS] : Rebalance process tries to connect to snapd and in case when snapd crashes it might affect rebalance process <https://bugzilla.redhat.com/show_bug.cgi?id=1175758>
13:00 glusterbot News from resolvedglusterbugs: [Bug 1065616] Host name is not updating <https://bugzilla.redhat.com/show_bug.cgi?id=1065616>
13:00 glusterbot News from resolvedglusterbugs: [Bug 1115199] Unable to get lock for uuid,  Cluster lock not held <https://bugzilla.redhat.com/show_bug.cgi?id=1115199>
13:00 glusterbot News from resolvedglusterbugs: [Bug 960818] Installing glusterfs rpms on a pristine f19 system throws "error reading information on service glusterfsd". <https://bugzilla.redhat.com/show_bug.cgi?id=960818>
13:00 glusterbot News from resolvedglusterbugs: [Bug 908466] Hadoop benchmark TestDFSIO fails with “java.io.IOException: /mnt/glusterfs/b*/TestDFSIO/io_control/in_file_test_io_2 does not exist.” when the file exists <https://bugzilla.redhat.com/show_bug.cgi?id=908466>
13:00 glusterbot News from resolvedglusterbugs: [Bug 1009134] sequential read performance  not optimized for libgfapi <https://bugzilla.redhat.com/show_bug.cgi?id=1009134>
13:00 glusterbot News from resolvedglusterbugs: [Bug 1083324] DHT rsync copies directories without any modification from source to destination <https://bugzilla.redhat.com/show_bug.cgi?id=1083324>
13:00 glusterbot News from resolvedglusterbugs: [Bug 832609] Glusterfsd hangs if brick filesystem becomes unresponsive, causing all clients to lock up <https://bugzilla.redhat.com/show_bug.cgi?id=832609>
13:00 glusterbot News from resolvedglusterbugs: [Bug 802036] readlink fails on CIFS mount <https://bugzilla.redhat.com/show_bug.cgi?id=802036>
13:00 glusterbot News from resolvedglusterbugs: [Bug 875412] cannot replace brick in distributed replicated volume <https://bugzilla.redhat.com/show_bug.cgi?id=875412>
13:00 glusterbot News from resolvedglusterbugs: [Bug 916934] Rebalance failures/Very slow <https://bugzilla.redhat.com/show_bug.cgi?id=916934>
13:00 glusterbot News from resolvedglusterbugs: [Bug 765399] 1 gluster box locks up the filesystem due to a kernel errors <https://bugzilla.redhat.com/show_bug.cgi?id=765399>
13:00 glusterbot News from resolvedglusterbugs: [Bug 844187] Change log rotate command to maintain consistency with other CLI <https://bugzilla.redhat.com/show_bug.cgi?id=844187>
13:00 glusterbot News from resolvedglusterbugs: [Bug 854156] Support for hadoop distros <https://bugzilla.redhat.com/show_bug.cgi?id=854156>
13:00 glusterbot News from resolvedglusterbugs: [Bug 854541] Need better practise on (not) editing /usr/share/java/conf/core-site.xml <https://bugzilla.redhat.com/show_bug.cgi?id=854541>
13:01 plarsen joined #gluster
13:04 pppp joined #gluster
13:09 bturner_ joined #gluster
13:10 rjoseph Slashman: did you looked at http://www.gluster.org/community/documentation/index.php/Gluster_3.2:_Activating_POSIX_ACLs_Support_on_Client ?
13:12 julim joined #gluster
13:13 aaronott1 joined #gluster
13:13 dblack joined #gluster
13:15 raatti left #gluster
13:19 dusmant joined #gluster
13:24 Anjana joined #gluster
13:26 aravindavk joined #gluster
13:28 ira joined #gluster
13:29 Bardack mmm i m investigating some glusterFS configuration made by my previous colleagues and I ve an understanding issue
13:29 Bardack we are using a backup application (netbackup), and we created 1 gluster volume of 40TB made of 20x2TB bricks on a gluster server
13:30 Bardack we are starting 3 backup policy at the same time, every policy will be splitted into 5 jobs running at the same time
13:30 Bardack we were expecting to see something like 15 bricks used (io) during that backup
13:30 Bardack but only 3 are 100% used, and the others are sleeping
13:30 glusterbot Bardack: Please don't naked ping. http://blogs.gnome.org/markmc/2014/02/20/naked-pings/
13:31 dgandhi joined #gluster
13:31 squizzi joined #gluster
13:32 Bardack we were thinking : ohh ok 3 policies, from 3 differents clients, so spreading is only done on client side … but basically it's the backup host who does the real backup, so it's the only client …
13:32 Bardack is there a way to configure glusterFS saying: every job should be written on a separated disk if possible '
13:34 DV joined #gluster
13:39 nsoffer joined #gluster
13:41 hamiller joined #gluster
13:45 Bardack but well, it's a distributed volume, so it should be file based … full file or part of the file ? …
13:45 DV joined #gluster
13:47 gem joined #gluster
13:48 DV_ joined #gluster
13:50 bene2 joined #gluster
13:51 autoditac joined #gluster
13:52 atalur joined #gluster
13:53 TheSeven joined #gluster
13:56 Bardack filenames are indeed very very similar, but not exactly so …
13:57 warci joined #gluster
13:57 msvbhat Bardack: If you have a distribute only volume, the files reside in one of the bricks
13:57 msvbhat Bardack: Full file that is
13:58 msvbhat Bardack: And they get distributed based on the hash value which is computed
13:58 Bardack yeah, only 1 volume, made of 20 bricks
13:58 warci the question is: how does the elastic hashing algorithm work. Is it 'random' enough?
13:59 warci each of the 20 bricks has a range of hashes it accepts
13:59 Bardack so if i ve a MsSQL policy, that can run its backup with 8 streams, every stream will write a file (file.ext~<nb>of<total>.<date>), it should write to 8 different disks
13:59 warci will multiple files with very small variations in the name hash within the same range
14:01 msvbhat Yes. It's 'random' enough to say thet if write 100 files distributed across 20 bricks, each bricks receives ~5 files. Not that it's nearly 5. It may vary sometimes by small margin
14:01 Bardack mmm
14:01 warci even if the filenames only change one character?
14:01 msvbhat Bardack: You can not certainly say that each of them will go to different bricks
14:01 rafi joined #gluster
14:02 sripathi joined #gluster
14:02 Bardack currently i ve 1 policy running, 8 streams, filename is: /SPR-SQL13.MSSQL7.SPR-SQL13.db.DWH_PENSION.~.7.002of008.20150526151807..C, only 1 brick is used
14:02 Bardack so I've from 001of008 to 008of008
14:02 msvbhat warci: AFAIK yes, even if only small variations are there
14:02 Bardack but as soon as i launch another policy, hop it uses a new brick
14:03 msvbhat Bardack: If it's distributing to only one brick, that's not the expected behaviour
14:03 warci mmm could it be something got messed up with the layout? We started out with 10 disks, and added 10 more afterwards. I did run the 'fix-layout' command but.....
14:03 msvbhat You should've run the rebalance.
14:03 msvbhat Yes, fix-layout is part of rebalance
14:04 msvbhat Bardack: Just to make sure, your tool is writing to gluster mountpoint right?
14:04 warci in theory, rebalance shouldn't be needed, no? (although it's a good idea :) )
14:05 msvbhat warci: Well, fix-layout is needed for sure. But yeah full rebalance is not needed, but highly recommnded
14:06 Bardack backup server as a gluster share, and it's writting to it
14:07 msvbhat Bardack: Okay, then it should've distributed to all the bricks.
14:07 Bardack but it doesnt. so there is an issue :)
14:07 Bardack what can i do ? :
14:07 Bardack if anything can be usefull …
14:07 warci mmm could it be you need to stop/start the brick after a fix-layout?
14:08 warci anyway, there's data on the "new" bricks, so i guess gluster sees them
14:08 msvbhat Bardack: Well, Can you send a mail to gluster-users ML? Some DHT developer will have more insight
14:08 msvbhat warci: No need to stop/start the bricks
14:08 Bardack okido, thx
14:09 msvbhat warci: Yes, ola data will get rebalanced *only* if you full rebalance
14:09 jiffin joined #gluster
14:09 msvbhat Bardack: With some luck some DHT developer see you here in IRC, else send a mail.
14:10 * msvbhat will be afk for some time
14:10 Bardack thx for your help msvbhat
14:10 Bardack and warci , come back to your previous employeer :p NOW
14:10 warci lol. NEVER
14:14 deepakcs joined #gluster
14:17 lexi2 joined #gluster
14:18 archit_ joined #gluster
14:19 wushudoin joined #gluster
14:20 glusterbot News from newglusterbugs: [Bug 1225077] Fix regression test spurious failures <https://bugzilla.redhat.com/show_bug.cgi?id=1225077>
14:20 glusterbot News from newglusterbugs: [Bug 1225072] OpenSSL multi-threading changes break build in RHEL5 <https://bugzilla.redhat.com/show_bug.cgi?id=1225072>
14:20 glusterbot News from newglusterbugs: [Bug 1193474] Package libgfapi-python for its consumers <https://bugzilla.redhat.com/show_bug.cgi?id=1193474>
14:21 nage joined #gluster
14:31 dusmant joined #gluster
14:33 deepakcs JustinClift: ping
14:33 glusterbot deepakcs: Please don't naked ping. http://blogs.gnome.org/markmc/2014/02/20/naked-pings/
14:33 deepakcs JustinClift: there ? :)
14:40 kshlm joined #gluster
14:48 hchiramm joined #gluster
14:48 wushudoin joined #gluster
14:49 pdrakewe_ joined #gluster
14:50 glusterbot News from newglusterbugs: [Bug 1223839] /lib64/libglusterfs.so.0(+0x21725)[0x7f248655a725] ))))) 0-rpc_transport: invalid argument: this <https://bugzilla.redhat.com/show_bug.cgi?id=1223839>
14:51 kdhananjay joined #gluster
14:54 uxbod any thoughts on the initial geo-rep sync ? was expecting it would replicate all existing files and not just new ones
14:57 wushudoin| joined #gluster
14:58 JustinClift Hmmm
14:58 msvbhat uxbod: geo-rep should sync existing files as well.
15:03 pdrakeweb joined #gluster
15:03 wushudoin| joined #gluster
15:08 pdrakewe_ joined #gluster
15:10 uxbod nope, only new files were syncd
15:10 uxbod am using glusterfs-geo-replication-3.7.0-2.el6.x86_64
15:11 msvbhat Hmm... Initial syncing always happens through xsync (That's because changelogs are not generated before geo-rep start)
15:11 msvbhat And xsync can not handles deletes and/or renames
15:12 B21956 joined #gluster
15:12 msvbhat And once initial data is synced, they geo-rep moves to changelog based syncing
15:12 msvbhat If you have done any deletes and/or renames they will not be synced
15:13 uxbod wonder if its due to the volume being active, and in use, while the geo-rep is set up
15:14 arao joined #gluster
15:16 msvbhat uxbod: No, that shouldn't be.
15:16 msvbhat volume *should* be running when you start geo-rep
15:19 bturner_ joined #gluster
15:20 bturner_ joined #gluster
15:22 arao joined #gluster
15:24 jfdoucet joined #gluster
15:24 Innominate joined #gluster
15:26 jfdoucet I upgraded glusterfs to 3.7.0-2 yesterday and I had an issue where the peers would not connect. I had to downgrade it back to the previous version. Logs don't say much. Is this a known issue ? Anybody else had that ?
15:31 B21956 joined #gluster
15:33 stickyboy joined #gluster
15:37 haomaiwa_ joined #gluster
15:46 pdrakeweb joined #gluster
15:46 DV_ joined #gluster
15:48 pdrakeweb joined #gluster
15:51 pdrakewe_ joined #gluster
15:55 cholcombe joined #gluster
16:00 baoboa joined #gluster
16:00 squizzi joined #gluster
16:03 pppp joined #gluster
16:07 coredump joined #gluster
16:24 kkeithley @ppa
16:24 glusterbot kkeithley: The official glusterfs packages for Ubuntu are available here: 3.4: http://goo.gl/M9CXF8 3.5: http://goo.gl/6HBwKh 3.6: http://goo.gl/XyYImN -- See more PPAs for QEMU with GlusterFS support, and GlusterFS QA releases at https://launchpad.net/~gluster -- contact semiosis with feedback
16:24 pdrakeweb joined #gluster
16:25 soumya joined #gluster
16:29 Trefex joined #gluster
16:29 nsoffer joined #gluster
16:29 Trefex hey all. i have a distributed setup with 2 nodes and 3 bricks each. i have 1 additional node which acts as export for NFS and SMB to outside world
16:30 Trefex is it possible to query the status of the gluster volume from that controlling node or only from the nodes actually in the gluster pool ?
16:31 JoeJulian You can run commands on a remote host using the --remote-host switch
16:32 Trefex JoeJulian: so the commands have to be executed on the gluster nodes?
16:32 Trefex JoeJulian: there's no such thing as a client?
16:32 Trefex it might seem obvious, but just starting out here...
16:33 JoeJulian Well, every endpoint that uses or hosts gluster is a "node". The gluster cli tool is used to interface with glusterd, the management daemon, which will only be running on the servers that are part of the trusted pool.
16:33 wushudoin| left #gluster
16:34 Trefex ok, so the controller in principle only mounts the gluster volume, and as such cannot interact with the gluster itself
16:34 Trefex makes sense come to think of it
16:35 JoeJulian Yeah, the client is only the filesystem client.
16:36 Trefex JoeJulian: thanks for that. perhaps you can help with another issue
16:36 Trefex on CentOS, gluster is starting nfs which needs rpcbind to start correctly
16:36 Trefex however i'm not able to start rpcbind automatically, which means NFS never starts
16:36 Trefex [root@stor106 ~]# systemctl is-enabled rpcbind
16:36 Trefex static
16:36 Trefex even if i do a systemctl enable rpcbind
16:39 JoeJulian does /var/run/rpcbind.lock exist?
16:39 haomaiwa_ joined #gluster
16:40 Trefex JoeJulian: it does, but i started the service manually
16:42 JoeJulian I had an issue after an upgrade where that lock file had the wrong selinux context.
16:43 Trefex we disabled SElinux
16:43 Trefex on these machines
16:43 ndevos Trefex: maybe the workaroung is in the last line of the 1st comment in bug 1181779 ?
16:43 glusterbot Bug https://bugzilla.redhat.com:443/show_bug.cgi?id=1181779 unspecified, unspecified, rc, steved, ON_QA , rpcbind prevents Gluster/NFS from registering itself after a restart/reboot
16:43 ndevos *workaround even
16:44 Trefex -r--r--r--   1 root    root       0 May 26 18:37 rpcbind.lock
16:44 glusterbot Trefex: -r--r--r's karma is now -1
16:44 ndevos poor -r--r--r
16:44 glusterbot ndevos: -r--r's karma is now -1
16:44 Trefex whut haha glusterbot
16:44 * ndevos blinks
16:44 Trefex JoeJulian: is this correct?
16:45 JoeJulian It is.
16:46 JoeJulian No clue why it's not starting. The status output after a boot where it fails might be useful.
16:46 Trefex from which logs?
16:47 JoeJulian journalctl -u rpcbind
16:48 JoeJulian and I assume you know not to ,,(paste) in chat
16:48 glusterbot For RPM based distros you can yum install fpaste, for debian, ubuntu, and arch it's pastebinit. Then you can easily pipe command output to [f] paste [binit] and it'll give you a URL.
16:48 Trefex sure ;)
16:50 Trefex ok i shall reboot
16:50 ndevos Trefex: also check with 'rpcinfo -p' if the nfs program is registered already, before gluster/nfs is running (or after it failed)
16:50 ndevos Trefex: if that is the case, you really are hitting that ^^ bug
16:51 Trefex rpc info http://paste.fedoraproject.org/225685/65906914
16:51 Trefex the journalctl is not very helpful http://paste.fedoraproject.org/225686/43265910
16:52 JoeJulian systemctl status rpcinfo
16:53 Trefex the machine is rebooting
16:53 Trefex gonna take 5 mins, is 240 TB
16:53 ndevos Trefex: that rpcinfo output shows that there is an nfs server registered at rpcbind, only one can be registered at the time - I bet you for a cookie that you hit the above bug
16:53 JoeJulian 5 mins? must be ext4.
16:53 Trefex ZFS
16:53 Trefex but the booting sequence does all kinds of HW checks
16:54 Trefex that's pre-OS boot
16:54 JoeJulian That would drive me insane really quickly.
16:54 Trefex memcheck, disk probe and what not
16:54 Trefex believe me, i'm close :D
16:55 ndevos I had some IBM systems a couple of years age, they were configured with UEFI boot, and it took 5-10 minutes to get into grub, you really think twice before rebooting
16:55 Trefex we do haha
16:56 Trefex on one machine it does 4 disk checks on all disks, for 100 disks this can take easily 2-3 mins
16:56 Trefex just for the disks, nevermind all the other stuff,
16:57 ndevos my servers were only 2U, and had just a few disks, I never found out where it spent its time during boot
16:58 Trefex JoeJulian: http://paste.fedoraproject.org/225694/14326594
16:58 Trefex straight from reboot
16:58 Trefex ndevos: maybe some memcheck?
16:58 JoeJulian Well... not found indicates that systemd cannot find an rpcinfo.service file
16:59 ndevos Trefex: maybe, but it really was not obvious while it was booting
16:59 kshlm Trefex, can you try activating the rpcbind.socket service
16:59 ndevos oh, and the systemd service is called "rpcbind", not "rpcinfo"
16:59 JoeJulian and it shouldn't becuase it's rpcbind
16:59 kshlm s/service/unit/
16:59 glusterbot What kshlm meant to say was: An error has occurred and has been logged. Please contact this bot's administrator for more information.
16:59 Trefex now it seems "running"
16:59 kshlm JoeJulian, ^
17:00 JoeJulian yeah... stupid upgrade broke that and I haven't had the time to figure out why.
17:00 ndevos the systemd unit for the rpcbind service is called rpcbind.service and has a unit called rpcbind.socket too
17:00 kshlm Trefex, why seems?
17:01 Trefex no clue
17:01 Trefex just rebooted and now it's running,
17:01 kshlm rpcbind can apparently only be activated via the socket file.
17:01 ndevos Trefex: still no nfs server in 'gluster volume status $VOLUME'? but "rpcinfo -p" lists "nfs" again, thats the bug!
17:04 bene2 joined #gluster
17:05 Trefex no now it's all fine
17:05 Trefex but i guess i hit the bug previously
17:05 Trefex thanks for the help, gotta rush, be back tomorrow
17:05 ndevos cya Trefex
17:11 CyrilPeponnet Hey Gluster guys :) Long time no see !
17:11 JoeJulian Greetings and salutations.
17:12 CyrilPeponnet So my geo-rep is now in changeling mode Yeah! \o/
17:12 CyrilPeponnet but... because there is always a but,
17:12 CyrilPeponnet some files has been changed on the slaves, and the gfid changed, so master changes are no longer propagated (except rename) to the slave
17:13 JoeJulian I hope you're filing bugs for all these steps you've had to do manually. This shouldn't be this complicated.
17:13 CyrilPeponnet I really must to pass the slave as RO volume for that I think
17:13 CyrilPeponnet Sure I do :)
17:14 CyrilPeponnet so the question is... how to sync the gfid from master to slave in the case files have been changed on the slave ?
17:14 CyrilPeponnet (I check some of them and gfid is not the same anymore)
17:14 hagarth CyrilPeponnet: do you have a list of changed files?
17:14 hagarth CyrilPeponnet: how did the files change on the slave?
17:14 CyrilPeponnet Nope... I can try a rsync dry run but I have like 5TB of data
17:15 CyrilPeponnet @hagarth by mistake... some users start to change them
17:15 CyrilPeponnet I already skinned those users, and they are rotten in hell right now.. but I need to fix my vol !
17:16 ekuric joined #gluster
17:16 hagarth CyrilPeponnet: hmm, the first step is to identify such files and set their gfids manually to correspond with the ones on master
17:17 CyrilPeponnet ok a rsync dry run should do the trick, then rsync the content and then sync the gfid ?
17:17 hagarth are you using 3.5.x or 3.7.0? I forget your version
17:17 CyrilPeponnet 3.5.2
17:18 dgbaley joined #gluster
17:18 Rapture joined #gluster
17:18 hagarth right, the rsync based approach should work.
17:19 dgbaley Hey. New to Gluster. Does the order of bricks within a replica set matter?
17:19 CyrilPeponnet How can I then fix the gfid ? using some getfattr and setfattr to /mnt/vol_mountend/file ?
17:20 JoeJulian ~brick order | dgbaley
17:20 glusterbot dgbaley: Replicas are defined in the order bricks are listed in the volume create command. So gluster volume create myvol replica 2 server1:/data/brick1 server2:/data/brick1 server3:/data/brick1 server4:/data/brick1 will replicate between server1 and server2 and replicate between server3 and server4.
17:22 hagarth CyrilPeponnet: yes, once you have the list perform getfattr for trusted.gfid from the bricks directly and set those gfids on the slave bricks directly. getfattr/setfattr on the mount point would not work for gfids.
17:22 dgbaley JoeJulian: That I get, but does being the 1st brick within a set have any particular performance penalty/boost? server1 has multiple bricks. an initial listing of bricks would lead to server1's bricks always being first within a given set, should I try to avoid that?
17:23 CyrilPeponnet @hagarth and I guess I have to do that on both replicat. Maybe I should stop the volume before ?
17:24 dgbaley Any tips for my setup would be nice too. I have 3 servers with 5 disks each, they serve OpenStack images/disks. I was planning on doing a triple-replicated/distributed volume.
17:25 JoeJulian That's not an uncommon use.
17:30 jiffin joined #gluster
17:32 hagarth CyrilPeponnet: yes. if there's ongoing I/O, better to stop the volume and then proceed.
17:33 jmarley joined #gluster
17:33 jmarley joined #gluster
17:39 Gill joined #gluster
17:42 gem joined #gluster
17:42 pdrakewe_ joined #gluster
17:43 rafi joined #gluster
17:44 jiffin ping ndevos
17:44 Prilly joined #gluster
17:44 Trefex joined #gluster
17:45 Trefex joined #gluster
17:50 CyrilPeponnet @hagarth in fact with rsync I can see that lot of files are missing... the hybrid crawl may have fail
17:51 CyrilPeponnet how to restore that ? copy the missing file to the bricks and set proper gfid ? (and then restart the volume)
17:52 CyrilPeponnet of copy to the vol stop the vol, fix the gfid and restart the vol ?
17:52 haomaiwa_ joined #gluster
17:53 hagarth CyrilPeponnet: touch on the missed files might trigger a re-sync through geo-replication. I would defer this question to Aravinda/Kotresh for a more accurate answer & procedure. Maybe sending a note on gluster-users is what we need to do here.
17:55 CyrilPeponnet and one more thing if I pass the slave vol in RO is it file with geo-rep process ?
17:56 CyrilPeponnet I mean to avoid further issue I really want to make it RO but is it geo-rep safe ?
17:56 Prilly joined #gluster
17:57 arao joined #gluster
17:57 hagarth CyrilPeponnet: again a good question for them .. I think geo-rep syncing might also fail if read-only is enabled for slave vol.
17:57 hagarth clients could be mounted with ro if that is an option
17:58 hagarth so that geo-rep gets to write/update and other clients get to do only ro
17:59 rafi joined #gluster
18:00 bturner_ joined #gluster
18:00 Gill joined #gluster
18:01 kkeithley joined #gluster
18:01 Trefex joined #gluster
18:02 swankier joined #gluster
18:03 swankier Hi there.  in a three node setup, I have one node that is currently 'read-only'.  unmounting, restarting gluster, and remounting doesn't help.
18:03 swankier I don't understand why it is in this state nor how to resolve it
18:03 swankier could someone please offer advice?
18:11 wkf joined #gluster
18:12 Prilly joined #gluster
18:30 shaunm_ joined #gluster
18:31 glusterbot News from resolvedglusterbugs: [Bug 996290] mapred system directory is not created when using gluster hadoop shim <https://bugzilla.redhat.com/show_bug.cgi?id=996290>
18:37 deniszh joined #gluster
18:45 rotbeard joined #gluster
18:49 rafi joined #gluster
18:55 prilly_ joined #gluster
19:00 wkf joined #gluster
19:01 cholcombe gluster: any issues with running a gluster server as an unpriv'd user?
19:01 cholcombe just trying to make a little testing setup to throw away
19:02 anil joined #gluster
19:03 ShaunR joined #gluster
19:05 kkeithley cholcombe: yes, it won't work
19:05 cholcombe :(
19:05 cholcombe what's the issue?
19:05 nsoffer joined #gluster
19:09 nsoffer joined #gluster
19:11 prilly_ joined #gluster
19:13 dgbaley Is it common/reasonable to have multiple bricks on a filesystem? Or can 1 brick be assigned to multiple volumes?
19:15 JoeJulian It isn't uncommon to have multiple bricks on a filesystem. I preferred to have separate filesystems per brick using lvm to carve up storage devices. As a brick exceeds a threshold, I would add extents and expand.
19:19 dgbaley This is what I was asking before: http://pastie.org/10208651. Each of the 5 replica sets has the host order 2,4,8. Should I mix that up?
19:20 JoeJulian No, that's fine.
19:21 dgbaley Cool. I like simple =). I'm migrating away from ceph which liked to hold my data hostage regularly and has been difficult to debug
19:26 prilly_ joined #gluster
19:46 marcoceppi joined #gluster
19:46 marcoceppi joined #gluster
19:53 pousley joined #gluster
19:54 pousley left #gluster
19:59 pousley joined #gluster
20:03 dgbaley JoeJulian: how does the client choose which server to do a read from? Is it per read? Cycles over time? Last known good? Per file?...
20:06 acampbell joined #gluster
20:15 wkf joined #gluster
20:30 badone_ joined #gluster
20:40 prilly_ joined #gluster
20:52 Prilly joined #gluster
21:17 papamoose joined #gluster
21:27 ShaunR why when i do a systemctl stop glusterfsd and systemctl stop glusterd do i still see a running glusterfs process
21:27 ShaunR is this a bug in the centos7 rpms?
21:31 stickyboy joined #gluster
21:35 cholcombe gluster: if i put multiple bricks on the same server and add the force flag it seems to ignore that and still tell me this setup isn't optimal
21:35 cholcombe is there anyway i can get it to stop asking that question?
21:42 wushudoin joined #gluster
21:51 Prilly joined #gluster
21:55 JoeJulian ShaunR: No, not really, it's just that glusterfsd.service doesn't actually own any processes. Stopping glusterd should only stop glusterd since stopping bricks can be harmful (update all your server rpms at the same time and embrace split-brain).
21:56 JoeJulian glusterfs services for nfs and shd could, arguably, be stopped safely. There's just no singly valid point-of-view on that.
21:56 JoeJulian cholcombe: Always works for me. What version?
21:57 cholcombe 3.6.x
21:57 cholcombe @JoeJulian: i got it to work by just using 1 brick.  that's fine for testing
21:57 cholcombe it's just kind of a pain to pipe stdin and send things to the command
22:04 JoeJulian # gluster --mode=script volume create test2 replica 2 questor:/tmp/test2vol/brick{1,2} force
22:04 JoeJulian volume create: test2: success: please start the volume to access data
22:04 JoeJulian ... I knew I remembered something for that.
22:11 ShaunR well, NFS is def faster
22:12 ShaunR handling a ton of small files alot better
22:16 lexi2 joined #gluster
22:18 _Bryan_ joined #gluster
22:23 rotbeard joined #gluster
22:33 cholcombe @JoeJulian: so the trick is saying --mode=script?  What does that change?
22:33 jmarley joined #gluster
22:34 cholcombe @JoeJulian: ok i looked it up.  yes that's exactly what i'm looking for!
22:34 plarsen joined #gluster
22:38 JoeJulian ShaunR: Or /not/ handling them... that's why it's faster. :p
22:38 ShaunR JoeJulian: i'm seeing a performance improvemnt with nfs
22:38 JoeJulian Because, obviously, if you don't do something, it takes less time than if you do.
22:39 JoeJulian But as long as you understand the deficiency and it works for your use case, more power to you.
22:39 rotbeard joined #gluster
22:40 ShaunR deficiency? whats the downside?
22:43 ShaunR JoeJulian: i'm just trying to create a decient performing lamp cluster running a joomla site
22:43 ShaunR but the performance is horrible.,
22:43 JoeJulian You're caching things locally on the client. If those things change on the server, your client won't know about it.
22:44 JoeJulian joomla should run fine, as long as you're not reloading your scripts all the time.
22:44 JoeJulian @php
22:44 glusterbot JoeJulian: (#1) php calls the stat() system call for every include. This triggers a self-heal check which makes most php software slow as they include hundreds of small files. See http://joejulian.name/blog/optimizing-web-performance-with-glusterfs/ for details., or (#2) It could also be worth mounting fuse with glusterfs --attribute-timeout=HIGH --entry-timeout=HIGH --negative-timeout=HIGH
22:44 glusterbot JoeJulian: --fopen-keep-cache
22:45 JoeJulian php's not very good.
22:45 ShaunR JoeJulian: i did read your acticle about improving perfomance and you talked about that with APC
22:45 JoeJulian Did you see the one about how php lies to you about writing a file? That was the last straw for me.
22:47 CyrilPeponnet Hey guys :) me again, regarding time sync in geo-rep, my master and slave are using ntp but they are in different timezone. Is geo-rep clever enough with that ?
22:47 JoeJulian should be
22:48 JoeJulian the stat structure should handle that.
22:48 JoeJulian @google statfs
22:48 glusterbot JoeJulian: statfs(2): file system statistics - Linux man page: <http://linux.die.net/man/2/statfs>; statfs(2) - Linux manual page - man7.org: <http://man7.org/linux/man-pages/man2/statfs.2.html>; StatFs | Android Developers: <http://developer.android.com/reference/android/os/StatFs.html>; statfs(2) Mac OS X Developer Tools Manual Page - Apple Developer:
22:48 glusterbot JoeJulian: <https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man2/statfs.2.html>; statfs(2) - The FreeBSD Project: <http://www.freebsd.org/cgi/man.cgi?query=statfs&amp;sektion=2>; statfs(2) - The FreeBSD Project: <http://www.freebsd.org/cgi/man.cgi?query=statfs&amp;sektion=2&amp;manpath=FreeBSD+5.0-RELEASE>; UNIX man pages : statfs (2) - UNIXhelp for users: (1 more message)
22:48 JoeJulian Gah, I meant @lucky
22:49 ShaunR JoeJulian: unfortunately this is for a customer, plus just about everything these days is php driven it seams
22:50 cyberbootje joined #gluster
22:50 ShaunR JoeJulian: i'm just trying to make this work, i would love for it too because i have customers constantly asking for clusters for their wordpress sites too.
22:51 ShaunR this is really my first time playing with gluster, i've used ceph in the past for block storage.  I started playing with cephfs for this but that doesnt look to be ready yet.
22:52 Vortac joined #gluster
22:53 Vortac Is there an easy way to tell if a replicated volume is in sync?
22:53 JoeJulian If you really want to experiment, try using http://quercus.caucho.com/ with the gluster jni.
22:54 JoeJulian Vortac: "gluster volume heal info" is generally accurate.
22:54 JoeJulian Otherwise it's building hash trees and comparing them.
22:54 ShaunR JoeJulian: those mount args above... is it talking about just appending them to the fstab like so.... gluster1:/gv0   /storage        glusterfs       defaults,_netdev,log-level=WARNING,log-file=/var/log/gluster.log,attribute-timeout=HIGH,entry-timeout=HIGH,negative-timeout=HIGH,fopen-keep-cache       1 2
22:55 Vortac JoeJulian: That command shows number of entires: 0 for all bricks.. so in sync?
22:55 JoeJulian ShaunR: yes
22:55 JoeJulian Vortac: yes
22:55 ShaunR JoeJulian: oh also, i'm mounting the fs and hosting gluster on the same servers... so app1,2,3 is a apache/php/mysql/gluster gluster
22:56 ShaunR so i'm mounting each volume locally really.
22:56 JoeJulian @mount server
22:56 glusterbot JoeJulian: (#1) The server specified is only used to retrieve the client volume definition. Once connected, the client connects to all the servers in the volume. See also @rrdns, or (#2) One caveat is that the clients never learn of any other management peers. If the client cannot communicate with the mount server, that client will not learn of any volume changes.
22:56 cyberbootje joined #gluster
22:56 JoeJulian ... oh, you meant the nfs mount.
22:56 JoeJulian I'm not thread safe.
22:57 ShaunR both
22:57 ShaunR i've done both fuse and nfs this way
22:57 JoeJulian I remember there was some problem with mounting nfs locally... I don't know that I ever knew the details on what the problem was though.
22:58 ShaunR hmm... 'glusterfs: unknown negative timeout HIGH'
22:58 JoeJulian Maybe he just meant some high number. Try 600
22:58 ShaunR ya, that worked
23:06 Vortac joined #gluster
23:09 gildub joined #gluster

| Channels | #gluster index | Today | | Search | Google Search | Plain-Text | summary