Perl 6 - the future is here, just unevenly distributed

IRC log for #gluster, 2016-11-30

| Channels | #gluster index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:03 panina joined #gluster
00:26 RameshN joined #gluster
00:32 farhorizon joined #gluster
00:36 farhorizon joined #gluster
00:54 shdeng joined #gluster
01:44 haomaiwang joined #gluster
02:04 jeremyh joined #gluster
02:05 ashp joined #gluster
02:05 ashp Hey guys, quick question I hope someone can help me with (as it's now an ~emergency~)
02:05 ashp I have a machine that mounts a glusterfs volume and while doing a find . in it
02:05 ashp it's slowly used 95% of the memory in the machine
02:06 ashp I don't have anything crazy in the logs, just high memory usage
02:06 ashp 9910 root      20   0 3841456 3.280g      0 S  0.2 89.3  94:45.75 glusterfs
02:12 Gambit15 joined #gluster
02:13 derjohn_mobi joined #gluster
02:14 haomaiwang joined #gluster
02:35 B21956 joined #gluster
02:39 k4n0 joined #gluster
02:46 jiffin joined #gluster
02:48 ilbot3 joined #gluster
02:48 Topic for #gluster is now Gluster Community - http://gluster.org | Documentation - https://gluster.readthedocs.io/en/latest/ | Patches - http://review.gluster.org/ | Developers go to #gluster-dev | Channel Logs - https://botbot.me/freenode/gluster/ & http://irclog.perlgeek.de/gluster/
02:49 jiffin joined #gluster
02:56 k4n0 joined #gluster
03:12 Lee1092 joined #gluster
03:14 haomaiwang joined #gluster
03:19 cliluw joined #gluster
03:21 magrawal joined #gluster
03:32 JoeJulian ashp What version?
03:48 Gnomethrower joined #gluster
03:52 Peppard joined #gluster
03:52 atinm joined #gluster
03:56 jiffin joined #gluster
04:03 ppai joined #gluster
04:11 Shu6h3ndu joined #gluster
04:14 haomaiwang joined #gluster
04:15 itisravi joined #gluster
04:17 k4n0 joined #gluster
04:17 RameshN joined #gluster
04:25 satya4ever joined #gluster
04:28 Prasad joined #gluster
04:39 Saravanakmr joined #gluster
04:42 buvanesh_kumar joined #gluster
04:48 k4n0 joined #gluster
05:00 sanoj joined #gluster
05:02 Prasad joined #gluster
05:14 haomaiwang joined #gluster
05:18 Prasad joined #gluster
05:19 LiftedKilt joined #gluster
05:20 karthik_us joined #gluster
05:23 aravindavk joined #gluster
05:23 ashiq joined #gluster
05:24 prasanth joined #gluster
05:25 buvanesh_kumar joined #gluster
05:30 ndarshan joined #gluster
05:31 kramdoss_ joined #gluster
05:32 suliba joined #gluster
05:35 skoduri joined #gluster
05:36 ankitraj joined #gluster
05:38 d0nn1e joined #gluster
05:39 rafi joined #gluster
05:39 prth joined #gluster
05:43 scuttle|afk joined #gluster
05:43 riyas joined #gluster
05:45 ndk_ joined #gluster
05:45 Karan joined #gluster
05:46 sbulage joined #gluster
05:46 prth joined #gluster
05:48 rastar joined #gluster
05:49 kotreshhr joined #gluster
05:53 nishanth joined #gluster
05:54 derjohn_mobi joined #gluster
05:56 apandey joined #gluster
05:56 magrawal joined #gluster
06:04 Prasad_ joined #gluster
06:11 hgowtham joined #gluster
06:14 haomaiwang joined #gluster
06:16 atinm joined #gluster
06:20 skoduri joined #gluster
06:21 Muthu joined #gluster
06:24 masuberu joined #gluster
06:24 kdhananjay joined #gluster
06:27 masber joined #gluster
06:28 prth joined #gluster
06:28 msvbhat joined #gluster
06:30 apandey joined #gluster
06:31 jiffin1 joined #gluster
06:43 hchiramm joined #gluster
06:44 jkroon joined #gluster
06:46 mhulsman joined #gluster
06:47 rastar joined #gluster
07:08 Gambit15 joined #gluster
07:13 ashiq joined #gluster
07:14 haomaiwang joined #gluster
07:15 atrius joined #gluster
07:19 Prasad joined #gluster
07:34 Gnomethrower joined #gluster
07:34 [diablo] joined #gluster
07:38 prth joined #gluster
07:41 ShwethaHP joined #gluster
07:42 jiffin1 joined #gluster
07:55 atinm joined #gluster
07:57 ndevos Ryllise: sorry, I cant find anyone that knows a company/person that does consultancy for Gluster in Australia... Red Hat does, but that means you need to use their product, not community packages
08:01 Philambdo joined #gluster
08:03 skoduri joined #gluster
08:04 TvL2386 joined #gluster
08:05 arc0 joined #gluster
08:14 haomaiwang joined #gluster
08:15 Karan joined #gluster
08:19 DV__ joined #gluster
08:25 devyani7 joined #gluster
08:26 fsimonce joined #gluster
08:26 devyani7 joined #gluster
08:31 mhulsman joined #gluster
08:41 sanoj joined #gluster
08:45 ivan_rossi joined #gluster
08:48 jri joined #gluster
09:12 rastar joined #gluster
09:13 apandey joined #gluster
09:14 ShwethaHP joined #gluster
09:14 haomaiwang joined #gluster
09:17 Slashman joined #gluster
09:21 phileas joined #gluster
09:27 hgowtham joined #gluster
09:30 panina joined #gluster
09:32 jiffin1 joined #gluster
09:35 buvanesh_kumar joined #gluster
09:36 masuberu joined #gluster
09:37 jeremyh joined #gluster
09:50 atinm joined #gluster
09:50 skoduri joined #gluster
09:50 flying joined #gluster
09:54 derjohn_mobi joined #gluster
09:56 poornima_ joined #gluster
09:57 susant joined #gluster
10:02 k4n0|afk joined #gluster
10:06 aravindavk joined #gluster
10:14 haomaiwang joined #gluster
10:16 jeremyh joined #gluster
10:17 d4n13L_ joined #gluster
10:18 aneil joined #gluster
10:18 jerrcs__ joined #gluster
10:19 martinet1 joined #gluster
10:19 RameshN joined #gluster
10:20 logan- joined #gluster
10:20 siel joined #gluster
10:21 amye_ joined #gluster
10:22 DV__ joined #gluster
10:23 twisted` joined #gluster
10:23 rastar joined #gluster
10:23 Peppard joined #gluster
10:23 Lee1092 joined #gluster
10:24 rideh joined #gluster
10:24 d0nn1e joined #gluster
10:24 virusuy joined #gluster
10:24 kshlm joined #gluster
10:24 Nebraskka joined #gluster
10:24 Kins joined #gluster
10:24 ndk_ joined #gluster
10:26 dataio joined #gluster
10:26 atrius joined #gluster
10:31 jesk joined #gluster
10:32 coredumb joined #gluster
10:35 jiffin1 joined #gluster
10:38 ankitraj joined #gluster
10:39 k4n0 joined #gluster
10:42 buvanesh_kumar joined #gluster
10:48 buvanesh_kumar joined #gluster
10:58 hackman joined #gluster
10:58 jeremyh1 joined #gluster
11:00 sbulage joined #gluster
11:03 Ryllise joined #gluster
11:04 hgowtham joined #gluster
11:14 ankitraj joined #gluster
11:14 haomaiwang joined #gluster
11:16 msvbhat joined #gluster
11:17 kenansulayman joined #gluster
11:22 atinm joined #gluster
11:23 flomko joined #gluster
11:28 kshlm #announcement Weekly community meeting starts in ~30 minutes in #gluster-meeting. Add updates and topics to https://bit.ly/gluster-community-meetings
11:28 glusterbot Title: Gluster Community Meeting - HackMD (at bit.ly)
11:31 ankitraj joined #gluster
11:32 Ulrar joined #gluster
11:32 cloaked1 joined #gluster
11:32 cloaked1 joined #gluster
11:48 bluenemo joined #gluster
11:50 primehaxor joined #gluster
12:05 jdarcy joined #gluster
12:09 arc0 joined #gluster
12:10 buvanesh_kumar joined #gluster
12:14 haomaiwang joined #gluster
12:22 kotreshhr left #gluster
12:24 k4n0 joined #gluster
12:33 sanoj joined #gluster
12:42 rwheeler joined #gluster
12:54 k4n0 joined #gluster
12:57 kdhananjay joined #gluster
12:58 prth joined #gluster
13:07 johnmilton joined #gluster
13:08 ashiq joined #gluster
13:09 TvL2386 joined #gluster
13:12 Gambit15 joined #gluster
13:14 haomaiwang joined #gluster
13:19 msvbhat joined #gluster
13:24 sbulage joined #gluster
13:34 ira joined #gluster
13:47 Shu6h3ndu joined #gluster
13:56 unclemarc joined #gluster
13:59 primehaxor joined #gluster
14:06 RameshN joined #gluster
14:08 mhulsman joined #gluster
14:08 haomaiwang joined #gluster
14:13 skylar joined #gluster
14:14 haomaiwang joined #gluster
14:14 k4n0 joined #gluster
14:16 circ-user-ht8F7 joined #gluster
14:18 jri_ joined #gluster
14:24 jiffin joined #gluster
14:27 jiffin1 joined #gluster
14:30 mhulsman joined #gluster
14:34 theron joined #gluster
14:41 sanoj joined #gluster
14:45 alvinstarr JoeJulian: Back to yesterdays question about rsync. Volumes A and B are on the same server and B is setup to geo-replicate offsite to volume C.  I tried to replicate C from A but it would never complete the initial copy so I then built B and replicated it to C and filling B from scratch let me replicate to C.
14:46 alvinstarr JoeJulian: A is a production volume. I can move production to B but I do not have the 6 hours as a maintenance window that rsync would take.
14:48 nishanth joined #gluster
14:59 sbulage joined #gluster
14:59 sage__ joined #gluster
15:05 Gambit15 joined #gluster
15:08 skoduri joined #gluster
15:09 vbellur joined #gluster
15:11 dnorman joined #gluster
15:14 haomaiwang joined #gluster
15:14 theron joined #gluster
15:15 annettec joined #gluster
15:26 fcoelho joined #gluster
15:30 hchiramm joined #gluster
15:33 squizzi joined #gluster
15:51 Caveat4U joined #gluster
15:53 Caveat4U joined #gluster
15:55 Wizek joined #gluster
15:55 plarsen joined #gluster
15:55 glusternovice joined #gluster
15:56 glusternovice hello all
15:56 farhorizon joined #gluster
15:57 glusternovice I am having issues mounting gluster client which worked before but at some point (no idea if it was a reboot or something else) it stopped so I checked logs and see it failed to mount
15:57 glusternovice any idea what info I can give so someone can tell what is it that may be going wrong?
16:00 unclemarc joined #gluster
16:08 cloph_away paste the exact error message when trying to mount.
16:09 mhulsman joined #gluster
16:13 johnjones joined #gluster
16:14 haomaiwang joined #gluster
16:17 hackman joined #gluster
16:31 rastar joined #gluster
16:36 dnorman joined #gluster
16:39 hackman joined #gluster
16:42 susant joined #gluster
16:45 wushudoin joined #gluster
16:46 wushudoin joined #gluster
16:49 farhorizon joined #gluster
16:53 k4n0 joined #gluster
16:56 JoeJulian glusternovice: Check the client log file in /var/log/glusterfs. The filename is based on the mountpoint.
17:00 glusternovice joined #gluster
17:00 glusternovice @joeljulian
17:00 glusternovice I did
17:00 vbellur joined #gluster
17:00 glusternovice anyway I can show add error message here for you to see?
17:02 JoeJulian @paste
17:02 glusterbot JoeJulian: For a simple way to paste output, install netcat (if it's not already) and pipe your output like: | nc termbin.com 9999
17:02 JoeJulian or use fpaste.org
17:04 glusternovice ok give me couple of mins while I do that
17:05 skylar joined #gluster
17:07 glusternovice @joejulian here is the text from log file
17:07 glusternovice https://paste.fedoraproject.org/494217/48052565/
17:07 glusterbot Title: #494217 • Fedora Project Pastebin (at paste.fedoraproject.org)
17:08 JoeJulian "gfs/data" is not a valid volume name.
17:12 theron joined #gluster
17:13 glusternovice thank you @joejulian.. I think I am missing something. I have another machine where I set this up but that wasn't is rebooted yet .. I will have to retrace my steps and see what I did to mount  the volume and what the names were
17:13 glusternovice I am drawing blank at this point so just have to think about it again
17:13 JoeJulian "gluster volume list" will tell you the name of the volumes available.
17:32 glusternovice @joejulian should I run that on server or client?
17:34 JoeJulian glusternovice: That'll need to be run on a server that's part of the peer group.
17:34 dnorman joined #gluster
17:35 glusternovice thanks I am going to see what the command was I used .. for the life of me I dont remember so I am gonna have to go to the one that still works.. thank you I wouldn't have figured out what you pointed out to me
17:35 shaunm joined #gluster
17:36 k4n0 joined #gluster
17:37 glusternovice got it working by looking at the one that is still working.thank you joe
17:43 glusternovice joe one last question, why wouldn't it mount when I have this line in the fstab? gluster01:/gv0 /shared glusterfs defaults,_netdev,backupvolfile-s​erver=gluster02,fetch-attemps=10 0 0
17:43 glusternovice did I miss anything when I put this entry in fstab?
17:44 JoeJulian sysvinit or systemd?
17:44 prth joined #gluster
17:47 ankitraj joined #gluster
17:49 theron joined #gluster
17:52 glusternovice @joejulian I am going to sound stupid but I have no idea
17:52 glusternovice I just made an entry in /etc/fstab and I always thought that would take care of it at start up
17:53 JoeJulian What distro are you using?
17:53 glusternovice ubuntu
17:53 JoeJulian version?
17:54 glusternovice 14.04
17:54 joef101 joined #gluster
17:54 JoeJulian sysvinit/upstart.
17:55 JoeJulian One problem you have is that ubuntu doesn't use the _netdev flag. It has no way to suggest that the volume should be mounted after the network starts and after glusterd starts.
17:56 glusternovice ah I see, so is there something else that ubuntu understands?
17:56 JoeJulian What it does is it tries to mount stuff before the network is started, then tries the stuff that failed again afterward (except for nfs or samba that it knows to treat differently).
17:57 joef101 <@JoeJulian> - I've learned something new, i though _netdev was standard between distro's
17:57 JoeJulian But you still have the chicken and egg problem. The mount tries to happen before the volume is started.
17:58 JoeJulian It seems, anecdotally, the majority of older ubuntu users just do their mounts in rc.local.
17:58 JoeJulian joef101: I thought so too, but I read through their source and found no such flag.
17:59 joef101 <@JoeJulian> - Is it the same case for Debian?
17:59 glusternovice so this is a ubuntu specific problem?
17:59 JoeJulian systemd makes this possible, now, with the x-systemd.require mount option.
18:00 JoeJulian It's an upstart problem which ubuntu and really old debian made heavy use of.
18:00 JoeJulian debian's been using systemd for a couple years now, iirc, and ubuntu has since 15.04.
18:01 glusternovice I can update the system to version 16 is that helps?
18:01 glusternovice this is all staging/ testing so any updates to the system are not a problem
18:01 glusternovice if that helps*
18:02 JoeJulian If you did that, you could then use x-systemd.require=glusterd.service as a mount option (assuming the service unit is named the same in ubuntu).
18:02 theron joined #gluster
18:04 glusternovice https://www.gluster.org/pipermail/gl​uster-users/2016-October/028777.html @joejulian
18:04 glusterbot Title: [Gluster-users] Reliably mounting a gluster volume (at www.gluster.org)
18:06 JoeJulian Interesting. I never have that problem with Arch or CentOS. I wonder why...
18:06 glusternovice @JoeJulian what would be a distro that won't have an issue? I am on AWS so the options are limited to RedHat, SUSE, Amazon Linux and Ubuntu
18:06 JoeJulian I think I still have an open feature request to do dbus integration, but that's never been worked on afaict.
18:07 JoeJulian The first three are all based on RHEL. I'd be torn between RHEL and SUSE.
18:08 ivan_rossi left #gluster
18:08 JoeJulian but that's just opinion. Everyone has their own favorite linux flavor.
18:08 joef101 <glusternovice> - Is there any reason why you can't use CentOS on AWS.
18:09 glusternovice I have used Ubuntu before so I was kinda used to it (again not really the way you may use it)
18:09 glusternovice @joef101 I dont see the option to pick that as an image
18:09 JoeJulian There are a ton of admins that use ubuntu successfully. I've just never agreed with many of their design philosophies.
18:10 glusternovice also really no idea on how to use Centos (syntax wise -- of course not difficult to look up if I have to)
18:11 glusternovice there are AMIs through community that I could pick .. which one would you go through ?
18:11 JoeJulian The only differences are packaging systems, some configuration location differences, and the fact that you have to actually enable and start services you want instead of having unconfigured services starting themselves as soon as they're installed.
18:13 glusternovice @JoeJulian : my only requirement at this point is I should be able to get up and running soon. I have been given some deadlines so which choice makes my life a little easier, I would appreciate that
18:15 joef101 <glusternovice> How many GlusterFS nodes you planning to use?
18:15 squizzi_ joined #gluster
18:15 glusternovice 2
18:19 joef101 <glusternovice> What type of volume? I'm asking because i'm also using 2 nodes which might lead to split-brain. I'm currently trying to recover from GlusterFS split-brain in Prod :)
18:19 joef101 Use 3 nodes if possible.
18:20 JoeJulian I would recommend at the very minimum adding an arbiter.
18:20 haomaiwang joined #gluster
18:20 glusternovice adding 3rd won't be a problem
18:20 glusternovice i have set it up as replication
18:21 glusternovice I think I will definitely setup a 3rd one to avoid split-brain issue.
18:22 glusternovice I have enough headaches and would rather avoid this one if I can right now
18:24 jiffin joined #gluster
18:27 joef101 <@JoeJulian> I need your help pls, I'm trying to recover a directory which is in split-brain state.
18:27 joef101 I've deleted non matching gfid link in .gluster directory and i triggered self-heal but that particular directory is still in split-brain state.
18:28 JoeJulian paste up the ,,(extended attribues) for the directory in question from all of your bricks.
18:28 glusterbot I do not know about 'extended attribues', but I do know about these similar topics: 'extended attribute', 'extended attributes'
18:28 joef101 <ndevos> advised me that i need to delete the directory plus gfid link.
18:29 JoeJulian bah ,,(extended attributes)
18:29 glusterbot (#1) To read the extended attributes on the server: getfattr -m .  -d -e hex {filename}, or (#2) For more information on how GlusterFS uses extended attributes, see this article: http://pl.atyp.us/hekafs.org/index.php/​2011/04/glusterfs-extended-attributes/
18:29 joef101 <@JoeJulian> - http://imgur.com/a/eItT2
18:29 glusterbot Title: Imgur: The most awesome images on the Internet (at imgur.com)
18:29 JoeJulian srsly?
18:30 JoeJulian Nifty. gfid mismatches. That sucks.
18:31 JoeJulian Since it's a directory, I would change the xattr to make them match and delete the .glusterfs entry for the /old/ gfid.
18:32 JoeJulian Once you fix all the directories, I suspect you'll probably need to rebalance fix-layout.
18:32 JoeJulian But I was also curious about the rest of the attributes. trusted.afr attributes can also keep the directories reporting split-brain.
18:35 joef101 <@JoeJulian> - Shit, i was hoping just by deleting .gfid link will fix the issue.
18:37 joef101 <@JoeJulian> - can you please point me to documentation on how to fix this?
18:39 rafi joined #gluster
18:46 JoeJulian Well, the extended attribute factoid has a link has some of the why and how.
18:46 JoeJulian Add to that https://joejulian.name/blog/what-is-​this-new-glusterfs-directory-in-33/
18:46 glusterbot Title: What is this new .glusterfs directory in 3.3? (at joejulian.name)
18:49 farhorizon joined #gluster
18:51 joef101 <@JoeJulian> Thanks, will read it.
19:00 btpier joined #gluster
19:06 dnorman joined #gluster
19:07 vbellur joined #gluster
19:11 btpier left #gluster
19:22 msvbhat joined #gluster
19:23 theron joined #gluster
19:25 dnorman joined #gluster
19:50 Ryan__ joined #gluster
20:00 d0nn1e joined #gluster
20:07 farhorizon joined #gluster
20:08 Caveat4U joined #gluster
20:11 Caveat4U_ joined #gluster
20:15 mhulsman joined #gluster
20:18 Caveat4U joined #gluster
20:29 dnorman joined #gluster
20:30 Caveat4U joined #gluster
20:31 johnmilton joined #gluster
20:37 Wojtek944 joined #gluster
20:50 Wojtek944 I get this message that loops in etc-glusterfs-glusterd.vol.log; What could be the cause? I can't find any keep-alive settings in /var/lib/glusterd/
20:50 Wojtek944 [2016-11-30 02:36:43.631675] W [socket.c:869:__socket_keepalive] 0-socket: failed to set TCP_USER_TIMEOUT -1000 on socket 8, Invalid argument
20:50 Wojtek944 [2016-11-30 02:36:43.631717] E [socket.c:2965:socket_connect] 0-management: Failed to set keep-alive: Invalid argument
20:51 JoeJulian which version is that? line 2965 in socket.c in 3.8.6 doesn't seem to have anything to do with that.
20:52 Wojtek944 it is 3.7.5
20:53 Wojtek944 All our other 3.7.5 clusters do not have these messages. Only this single one. Had a weird quorum bug on it and looking for a cause.
20:54 mhulsman joined #gluster
20:54 JoeJulian Well, that's in "socket_connect" so it must be during an attempt to connect to something. Would be nice if the error message gave some clue what it's trying to connect to.
20:55 JoeJulian "socket 8" I think you could figure out what that is with lsof
21:16 k4n0 joined #gluster
21:17 mhulsman joined #gluster
21:23 Caveat4U joined #gluster
21:25 Caveat4U joined #gluster
21:27 ahino joined #gluster
21:34 derjohn_mob joined #gluster
21:35 martin_pb joined #gluster
21:36 bhakti joined #gluster
21:49 k4n0 joined #gluster
22:00 Jacob843 joined #gluster
22:00 farhorizon joined #gluster
22:06 dnorman joined #gluster
22:08 martin_pb joined #gluster
22:10 nathwill joined #gluster
22:20 farhoriz_ joined #gluster
22:20 k4n0 joined #gluster
22:31 theron joined #gluster
22:38 dnorman joined #gluster
22:39 Caveat4U joined #gluster
22:50 k4n0 joined #gluster
22:54 masber joined #gluster
23:04 Caveat4U joined #gluster
23:06 vbellur joined #gluster
23:08 nathwill joined #gluster
23:08 haomaiwang joined #gluster
23:09 farhorizon joined #gluster
23:32 Caveat4U joined #gluster
23:38 kraynor5b joined #gluster
23:39 Marbug_ joined #gluster
23:40 primehaxor joined #gluster
23:41 cholcombe joined #gluster
23:43 Caveat4U joined #gluster
23:52 farhorizon joined #gluster

| Channels | #gluster index | Today | | Search | Google Search | Plain-Text | summary