Perl 6 - the future is here, just unevenly distributed

IRC log for #gluster, 2018-01-22

| Channels | #gluster index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:08 kettlewe_ joined #gluster
00:15 kettlewell joined #gluster
00:27 decayofmind joined #gluster
00:42 kettlewe_ joined #gluster
00:46 decayofmind joined #gluster
01:23 kettlewell joined #gluster
01:45 decayofmind joined #gluster
01:53 atrius joined #gluster
02:01 Shu6h3ndu joined #gluster
02:03 atinm joined #gluster
02:06 kettlewell joined #gluster
02:10 Shu6h3ndu joined #gluster
02:55 susant joined #gluster
02:56 ilbot3 joined #gluster
02:56 Topic for #gluster is now Gluster Community - https://www.gluster.org | Documentation - https://gluster.readthedocs.io/en/latest/ | Patches - https://review.gluster.org/ | Developers go to #gluster-dev | Channel Logs - https://botbot.me/freenode/gluster/ & http://irclog.perlgeek.de/gluster/
03:03 itisravi joined #gluster
03:11 kramdoss_ joined #gluster
03:12 gyadav joined #gluster
03:34 aravindavk joined #gluster
03:38 kramdoss_ joined #gluster
03:40 kotreshhr joined #gluster
03:41 Vishnu_ joined #gluster
03:43 Vishnu__ joined #gluster
03:50 kpease joined #gluster
03:57 kdhananjay joined #gluster
04:10 psony joined #gluster
04:12 jiffin joined #gluster
04:17 kramdoss_ joined #gluster
04:18 hgowtham joined #gluster
04:25 skumar joined #gluster
04:26 jkroon joined #gluster
04:27 hgowtham joined #gluster
04:27 skumar_ joined #gluster
04:27 DV joined #gluster
04:32 sunnyk joined #gluster
04:33 sunkumar joined #gluster
04:38 poornima joined #gluster
04:46 rafi1 joined #gluster
04:52 sanoj joined #gluster
04:56 nbalacha joined #gluster
04:57 sankarshan joined #gluster
05:19 decayofmind joined #gluster
05:20 ndarshan joined #gluster
05:24 gyadav joined #gluster
05:31 decayofmind joined #gluster
05:42 Prasad joined #gluster
05:50 msvbhat joined #gluster
05:54 poornima joined #gluster
06:15 vbellur joined #gluster
06:35 rouven joined #gluster
06:39 Saravanakmr joined #gluster
06:46 [diablo] joined #gluster
06:48 xavih joined #gluster
06:59 msvbhat joined #gluster
07:12 sanoj joined #gluster
07:21 jtux joined #gluster
07:24 ppai joined #gluster
07:25 apandey joined #gluster
07:27 varshar joined #gluster
07:27 jkroon joined #gluster
07:30 sunnyk joined #gluster
07:34 gyadav joined #gluster
07:46 kramdoss_ joined #gluster
07:55 gyadav joined #gluster
07:56 msvbhat joined #gluster
07:57 Klas are there any best practices for upgrading major versions of glusterfs?
07:58 Klas we realize that to be in supported version, we need to perform two major releases each year, currently considering once during summer and maybe once during winter holidays (at university, so that correlates highly with activity), but we are unsure as to how we are supposed to roll it out in a controlled fashion, preferrably without downtime and migration...
07:59 Jules- joined #gluster
08:00 Klas would this: http://docs.gluster.org/en/latest/Upgrade-Guide/upgrade_to_3.10/ be a good basis?
08:00 glusterbot Title: Upgrade to 3.10 - Gluster Docs (at docs.gluster.org)
08:02 Klas and is there any advantage to doing "small" steps or can they be ad hoc large?
08:06 inodb joined #gluster
08:10 samppah Klas: that's pretty good guide. Personally, I have been pretty conservative when doing updates. I'll stay longer with version which I know to work for sure and update when there is important features available and I'm sure version doesn't have any bugs that would affect my use case.
08:12 ndarshan joined #gluster
08:17 Klas I would prefer to use an actual stable version, but that concept doesn't exist in gluster
08:18 Klas an answer might be to run mayor upgrades only once a year and accept that we are out of support for some months every year
08:18 Klas samppah: is there a good way to perform rollbacks (generally) when things break?
08:18 msvbhat joined #gluster
08:19 Klas we roll our own packages from source, so I assume it would just be to install an older version above the new one?
08:22 samppah I use prebuilt rpm packages and it has been possible just to reinstall older version
08:27 rouven joined #gluster
08:30 ndevos Klas: the Long-Term-Maintenance releases get updates for a year, so it is not required to update every 6 months - 'Post 3.8' on https://www.gluster.org/release-schedule/
08:30 glusterbot Title: Gluster » Release Schedule (at www.gluster.org)
08:31 fsimonce joined #gluster
08:31 samppah ndevos: Hi! Do you have any idea if it's possible to clear locks on shards? (Re: http://lists.gluster.org/pipermail/gluster-users/2018-January/033297.html)
08:31 glusterbot Title: [Gluster-users] Stale locks on shards (at lists.gluster.org)
08:31 gyadav joined #gluster
08:32 ndevos samppah: sorry, I dont :-/
08:32 ndevos maybe krudika does, but she'd not here...
08:32 samppah ndevos: np, thanks anyway :)
08:32 samppah does she use irc?
08:34 nbalacha kdhananjay, ^^^
08:34 Klas ndevos: unless you upgrade the same day, yes, it is
08:34 Klas generally, upgrading to new major version should always be done 3-6 months after new release
08:34 Klas no specific to gluster of course
08:35 ndevos it's also "Krutika" - not sure why my tab completion didnt work for kd<tab>
08:36 ndevos Klas: well, we tried to create the release schedule so that users only need to upgrade once a year
08:36 Klas ndevos: then you need 18-24 month release cycle
08:37 Klas not 6-12 month =P
08:38 ndevos Klas: we had 18 months before, but that didnt work out for us developers, we want features to land sooner and get feedback earlier after we developed them
08:39 Klas I understand why devs want evergreen, but the result for proper lifecycle management means that you need 2 releases every year to be in support
08:39 ndevos the LTM/STM is sort of a middle way, get feedback from users interested in new features soon, but fix bugs in older releases for a while too
08:41 Klas yes, but short term is generally one year
08:41 ndevos unfortunately we dont have the manpower to maintain an older release for longer, 2 LTM releases and possibly one STM release makes us do an update 3x a month
08:41 Klas and long-term way longer =P
08:42 Klas I accept that it is that way, and you don't need to defend it
08:42 ndevos our short term is 3 months, 12 months is pretty long :)
08:42 Klas 12 months is way short =P
08:42 cyberbootje if someone could lend a hand it would be nice, i'm kinda stuck with slow write and read speeds on a gluster volume while the network and bricks are fast enough
08:42 Klas but, need to fix crap ;)
08:42 Klas (not related)
08:43 ndevos and now someone can throw in a sales pitch for Red Hat Gluster Storage ;-)
08:45 cyberbootje even tried ramdisks as bricks and got max ~7Gbit at the client so it has to be something with the connection and the native gluster fuse client
08:46 msvbhat joined #gluster
08:46 samppah cyberbootje: what kind of use case you have?
08:47 ndevos cyberbootje: make sure you write in large chunks, not just a few bytes (plain 'dd' vs 'dd bs=4M')
08:48 cyberbootje ndevos: for example i tried dd bs=1M
08:49 ndevos cyberbootje: you could increase that, and run more dd's in parallel
08:49 cyberbootje max i get is around 7Gbit while i know iperf does well over 9.6Gbit and a dd on the brick also does over 800MB
08:50 kramdoss_ joined #gluster
08:50 ndevos cyberbootje: in case you hit a bottleneck in the userspace<->kernel<->fuse<->glusterfs part, you can also have multiple mountpoints on the same client, others achieved more throughput that way
08:50 ivan_rossi joined #gluster
08:51 ivan_rossi left #gluster
08:51 cyberbootje hmm
08:52 cyberbootje i can try doing 2 x dd on different mounts i guess
08:54 ndevos even 2 dd's on the same mount could perform better than a single one
08:55 cyberbootje i'm testing it now, that doesn't seem to be the case
08:55 cyberbootje maybe a tiny bit, 40MB more together
08:57 cyberbootje ah i see, doing 2 x dd to 2 different mounts, same volume gives me flat 9.12Gbit
08:59 cyberbootje but my use case is as follows, i'm trying to get the most performance for a KVM VM cluster setup
09:00 cyberbootje i guess i'm not going to have the most out of the hardware in means of performance if i use gluster ?
09:00 msvbhat joined #gluster
09:01 kdhananjay samppah: yeah,so shard protects .shard directory from being accessed/made visible to the outside world (that's why it's hidden too!).
09:02 kdhananjay samppah: but if you say your hypervisor went down, then after some time all the locks that were previously granted to that node should be released automatically..
09:03 samppah kdhananjay: thank you for your answer! For some reason that automatic release of locks doesn't seem to work.
09:03 samppah i can recheck statedump again
09:08 poornima joined #gluster
09:12 cyberbootje ndevos: is there any way to tweak something to push out more speed ?
09:15 ndevos cyberbootje: in that case, you could maybe get more throughput by using qemu+gfapi instead of fuse mounts, that will basically give you a gfapi-mount inside qemu per VM
09:20 cyberbootje nvdevos: as fas asi can see i would need to recompile qemu to be able to use gfapi ?
09:24 apandey joined #gluster
09:29 Klas ndevos: yeah, it does seem like to design of the project is "if you actually want it, run RHGS, everything else is basically for dev purposes"
09:33 sanoj joined #gluster
09:35 psony|mtg joined #gluster
09:46 samppah kdhananjay: yeah, there are still locks granted to crashed node "inodelk.inodelk[0](ACTIVE)=type=WRITE, whence=0, start=0, len=0, pid = 3420, owner=d8b9372c397f0000, client=0x7f8858410be0, connection-id=ovirt8z2.xxx-5652-2017/12/27-09:49:02:946825-zone2-ssd1-vmstor1-client-0-7-0, granted at 2018-01-20 08:57:23" and host was restarted around 2018-01-20 09:02
09:53 msvbhat joined #gluster
09:53 misc joined #gluster
09:56 karthik_us joined #gluster
09:59 misc joined #gluster
10:01 Prasad joined #gluster
10:02 poornima joined #gluster
10:10 mbukatov joined #gluster
10:13 hgowtham joined #gluster
10:34 hgowtham joined #gluster
10:38 susant joined #gluster
10:39 psony joined #gluster
10:40 poornima joined #gluster
10:51 msvbhat joined #gluster
11:02 timmmey joined #gluster
11:06 timmmey Hello everyone
11:12 msvbhat joined #gluster
11:17 timmmmmey joined #gluster
11:23 jri joined #gluster
11:25 timmmmmey joined #gluster
11:29 rastar joined #gluster
11:43 shyam joined #gluster
11:44 kotreshhr joined #gluster
11:47 jri joined #gluster
11:55 kdhananjay joined #gluster
11:57 rafi joined #gluster
11:58 msvbhat joined #gluster
12:05 jri joined #gluster
12:15 kotreshhr joined #gluster
12:16 kettlewell joined #gluster
12:17 samppah kdhananjay: hi again :) there are still locks granted to crashed node "inodelk.inodelk[0](ACTIVE)=type=WRITE, whence=0, start=0, len=0, pid = 3420, owner=d8b9372c397f0000, client=0x7f8858410be0, connection-id=ovirt8z2.xxx-5652-2017/12/27-09:49:02:946825-zone2-ssd1-vmstor1-client-0-7-0, granted at 2018-01-20 08:57:23" and host was restarted around 2018-01-20 09:02
12:19 jri_ joined #gluster
12:19 Shu6h3ndu joined #gluster
12:20 buvanesh_kumar joined #gluster
12:29 jri joined #gluster
12:33 marbu joined #gluster
12:34 jri_ joined #gluster
12:35 decayofmind joined #gluster
12:42 atinm joined #gluster
12:48 kotreshhr left #gluster
12:50 JonathanD joined #gluster
12:57 ndarshan joined #gluster
12:59 phlogistonjohn joined #gluster
12:59 jri joined #gluster
13:04 jri_ joined #gluster
13:08 sunny joined #gluster
13:11 jri joined #gluster
13:17 jri_ joined #gluster
13:20 jri_ joined #gluster
13:23 darshan joined #gluster
13:28 jri joined #gluster
13:30 jri__ joined #gluster
13:34 shyam joined #gluster
13:34 jri joined #gluster
13:36 Drankis joined #gluster
13:38 rwheeler joined #gluster
13:39 shyam joined #gluster
13:51 ndarshan joined #gluster
13:53 decayofmind joined #gluster
14:08 Ashutto joined #gluster
14:10 Ashutto Hello. I have a split on a directory. I run a "gluster volume heal volname full" but I have no feedback about how is it going and the ext attributes:  trusted.afr.dirty=0x000000000000000000000000 and trusted.afr.volname-client-13=0x000000000000000000000000 are set
14:11 jiffin joined #gluster
14:11 plarsen joined #gluster
14:11 Ashutto my questions are: 1) how do i know healing is working? 2) is the volume heal the right way to get rid of my problem?
14:16 ndarshan joined #gluster
14:16 decayofmind joined #gluster
14:19 Ashutto c
14:22 Ashutto joined #gluster
14:22 Ashutto Hello. I have a problem with my gluster volume. It seems that a directory is in split but the heal doesn't work.
14:23 shyam joined #gluster
14:24 Ashutto Hello Shyam, may I ask you a question about my gluster volume in split ? :)
14:29 samppah Ashutto: is directory really splitted?
14:30 Ashutto no
14:31 samppah Is it possible that there are files in that directory that has to be healed/checked?
14:31 Ashutto it is not splitted, but this is the afr: trusted.afr.dirty=0x000000000000000000000000
14:31 Ashutto this is my first question :) . I run a "gluster volume heal volname full"
14:31 Ashutto but there is no lines on my logs that suggests heal is running
14:32 Ashutto or maybe done..
14:32 Ashutto the volume is modest, 14gb of small files, i know it is running as there is a different graph on I/O on the disks
14:32 samppah do you get any putput from gluster volume heal volname info
14:33 Ashutto all the bricks are in the same state:
14:33 Ashutto Status: Connected
14:33 Ashutto Number of entries: 0
14:33 samppah ok
14:39 Ashutto do you have a suggestion on how to understand if the healing process is working properly?
14:39 mbukatov joined #gluster
14:40 rouven_ joined #gluster
14:41 pdrakeweb joined #gluster
14:43 rouven joined #gluster
14:44 nbalacha joined #gluster
14:47 pdrakeweb joined #gluster
14:48 phlogistonjohn joined #gluster
14:48 pdrakewe_ joined #gluster
14:48 jmulligan joined #gluster
14:53 pdrakeweb joined #gluster
15:00 skylar1 joined #gluster
15:01 sunny joined #gluster
15:06 pdrakewe_ joined #gluster
15:10 pdrakeweb joined #gluster
15:22 bowhunter joined #gluster
15:25 jri joined #gluster
15:27 jri_ joined #gluster
15:30 Teraii joined #gluster
15:31 ThHirsch joined #gluster
15:39 Ashutto Is there a way to understand if the healing process is working?
15:40 Gugge_ joined #gluster
15:41 XpineX joined #gluster
15:41 portdirect joined #gluster
15:41 lunaaa joined #gluster
15:44 shyam joined #gluster
15:45 Jules- joined #gluster
15:45 decayofmind joined #gluster
15:46 wolfshappen joined #gluster
15:50 percevalbot joined #gluster
16:00 ashka hi, what is the minimal gluster version in which remove-brick properly moves files from the removed brick to other bricks in a distribute volume?
16:00 decayofmind joined #gluster
16:08 MrAbaddon joined #gluster
16:15 kpease joined #gluster
16:17 decayofmind joined #gluster
16:23 Guest83 joined #gluster
16:31 shyam left #gluster
16:32 DV joined #gluster
16:35 shyam joined #gluster
16:59 bowhunter joined #gluster
16:59 armyriad joined #gluster
17:00 mallorn Argh.  The new 3.13.2 build for CentOS has the same liburcu-bp.so.1 problem that 3.13.1 had.
17:02 mallorn Are we supposed to downgrade to userspace-rcu-0.7?
17:08 gyadav joined #gluster
17:13 Rakkin joined #gluster
17:17 shyam mallorn: Was just testing the same out as well, and it seems to be the case. Let me poke the packaging team to understand this better
17:18 shyam 3.13.2 should still be in the test repo, not published yet, so we maybe able to respin and correct this
17:19 kevinpz111 joined #gluster
17:19 kevinpz111 Hi guys
17:19 kevinpz111 I'm facing a little issue with gluster-kubernetes
17:20 kevinpz111 It seems that wehn I deploy it using the gk-deploy tool, it breaks my weave network, and I have to restart the weave-kube on my kubernetes worker to have the DNS resolution work again
17:20 DV joined #gluster
17:22 kevinpz111 Does someone have an idea on why it breaks my kubernetes network?
17:27 FuzzyVeg joined #gluster
17:27 ndevos kevinpz111: that is probably related to https://github.com/gluster/gluster-kubernetes/pull/421
17:27 glusterbot Title: vagrant: use a matching version of Weave Net by nixpanic · Pull Request #421 · gluster/gluster-kubernetes · GitHub (at github.com)
17:28 mallorn Thanks, shyam!
17:28 samppah Is it possible to force self healing to stop?
17:30 kevinpz111 ndevos: I'm not using vagrant :( I have a K8S already deployed, and I just use gk-deploy on it
17:32 ndevos kevinpz111: oh, then I'm not sure why that could be...
17:35 kevinpz111 Before installing the gluster-kubernetes, all work well, but when I start the gk-deploy tools, if I try to make nslookup DNS request on a centos container, I get the following issue : "reply from unexpected source:"
17:35 kevinpz111 And I'm sure it is caused by gluster installation, because if I don't install it, all work well :(
17:36 ndevos maybe one of the guys in #heketi has seen that before, you could ask there
17:36 JoeJulian That's a masqurade error
17:36 JoeJulian masquerade even
17:37 kevinpz111 JoeJulian: yep, it what I found, the K8S dns answer is not done by the services IP, but directly by the POD IP
17:38 kevinpz111 But that issue only appears when I start the gk-deploy script
17:39 kevinpz111 And so, the centos doesn't trust it to prevent DNS attacks
17:40 JoeJulian Is that a single pod that's giving that error, or is it cluster-wide?
17:40 kevinpz111 JoeJulian: Do you have any idea on why the gk-deploy seems to break the masquerade on the node?
17:40 kevinpz111 All my pods on the cluster, they can't have the DNS resolution working due to the masquerade error
17:41 JoeJulian None. I haven't used gk-deploy yet. So far, my use of gluster with kubernetes does not fall within the design spec of heketi.
17:41 kevinpz111 I have to restart my docker.io/weaveworks/weave-kube container on each of my worker to get it back
17:44 JoeJulian I'm looking through the kubernetes templates to see if I can see anything obvious.
17:54 msvbhat joined #gluster
17:54 rwheeler joined #gluster
17:58 rastar joined #gluster
18:03 ference joined #gluster
18:04 rastar joined #gluster
18:06 nadley joined #gluster
18:20 jkroon joined #gluster
18:21 shyam joined #gluster
18:26 legreffier joined #gluster
18:30 ference Hi, I wonder if you can help me with this: we are running a cluster with 5 nodes, all of which mount the same volume: `shared`. Users upload images with our app and the uploaded images then get rendered in subsequent requests. The nodes are load-balanced, so a user might upload on node 1, but the subsequent read might come from node 2. We received reports that sometimes it looks like the images have not been uploaded, because they "disappear" af
18:30 ference gluster volume info shared
18:30 ference
18:30 ference Volume Name: shared
18:30 ference Type: Replicate
18:30 ference Volume ID: 358fc80c-939f-40dc-9801-945ea27e1891
18:30 ference Status: Started
18:30 ference Snapshot Count: 0
18:30 ference Number of Bricks: 1 x 2 = 2
18:30 ference Transport-type: tcp
18:30 ference Bricks:
18:30 ference Brick1: td-idvos-file01:/glusterfs/brick1/shared
18:30 ference Brick2: td-idvos-file02:/glusterfs/brick1/shared
18:30 ference Options Reconfigured:
18:31 ference left #gluster
18:31 samppah ference: could you sent full output of gluster volume info to fpaste.org?
18:32 samppah :/
18:35 ference joined #gluster
18:36 rouven joined #gluster
18:36 glusterbot joined #gluster
18:37 nh2[m] is it normal that my glusterd has 2 glusterfsd child processes, even though I have only 1 volume? I'm trying to debug a race condition in gluster's startup code, and am not sure whether it's the presence of the 2 processes or their interaction that's wrong
18:46 msvbhat joined #gluster
18:49 JoeJulian nh2[m]: yeah I think that sounds wrong. glustershd and nfs should be glusterfs not glusterfsd.
18:49 illwieckz joined #gluster
18:51 nh2[m] JoeJulian: thanks. I'm in a multi-day debugging session to figure this out. I think it all originates in this https://github.com/gluster/glusterfs/blob/8efa3ee675a991410c6aa27dce40d4dd441d0935/xlators/mgmt/glusterd/src/glusterd-utils.c#L5954-L5968
18:51 glusterbot Title: glusterfs/glusterd-utils.c at 8efa3ee675a991410c6aa27dce40d4dd441d0935 · gluster/glusterfs · GitHub (at github.com)
18:51 shyam joined #gluster
18:53 JoeJulian nh2[m]: What's your volume info, and what version are you using?
18:53 nh2[m] unfortunately I find gluster's code is ridden by race conditions left and right, so it's really difficult to pinpoint the issue. My latest understanding is that because of the bad pidfile handling, there are 2 glusterfsds running for the same volume at the same time, but it all magically works nevertheless, until the other race condition comes in where they start exactly at the same time trying to create the same unix domain
18:53 nh2[m] socket in the changelog translator, at which point one fails
18:54 JoeJulian makes sense
18:54 JoeJulian Oh!
18:54 JoeJulian I wonder if that explains a symptom I've seen occasionally.
18:57 nh2[m] JoeJulian: I also found at least 4 more bugs in the changelog functionality, but it's just all so much I feel like I could file bugs all day long, so I'm just trying to fix the most severe ones and I will probably try to not use changelog and thus glusterfind/geo-rep at all because code quality is too low, it can't work :/ Really wish it did though
18:57 nh2[m] JoeJulian: volume info is this, very basic 3-node replica: http://lpaste.net/361879
18:57 glusterbot Title: gluster volume info :: lpaste — Lambda pastebin (at lpaste.net)
18:57 hvisage joined #gluster
18:58 JoeJulian Most of the start-up races should go away with glusterd2
19:00 nh2[m] JoeJulian: do you see `Unable to read pidfile:` in your logs? For me it happens every 4th server reboot or so
19:00 nh2[m] (that is, glusterd.log)
19:01 JoeJulian My last reboot has aged out. I'll set up some RPi servers when I get home.
19:03 nh2[m] I'm using automated deployments that bring up a test cluster per commit of my code so I literally see this issue all the time
19:04 JoeJulian So that would be in `glusterd_service_stop_nolock`
19:04 JoeJulian Which fails because the process already stopped.
19:04 JoeJulian I think
19:05 nh2[m] JoeJulian: my logs have it come from gf_is_service_running
19:05 nh2[m] (there are 3 ocurrences of this string in the code)
19:06 ference left #gluster
19:06 samppah JoeJulian: Do you happen to know if it's possible to force stop/start self heal deaemon?
19:07 samppah or what happens if process is just killed?
19:07 JoeJulian samppah: not really.. I mean you can disable the self-heal daemon through volume options. I've not tested to see if that stops a currently running heal.
19:09 bjonnh joined #gluster
19:09 * nh2[m] sent a long message: nh2[m]_2018-01-22_19:09:29.txt <https://matrix.org/_matrix/media/v1/download/matrix.org/gYfTdCrgnHbhNvGbAjGsbRWh>
19:09 bjonnh what is the best way for a server to access to add files to a volume? start a client and mount in another directory?
19:09 samppah JoeJulian: yeah, I was thinking about that too
19:10 JoeJulian bjonnh: yep!
19:13 bjonnh JoeJulian: thx
19:14 skylar1 joined #gluster
19:16 JoeJulian nh2[m]: `brickinfo->status = GF_BRICK_STARTING` _should_ prevent a race. I can't see any other way to get to where the same process is started twice.
19:17 JoeJulian lol
19:17 JoeJulian 5970:         * TBD: pray for GlusterD 2 to be ready soon.
19:23 nh2[m] JoeJulian: here is the full log between the two "starting a fresh brick process":
19:23 nh2[m] https://gist.github.com/nh2/4e04a9a5d168f28ee925074a6b9c1170
19:23 glusterbot Title: GlusterFS "starting fresh brick process" happening twice · GitHub (at gist.github.com)
19:29 JoeJulian nh2[m]: That almost looks like there are two glusterd running...
19:30 JoeJulian Is that even possible?
19:36 JoeJulian I do wish "[2018-01-22 04:41:06.094048] I [MSGID: 106005] [glusterd-handler.c:6071:__glusterd_brick_rpc_notify] 0-management: Brick 10.0.0.2:/data/glusterfs/myvol/brick1/brick has disconnected from glusterd." included the pid.
20:32 nh2[m] JoeJulian: yeah the total absence of pids in the logs is very annoying. I'm quite sure I see only 1 glusterd, as that is controlled by systemd. Do you know what "has disconnected from glusterd" means precisely? Is it a socket connection or the subprocess exiting?
20:36 Kassandry joined #gluster
21:07 bjonnh is there a way to boost the speed of glusterfs by not replicating for a short period of time?
21:07 bjonnh I am creating big VMs images
21:07 bjonnh and it seems to be much slower on glusterfs volume than on host
21:16 Rakkin joined #gluster
21:23 MrAbaddon joined #gluster
21:30 shyam joined #gluster
21:36 bjonnh also I'm getting "Mount process exited, code=exited status=1"
21:37 bjonnh when I try to mount the gluster volume at boot time
21:38 bjonnh (I can mount it manually)
21:38 bjonnh local_machine_hostname:/vm_drives /mnt/vm_drives glusterfs defaults,_netdev 0 0
21:38 bjonnh is what I have in my fstap
21:38 bjonnh b
21:40 shyam joined #gluster
21:40 bjonnh maybe I should use the other gluster host
21:47 Vapez joined #gluster
21:51 shyam joined #gluster
22:20 ference joined #gluster
22:21 ference left #gluster
22:25 ference joined #gluster
22:44 decayofmind joined #gluster
22:47 JoeJulian nh2[m]: finished lunch and meetings... In that case __glusterd_brick_rpc_notify received a RPC_CLIENT_DISCONNECT event which seems to come from RPC_TRANSPORT_DISCONNECT being set which seems to happen, afaict, at [2018-01-22 04:41:06.093668] I [socket.c:2475:socket_event_handler] 0-transport: EPOLLERR - disconnecting now
22:49 JoeJulian bjonnh: This is what I use:
22:49 JoeJulian gluster:vminstances /var/lib/nova/instances glusterfs defaults,x-systemd.require=glusterd.service 0 0
22:49 JoeJulian Either that or create a systemd.mount resource
22:51 JoeJulian Anyway... yes. You can just disconnect the client from one of the hosts. The heal will probably take about 4 times longer than the creation though, I suspect.
22:52 bjonnh it seems to work by using the other gluster host
22:52 bjonnh so I guess it is really that it is not started yet
22:52 JoeJulian It is.
22:52 bjonnh so your line may help
22:52 bjonnh thx
22:53 JoeJulian You're welcome. It's best to use round robin hostnames (imho) for mounts to allow the client to use any glusterd that's up, and the require to allow the client to work from a cold cluster boot.
23:03 shellclear joined #gluster
23:06 armyriad joined #gluster
23:15 bjonnh JoeJulian: my setup is probably to simple to start having round robin hostnames
23:16 bjonnh I don't really have dns
23:17 JoeJulian You can do that with hosts files too
23:20 ference left #gluster
23:25 burn^ joined #gluster
23:26 bjonnh how so?
23:27 tom[] joined #gluster
23:27 Somedream joined #gluster
23:27 lunaaa joined #gluster
23:28 shyam joined #gluster
23:28 tdasilva joined #gluster
23:28 kshlm joined #gluster
23:29 Chinorro joined #gluster
23:32 valkyr3e joined #gluster
23:49 decayofmind joined #gluster
23:54 tom[] joined #gluster

| Channels | #gluster index | Today | | Search | Google Search | Plain-Text | summary