Perl 6 - the future is here, just unevenly distributed

IRC log for #gluster, 2015-06-25

| Channels | #gluster index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:14 victori joined #gluster
00:20 hagarth joined #gluster
00:21 DV joined #gluster
00:47 elico joined #gluster
00:53 aaronott joined #gluster
00:59 smohan joined #gluster
01:07 shyam joined #gluster
01:10 wkf joined #gluster
01:19 elico joined #gluster
01:34 Folken joined #gluster
01:35 Folken hey when I do gluster heal datpoint info
01:35 Folken I get
01:35 Folken Brick DarkDream:/glusterfs/
01:35 Folken <gfid:e6c5ed1d-77cf-4b0e-941a-53d1abd0f9d1>
01:35 Folken how do I nuke that from the brick DarkDream
01:46 DV joined #gluster
01:46 DV__ joined #gluster
02:03 victori joined #gluster
02:09 corretico joined #gluster
02:15 elico joined #gluster
02:16 nangthang joined #gluster
02:22 elico joined #gluster
02:28 kdhananjay joined #gluster
02:38 glusterbot News from newglusterbugs: [Bug 1200914] pathinfo is wrong for striped replicated volumes <https://bugzilla.redhat.com/show_bug.cgi?id=1200914>
02:40 hagarth joined #gluster
02:52 elico joined #gluster
03:01 hagarth JoeJulian: around?
03:14 Folken anybody?
03:15 victori joined #gluster
03:15 aaronott joined #gluster
03:17 bharata-rao joined #gluster
03:18 atinm joined #gluster
03:19 elico joined #gluster
03:22 sripathi joined #gluster
03:37 kevein joined #gluster
03:38 glusterbot News from newglusterbugs: [Bug 1235512] quorum calculation might go for toss for a concurrent peer probe command <https://bugzilla.redhat.com/show_bug.cgi?id=1235512>
03:40 TheSeven joined #gluster
03:43 akay1 Folken: in case you havent found it yet, use the gfid-resolver to find out what the actual file is - https://gist.github.com/semiosis/4392640
03:46 krink joined #gluster
03:47 krink i have a split-brain volume.  need help with best solution to fix
03:49 RameshN joined #gluster
03:56 elico joined #gluster
04:06 shubhendu joined #gluster
04:08 maveric_amitc_ joined #gluster
04:13 sakshi joined #gluster
04:20 gem joined #gluster
04:27 rjoseph joined #gluster
04:29 dusmant joined #gluster
04:31 elico joined #gluster
04:40 Folken akay1: thanks
04:43 vimal joined #gluster
04:44 nbalacha joined #gluster
04:45 raghu joined #gluster
04:49 zeittunnel joined #gluster
04:50 elico joined #gluster
04:53 sripathi joined #gluster
04:56 pppp joined #gluster
04:57 elico joined #gluster
05:04 corretico joined #gluster
05:08 anrao joined #gluster
05:09 glusterbot News from resolvedglusterbugs: [Bug 1202745] glusterd crashed on one of the node <https://bugzilla.redhat.com/show_bug.cgi?id=1202745>
05:11 ndarshan joined #gluster
05:11 hgowtham joined #gluster
05:13 victori joined #gluster
05:16 ashiq joined #gluster
05:19 bharata-rao joined #gluster
05:19 victori joined #gluster
05:21 rafi joined #gluster
05:27 victori joined #gluster
05:29 jiffin joined #gluster
05:29 gem joined #gluster
05:30 Manikandan joined #gluster
05:30 spandit joined #gluster
05:32 Bhaskarakiran joined #gluster
05:36 spandit_ joined #gluster
05:39 glusterbot News from resolvedglusterbugs: [Bug 1217722] Tracker bug for Logging framework expansion. <https://bugzilla.redhat.com/show_bug.cgi?id=1217722>
05:44 kdhananjay joined #gluster
05:48 victori joined #gluster
06:03 maveric_amitc_ joined #gluster
06:05 Folken interesting
06:05 Folken when I run
06:05 Folken find /mnt -d -exec getfattr -h -n trusted.ec.heal {} \;
06:05 Folken I get a whole lot of /mnt/hts/Peppa Pig-57.mkv: trusted.ec.heal: Transport endpoint is not connected
06:06 soumya_ joined #gluster
06:07 overclk joined #gluster
06:08 nsoffer joined #gluster
06:09 glusterbot News from resolvedglusterbugs: [Bug 1226233] Mount broker user add command removes existing volume for a mountbroker user when second volume is attached to same user <https://bugzilla.redhat.com/show_bug.cgi?id=1226233>
06:10 spandit_ joined #gluster
06:13 atalur joined #gluster
06:21 deepakcs joined #gluster
06:37 anrao joined #gluster
06:37 jtux joined #gluster
06:38 vimal joined #gluster
06:44 nangthang joined #gluster
06:49 elico joined #gluster
06:56 gem joined #gluster
07:05 saurabh_ joined #gluster
07:06 shubhendu joined #gluster
07:09 glusterbot News from newglusterbugs: [Bug 1235538] Porting the left out gf_log messages to the new logging API <https://bugzilla.redhat.com/show_bug.cgi?id=1235538>
07:09 glusterbot News from newglusterbugs: [Bug 1235543] Upcall: Directory or file creation should send cache invalidation requests to parent directories <https://bugzilla.redhat.com/show_bug.cgi?id=1235543>
07:09 glusterbot News from newglusterbugs: [Bug 1211614] [NFS] Shared Storage mounted as NFS mount gives error "snap_scheduler: Another snap_scheduler command is running. Please try again after some time" while running any scheduler commands <https://bugzilla.redhat.com/show_bug.cgi?id=1211614>
07:11 bharata-rao joined #gluster
07:18 kdhananjay joined #gluster
07:20 elico joined #gluster
07:21 rjoseph joined #gluster
07:22 kshlm joined #gluster
07:23 meghanam joined #gluster
07:24 Saravana joined #gluster
07:27 vimal joined #gluster
07:27 elico joined #gluster
07:31 chirino_m joined #gluster
07:42 Trefex joined #gluster
07:45 autoditac joined #gluster
07:47 arao joined #gluster
07:49 elico joined #gluster
07:52 autoditac joined #gluster
07:53 ctria joined #gluster
08:01 teknologeek joined #gluster
08:02 teknologeek hi all
08:04 teknologeek I just upgraded glusterfs from 3.7.1 to 3.7.2. I use it on AWS and all nodes are created from the same GlusterFS custom AMI
08:04 teknologeek Problem is that i can't probes peer nodes because they have the same uuid
08:05 teknologeek Is there a work around or should I just downgrade to 3.7.1 ?
08:07 elico joined #gluster
08:18 teknologeek nobody has got some clue about this ?
08:34 elico left #gluster
08:37 spalai joined #gluster
08:38 sysconfig joined #gluster
08:38 ramteid joined #gluster
08:38 spandit joined #gluster
08:39 ghenry joined #gluster
08:39 ghenry joined #gluster
08:39 glusterbot News from resolvedglusterbugs: [Bug 1222065] GlusterD fills the logs when the NFS-server is disabled <https://bugzilla.redhat.com/show_bug.cgi?id=1222065>
08:43 mator ^^ my report
08:44 gem joined #gluster
08:44 mator ndevos, no luck with this nfs so far... starting from glusterfs-3.3.x version, look at https://bugzilla.redhat.com/show_bug.cgi?id=847821
08:45 glusterbot Bug 847821: low, medium, ---, bugs, NEW , After disabling NFS the message "0-transport: disconnecting now" keeps appearing in the logs
08:46 tanuck joined #gluster
08:46 atalur joined #gluster
08:46 ppai joined #gluster
08:53 arao joined #gluster
08:57 gem joined #gluster
09:03 ababu joined #gluster
09:05 atalur joined #gluster
09:08 harish_ joined #gluster
09:09 glusterbot News from newglusterbugs: [Bug 847821] After disabling NFS the message "0-transport: disconnecting now" keeps appearing in the logs <https://bugzilla.redhat.com/show_bug.cgi?id=847821>
09:19 corretico joined #gluster
09:20 teknologeek joined #gluster
09:23 teknologeek hi again, nobody has an answer ?
09:24 Ulrar Sorry, no idea, but you should wait until someone comes by
09:27 abyss_ teknologeek: as far as I understand you don't have issue with the same id because you upgrade you have the problem because you set up one gluster on one instance and you clone this.
09:29 ababu joined #gluster
09:34 teknologeek i found the problem thx
09:35 abyss_ :)
09:35 teknologeek sm1 started the gluster daemon before taking the snapshot
09:36 teknologeek so I had a /var/lib/glusterd/glusterd.info file generated in the AMI
09:36 teknologeek same on all nodes
09:39 jcastillo joined #gluster
09:39 kdhananjay joined #gluster
09:39 glusterbot News from newglusterbugs: [Bug 1231024] scrub frequecny and throttle change information need to be present in Scrubber log <https://bugzilla.redhat.com/show_bug.cgi?id=1231024>
09:45 jiffin1 joined #gluster
09:47 Ulrar How do the locks work ? If I have a VM running on a host storing it's disk file on glusterFS and that hypervisor fails and the VM tries to reboot on another hypervisor, will it be able to access it's disk ? If yes, what will happen if the first hypervisor wasn't actually dead but just had a network problem, will both be able to access the disk at the same time ? I'd guess the first one wouldn't be able to
09:47 Ulrar get back it's lock on the file and would just freez ?
09:51 meghanam joined #gluster
09:57 SpComb^ Ulrar: what hypervisor, and how does it lock the disk file?
10:09 Slashman joined #gluster
10:09 AdrianH joined #gluster
10:10 glusterbot News from newglusterbugs: [Bug 1235601] tar on a glusterfs mount displays "file changed as we read it" even though the file was not changed <https://bugzilla.redhat.com/show_bug.cgi?id=1235601>
10:15 necrogami joined #gluster
10:17 vmallika joined #gluster
10:18 husanu4 joined #gluster
10:19 LebedevRI joined #gluster
10:24 RameshN joined #gluster
10:26 husanu4 joined #gluster
10:27 jiffin joined #gluster
10:28 AdrianH Hi, I am trying to replace a crashed server, I have followed this http://www.gluster.org/community/documentation/index.php/Gluster_3.4:_Brick_Restoration_-_Replace_Crashed_Server but when I peer probe one of the servers from the new one I get "peer probe: failed: Failed to get handshake ack from remote server", any ideas ?
10:28 meghanam_ joined #gluster
10:29 husanu2 joined #gluster
10:30 curratore joined #gluster
10:30 curratore good morning
10:31 vovcia AdrianH: probe new server from inside the cluster
10:31 spalai1 joined #gluster
10:32 AdrianH Oh ok, I was doing it the other way round, thanks, will try that now
10:34 curratore anyone could help me with some questions plz?
10:35 AdrianH vovcia: so I am getting "peer probe: success. Host Gluster1 port 24007 already in peer list", restarted gluster on the new server (new server has same hostname and ip) but on the new server I am still getting "peer status: No peers present"
10:36 AdrianH curratore: just ask :)
10:36 vovcia curratore: and wait :)
10:37 curratore ok
10:37 curratore thxs
10:39 curratore I have an old replica with 2 bricks, I added a third brick and deleted the old one (named gfs-1)
10:39 curratore now i have again in my volume 2 bricks
10:39 msvbhat AdrianH: Try remoing a peer (gluster peer detach) and then adding it (peer probe)
10:39 corretico joined #gluster
10:40 AdrianH msvbhat: ok thanks
10:40 curratore status: started
10:40 rafi1 joined #gluster
10:40 curratore replicated
10:40 curratore and gfs-2 and 3 are connected
10:41 curratore the fact is that i don't know why the old info contained in gfs-1 (detached and deleted) and gfs2 active and replicated is not synced with gfs3
10:41 AdrianH msvbhat: getting "peer detach: failed: Brick(s) with the peer gluster1 exist in cluster"
10:42 curratore but if i add new info with clients from other sources this new info it is storaged and synced
10:43 rafi joined #gluster
10:45 curratore any test or advices to sync all the data from gfs2 to gfs3?
10:47 msvbhat AdrianH: Do you have active volumes in the cluster?
10:47 anrao joined #gluster
10:47 AdrianH yes one volume, this is a production setup, one of the servers died yesterday, so I scrapped it a got a new one, same setup, same hostname, same IP.
10:48 arao joined #gluster
10:49 curratore ye
10:49 curratore 1 called datastore
10:50 curratore the info is replicated, started, 1x2bricks, transport:tcp and the bricks names (gfs2 and gfs3)
10:50 curratore volume info I mean
10:51 msvbhat AdrianH: Same setup except for the brick right? I mean you don't have brick in that machine yet...
10:51 curratore yeye i have the new brick up and running
10:51 curratore in 2 different machines
10:52 curratore connected
10:53 curratore # gluster peer status Number of Peers: 1  Hostname: gfs-3.xxx.xxx Uuid: cd535706-272f-4123-b215-dd780c306f98 State: Peer in Cluster (Connected)
10:53 curratore gfs-3:/# gluster peer status Number of Peers: 1  Hostname: 172.16.0.2 Uuid: e4801476-f452-4c0a-bc3a-79f24bba451a State: Peer in Cluster (Connected) Other names: gfs-2.xxx.xxx
10:54 AdrianH msvbhat: no bricks with data, but the directories are ready same location same size, same version of gluster ( my setup is 4 servers: Distributed-Replicate 4 x 2)
10:56 AdrianH msvbhat: I've haven't got any data on the new server, was lost yesterday (thanks AWS) I thought I could jsut replace the dead server with the new one and Gluster would do it's magic and heal itself?
10:58 msvbhat AdrianH: Your new machine would have different UUID. I don't think in that case gluster can automagically heal things. I suppose it's a replicated setup?
10:58 msvbhat AdrianH: There should be a documentation somewhere for this use case
10:58 AdrianH msvbhat: Distrubuted and replicated
10:58 * msvbhat trying to find it
10:59 AdrianH msvbhat: I guess I am doing this wrong then, should I try to replace the 2 bricks
10:59 AdrianH msvbhat: instead of the  server
11:02 msvbhat AdrianH: No, you *should* be able to sync from it's replica pair. But I'm trying to find some blog/docs about this use case. I'm pretty sure it's been tried by someone already
11:03 msvbhat AdrianH: BTW what does the gluster volume info/status say? That still list the bricks from the node which was replaced?
11:03 AdrianH State: Peer in Cluster (Disconnected)
11:04 AdrianH sorry:
11:04 hagarth joined #gluster
11:05 AdrianH msvbhat: yes it lists all the bricks
11:05 AdrianH msvbhat: but "gluster volume status" doesn't list the bricks of the down server
11:06 msvbhat AdrianH: Hmm, then IMO following should work (don't try it yet, i'll try and find the docs)
11:08 msvbhat AdrianH: Actually scratch that, it wouldn't work :) :(
11:09 AdrianH msvbhat: ok thanks a lot for your help, I am also searching. Just a bit scared to F$$$ everything up, this is live and online and everything is still working fine
11:09 * msvbhat searches mail
11:10 msvbhat AdrianH: Oh, Live... I'll let you know if I find something :)
11:10 msvbhat Someone else here might know already
11:11 AdrianH msvbhat: great thanks, appreciate your help
11:18 curratore how could i sync a replicate volume connected started and who is replicating the new data but no the olds?
11:19 curratore peer status and volume info are ok
11:19 Manikandan joined #gluster
11:21 NTQ1 joined #gluster
11:21 R0ok_ joined #gluster
11:21 Ulrar SpComb^: I don't know, just thinking. Probably KVM
11:22 vmallika joined #gluster
11:22 NTQ1 Hi. Will it be a problem to create a brick in the root partition? I have a virtual server with only one partition. I want to create the volume in /var/glusterfs/gv0, but gluster is warning me, but I could use force to override this behavior.
11:23 SpComb^ Ulrar: then you should look into how qemu's glusterfs client implements locking
11:23 SpComb^ Ulrar: I have not used the glusterfs stuf, but I ca tell you that eg. the rbd client does no disk image locking at all
11:24 nsoffer joined #gluster
11:24 curratore NTQ1 adding force at end of command I think you can fix that
11:24 SpComb^ Ulrar: it might be something left to the higher-level cluster framework that cotrols kvm
11:24 NTQ1 curratore: I know that I can fix that with "force", but it is really a good idea?
11:25 SpComb^ Ulrar: s/kvm/qemu/
11:25 AdrianH NTQ1: I don't know if it will be a problem but I recommend using Gluster with LVM for the bricks, if you dont you won't be able to grow your bricks.
11:25 curratore NTQ1: I did it myself with a docker container and it worked, but I am still learning, maybe someone could advice about if it is a good idea ;)
11:26 Ulrar SpComb^: But if there is no locking don't have have huge risk of corruption in case of failover ?
11:26 SpComb^ Ulrar: or even libvirt
11:26 SpComb^ Ulrar: yes
11:26 SpComb^ Ulrar: and it bit me, which is why I looked into it :P
11:27 Ulrar I can imagine !
11:27 SpComb^ but I don't have an answer for kvm/qemu/libvirt/glusterfs
11:28 SpComb^ I do know that in kvm/qemu/ganeti/drbd the locking is omplemented by ganeti
11:28 Ulrar Are the performances good ? We do use drbd a lot for other stuff, might be worth a try
11:30 Manikandan joined #gluster
11:31 SpComb^ drbd? It's not the same as a distributed network storage like glusterfs
11:31 smohan joined #gluster
11:33 spalai1 left #gluster
11:34 kdhananjay joined #gluster
11:37 Ulrar SpComb^: Yeah but if the VM file is stored on it, no lock problem
11:37 Ulrar No ?
11:37 swebb joined #gluster
11:39 SpComb^ Ulrar: hmm? Ganeti uses a separate drbd resource on top of LVM for each disk, so the locking is inherent in the drbd master/slave state, no fs involved
11:40 glusterbot News from newglusterbugs: [Bug 1235629] Missing trusted.ec.config xattr for files after heal process <https://bugzilla.redhat.com/show_bug.cgi?id=1235629>
11:41 SpComb^ Ulrar: for migrations, the ganeti master controls the drbd master/slave state on the nodes, and failed nodes require manual failover (there was some STONITH as well, but can't remember what for)
11:42 Ulrar yeah that's what I thought, once the other side is master the original one will just become slave
11:43 Ulrar Looks interesting
11:44 raghu joined #gluster
11:44 SpComb^ Ulrar: you just really don't want to deal with TB drbd volumes, or rather, it's not exactly flexible
11:46 Ulrar I was thinking of using archipel + glusterfs, but ganeti looks like it's worth a try
11:48 AdrianH msvbhat: ok I've got further than before, I didn't see my ansible script didn't add the repo for Gluster3.5 and it installed 3.4 .... That's why I couldn't do this: http://www.gluster.org/community/documentation/index.php/Gluster_3.4:_Brick_Restoration_-_Replace_Crashed_Server
11:51 SpComb^ Ulrar: as I said, I have mot actually used glusterfs for vm block storage, so take my opinion with a grain of salt
11:55 ira joined #gluster
11:56 Ulrar yep, I'll try both and keep the one working best
11:58 zeittunnel joined #gluster
11:58 jcastill1 joined #gluster
11:59 spalai joined #gluster
12:03 jcastillo joined #gluster
12:04 AdrianH So I need to replace a peer, I've followed the : "Brick Restoration - Replace Crashed Server" documentation, everything went ok execpt the last command: " gluster volume heal $vol full" -> fails, when I run "sudo gluster volume status" I can see that the new server is listed but the new (and empty) bricks are offline. Any ideas ?
12:06 elico joined #gluster
12:12 al joined #gluster
12:13 jtux joined #gluster
12:15 rjoseph joined #gluster
12:15 vmallika joined #gluster
12:20 apahim joined #gluster
12:23 TheSeven AdrianH: if I were you I'd be happy that these are still offline ;) In my experience, as soon as they come online, all accesses to the volume that contain them will fail, until the heal has finished (if it ever will). I've failed to see a single successful self-heal so far.
12:24 TheSeven that is with gluster 3.6 and 3.7 on ubuntu and centos, all fails the same way
12:27 rafi joined #gluster
12:30 Folken AdrianH: sudo gluster
12:30 Folken AdrianH: volume heal volumename info
12:30 Folken I'd do that to see what it finds
12:40 AdrianH Folken: (sorry went to eat) just done that and it looks like it is listing all the file I have on Gluster?
12:41 Folken err they are the files it needs to heal I think
12:43 AdrianH Folken: well yeah, I lost a peer that had 2 bricks on it
12:44 AdrianH TheSeven: So what do you recommend, leaving it like this?
12:46 TheSeven AdrianH: so far I have no solution for that. I'm currently evaluating glusterfs, not using it in production yet, so I wasn't forced to fix such a situation yet
12:46 TheSeven but the disaster recovery simulations that I've done so far aren't looking good at all, and I'm considering to abandon glusterfs altogether for that reason
12:47 TheSeven others claim that they couldn't reproduce the problems that I'm seeing, so you might have better luck
12:47 TheSeven guess I'm just ranting a bit here...
12:48 wkf joined #gluster
12:50 husanu1 joined #gluster
12:52 TheSeven AdrianH: so if you want some advice... if this is a critical system for you in any way, set up a test lab to try out the next steps before proceeding any further
12:52 AdrianH TheSeven: GlusterFS has been doing great up to yesterday (AWS instance just died), we've had it in production for 1 year now, serving over 5TB (growing everyday) of data to a live site. Everything is backed up so worst come to the worst, I'll just spawn 4 new servers and put the data on it and scrap the old ones. But it really want to avoid doing t
12:52 AdrianH hat.
12:53 AdrianH TheSeven: that was what I was thinking of doing before starting again. But I don't feel this is a special case, I mean surely this happens all the time...
12:54 TheSeven I'd tend to assume that as well, but what I've seen tells me that any kinds of modifications to the volume structure in gluster, or any missing files on bricks, do cause serious problems even in triple-redundancy setups
12:55 TheSeven from what I've seen, self heal will only heal things on which there have been access attempts before the heal started
12:55 vincent_vdk joined #gluster
12:55 TheSeven these access attempts do of course fail, even already open handles seem to be invalidated
12:56 husanu4 joined #gluster
12:56 firemanxbr joined #gluster
12:56 eljrax TheSeven: What would you abandon gluster for instead?
12:57 TheSeven at least in the case of increasing the replica count, I've seen all other bricks forget about the files that hadn't been healed at that point, effectively killing 99% of the files on the volume
12:57 TheSeven eljrax: that's what i'm still wondering. possibly drbd + some cluster filesystem
12:58 TheSeven eljrax: if licensing cost wouldn't be a factor, I'd probably use windows server 2012 scaleout fileserver
12:58 TheSeven stability-wise that has a much better track record at one job where I worked with it
12:58 TheSeven gluster seems to work as long as everything works, but if something fails... oh well
12:59 TheSeven and if the whole point of using gluster is to compensate those failures, it doesn't really seem suitable for that job
12:59 eljrax Would DRBD + some cluster filesystem self-heal better?
12:59 eljrax I'm in the same situation as you, currently just exploring and not running in production
12:59 AdrianH what about Amazon Elastic File System?
13:00 eljrax And it does seem tricky when you fully lose a brick, like pretend it doesn't exist anymore.. died in a fire or something. I found it quicker to just recreate the volume and copy the files back onto the brick
13:00 TheSeven I'm fairly sure that drbd will heal better, however I'm not sure about the access performance of that, and it doesn't scale as easily as gluster
13:00 eljrax Which I realise isn't always an option if you have TB and TB of data
13:00 hagarth joined #gluster
13:00 TheSeven and, of course, it's a more complex setup with pacemaker etc.
13:01 julim joined #gluster
13:01 eljrax I got stuck in a scenario where I couldn't drop the brick, because it couldn't talk to the brick. And I couldn't add another node, because the replica count was wrong then. So I just started over, which made me a bit nervous.
13:02 TheSeven eljrax: replace-brick force is supposed to work in that case, but I haven't even got around to trying that yet, my glusterfs always self-destroyed in some way before I even got there ;)
13:02 TheSeven my experience so far is that doing an add-brick on a mounted gluster volume is not a safe operation
13:03 TheSeven or rather on a volume that's being used in some way
13:03 eljrax I've had success with that. Never had any problems. Removing a brick, even if healthy works, even though gluster tells me it didn't.
13:03 TheSeven actually you probably have to mount it and do an ls -l $(find /path) or something to even make it replicate
13:03 eljrax So I end up forcing that more often than not :)
13:05 eljrax I think a volume rebalance sorts that
13:05 eljrax You'll still see the old files in an `ls`, but they should be 0 byte 1000 mode DHT pointers
13:06 Pupeno_ joined #gluster
13:07 TheSeven this is what I used to make it eat my data somewhat reproducibly: http://pastie.org/pastes/10225618/text
13:07 TheSeven example output: http://pastie.org/pastes/10225619/text
13:10 theron joined #gluster
13:12 husanux3 joined #gluster
13:14 husanux8 joined #gluster
13:20 husanux2 joined #gluster
13:25 husanux6 joined #gluster
13:26 georgeh-LT2 joined #gluster
13:29 shyam joined #gluster
13:31 aaronott joined #gluster
13:36 Skinny_ good afternoon
13:37 Skinny_ Trying to get geo-replication working. SSH connection and both master and slave volumes are up  and runinng
13:37 elico joined #gluster
13:37 Skinny_ I Run :  gluster volume geo-replication vol01 remote_host:/vol01_dr create
13:37 Skinny_ Invalid Url: remote_host:/vol01_dr
13:37 Skinny_ in the logs : [2015-06-25 13:35:38.629228] E [glusterd-geo-rep.c:4042:glusterd_get_slave_info] 0-: Failed to normalize url
13:37 Skinny_ anyone that has the magic light bulb idea on this ?
13:39 ashiq joined #gluster
13:39 Skinny_ mounting the slave volume on any master node actually works
13:39 Pupeno joined #gluster
13:41 dgandhi joined #gluster
13:43 P0w3r3d joined #gluster
13:45 spalai left #gluster
13:46 curratore joined #gluster
13:49 freaj joined #gluster
13:51 kdhananjay joined #gluster
13:58 RameshN joined #gluster
13:58 elico joined #gluster
14:03 hgowtham joined #gluster
14:04 marbu joined #gluster
14:04 julim joined #gluster
14:13 nbalacha joined #gluster
14:13 hagarth joined #gluster
14:18 Blakey joined #gluster
14:18 mbukatov joined #gluster
14:22 kshlm joined #gluster
14:24 Blakey hey guys, I've setup 2 bricks, which I access from a third machine (mounted via fstab), but, when I halt one of the bricks the third machine cannot access the files on the volume... any ideas? tried googling for a few hours so far and can't find anything relevant...
14:25 mator you and we, need a mind reader on your (bricks) setup
14:27 Blakey https://www.howtoforge.com/how-to-install-glusterfs-with-a-replicated-volume-over-2-nodes-on-ubuntu-14.04  << ive setup the same as that...
14:28 sysconfig joined #gluster
14:29 Blakey here's my fstab entry which is where I suspect the problem is:
14:29 Blakey glusterfs-01:/datapoint /mnt/gluster glusterfs defaults,_netdev,backupvolfile-server=glusterfs-02 0 0
14:30 Ulrar Blakey: I think you are supposed to use a volfile for it to work
14:30 Ulrar Maybe not
14:31 curratore Blakey: have you got the same problem when you stop anyone of bricks or only stopping one of them?
14:32 mator Blakey, is glusterfs-01 server you're stoping brick access?
14:32 Blakey same problem when stopping either of them...
14:33 curratore I think that mator was thinking the same like me :p
14:33 mator you mean if you stop brick on glusterfs-02, there's no access to data on client which have mounted glusterfs-01 ?
14:33 soumya_ joined #gluster
14:34 Blakey yep, when trying to `ls` in the mount point it just locks up until the stopped brick comes back up
14:35 jmarley joined #gluster
14:38 jcastill1 joined #gluster
14:41 glusterbot News from newglusterbugs: [Bug 1203739] Self-heal of sparse image files on 3-way replica "unsparsifies" the image <https://bugzilla.redhat.com/show_bug.cgi?id=1203739>
14:42 mator Blakey, http://ur1.ca/mx6rl
14:42 mator works as expected
14:43 atalur joined #gluster
14:43 Trefex joined #gluster
14:43 mator notice node02 is disconnected (glusterd service stoped)
14:43 jcastillo joined #gluster
14:45 Blakey strange, just working on trying exactly what you did on yours
14:46 Blakey blakey@glusterfs-01:~$ sudo gluster vol info datapoint
14:46 Blakey
14:46 Blakey Volume Name: datapoint
14:46 Blakey Type: Replicate
14:46 Blakey Volume ID: 7e2356e4-20a5-47ac-82af-615d65e9ace8
14:46 Blakey Status: Started
14:46 Blakey Number of Bricks: 1 x 2 = 2
14:46 Blakey Transport-type: tcp
14:46 Blakey Bricks:
14:46 Blakey Brick1: glusterfs-01:/mnt/gluster
14:46 Blakey Brick2: glusterfs-02:/mnt/gluster
14:47 Blakey blakey@glusterfs-01:~$ sudo gluster peer status
14:47 Blakey Number of Peers: 1
14:47 Blakey Hostname: glusterfs-02
14:47 Blakey Uuid: fe97a997-8e91-4e78-95fe-9ded88d1196f
14:47 Blakey State: Peer in Cluster (Disconnected)
14:47 Folken is gluster running on glusterfs-02
14:48 Folken and does glusterfs-02 resolve it's ip correctly
14:48 Blakey all ip's resolve correctly...
14:48 mator Folken, it's a test with replicated volume when one node is down
14:49 Blakey is there a time period where it locks up during failover to the 2nd node (the backup vol file thingy)
14:49 Blakey ?
14:49 mator Blakey, how do you mount again ?
14:49 Blakey as the `ls` now works but it died for a minute or so
14:50 Blakey fstab, glusterfs-01:/datapoint /mnt/gluster glusterfs defaults,_netdev,backupvolfile-server=glusterfs-02 0 0
14:51 Blakey should `glusterfs-01:/datapoint` be `glusterfs-01:/mnt/datapoint`
14:51 Blakey ?
14:51 Folken dont think so
14:52 rjoseph joined #gluster
14:54 Blakey its about 40 seconds before it switches to the other brick, but it appears to be working now for some reason...
14:56 mator Blakey, remount mount poing
14:56 mator point
14:56 mator umount -f /mnt/gluster
14:56 mator mount /mnt/gluster
14:59 Blakey I would but i need it to be automatic, so when a brick crashes or gets taken down it wont lockup the mount on the 3rd machine, or at least not for 40 seconds odd, I'm wanting to store a maildir on it...
15:00 corretico joined #gluster
15:01 NTQ joined #gluster
15:02 chirino joined #gluster
15:03 zeittunnel joined #gluster
15:05 lpabon joined #gluster
15:06 curratore hello guys, I have a strange thing here, anyone knows how to sync a new member on a replica, which is replicating the new data?
15:07 TheSeven you mean increasing the replica count?
15:07 curratore I mean if I write new data it is replicated, but the old stuff is not replicated
15:07 curratore I added 1 new brick
15:07 curratore removed old one
15:07 curratore 1 x 2 bricks on volume again
15:07 curratore status connected 2 members
15:07 TheSeven exactly what I'm experiencing as well... unless I do an ls -l $(find /path) like thing at exactly the right time shortly after the add-brick before selfheal kicks in
15:08 curratore mmm
15:09 chirino joined #gluster
15:09 TheSeven if I didn't do that hack (or not at precisely the right time), gluster seems to lose track of most of the files, not only on the new brick but on all of them
15:10 TheSeven this is what I used to make it eat my data somewhat reproducibly: http://pastie.org/pastes/10225618/text
15:10 TheSeven example output: http://pastie.org/pastes/10225619/text
15:10 TheSeven and I guess that is somewhat similar to what you're doing
15:13 curratore but I have all data (old and new) on master (let's call gfs2) but on gfs3 only the new data
15:14 TheSeven yes, the data still exists in the bricks, it just seems like gluster forgets about that
15:15 curratore is there any other workaround to fix this?
15:16 curratore tried to detach and add after?
15:16 RameshN joined #gluster
15:16 curratore without find in exactly moment trick?
15:17 curratore I googled a lot looking for sth like this but I couldn't find a good solution
15:17 TheSeven I haven't found any so far, and I also haven't found any way to fix up the mess after it happened other than completely deleting and recreating the volume
15:18 NTQ1 joined #gluster
15:18 curratore completely delete? old brick and new?
15:22 jmarley joined #gluster
15:22 curratore any clue of the reason of this problem?
15:26 TheSeven curratore: no, I haven't understood the cause yet, but it's a big problem for me as well
15:26 jcastill1 joined #gluster
15:28 curratore hehe I can delete my volume, I have 7.5 TB inside
15:28 curratore *can't
15:32 jcastillo joined #gluster
15:39 rafi1 joined #gluster
15:43 ppai joined #gluster
15:52 RedW joined #gluster
15:56 kshlm joined #gluster
16:04 cholcombe joined #gluster
16:09 monotek1 joined #gluster
16:22 calavera joined #gluster
16:23 nangthang joined #gluster
16:23 rafi joined #gluster
16:25 curratore TheSeven: I have something different, Status: Transport endpoint is not connected after "gluster volume heal datastore info"
16:25 curratore on my first brick
16:27 jrm16020 joined #gluster
16:31 mribeirodantas joined #gluster
16:37 Blakey cheers for the help guys, laters
16:37 Blakey left #gluster
16:46 anrao joined #gluster
16:52 chirino joined #gluster
16:58 krink joined #gluster
17:01 arao joined #gluster
17:08 curratore joined #gluster
17:08 georgeh-LT2 joined #gluster
17:10 curratore left #gluster
17:12 bennyturns joined #gluster
17:15 rafi joined #gluster
17:18 RameshN joined #gluster
17:28 jbrooks joined #gluster
17:31 mribeirodantas joined #gluster
17:33 jbrooks_ joined #gluster
17:42 victori joined #gluster
17:42 Rapture joined #gluster
18:02 husanu7 joined #gluster
18:28 julim joined #gluster
18:49 chirino joined #gluster
18:55 vimal joined #gluster
18:55 NTQ joined #gluster
18:55 Rapture joined #gluster
18:59 julim joined #gluster
19:03 julim joined #gluster
19:03 chirino joined #gluster
19:07 kenansul- joined #gluster
19:10 DV__ joined #gluster
19:30 chirino joined #gluster
19:49 arao joined #gluster
19:53 Pupeno joined #gluster
19:59 theron_ joined #gluster
20:07 wkf joined #gluster
20:08 Peppard joined #gluster
20:10 Pupeno joined #gluster
20:26 lexi2 joined #gluster
20:27 DV joined #gluster
20:28 DV__ joined #gluster
20:31 chirino joined #gluster
20:38 TheCthulhu3 joined #gluster
20:41 calavera joined #gluster
20:47 julim joined #gluster
20:47 calavera joined #gluster
21:06 prg3 joined #gluster
21:13 paulc_AndChat joined #gluster
21:19 Pupeno joined #gluster
21:20 prg3 joined #gluster
21:22 Pupeno_ joined #gluster
21:25 paulc_AndChat joined #gluster
21:26 badone joined #gluster
21:28 calavera joined #gluster
21:30 Pupeno joined #gluster
21:33 calavera joined #gluster
21:34 Rapture joined #gluster
21:38 mribeirodantas_ joined #gluster
21:47 elico joined #gluster
22:03 smohan joined #gluster
22:03 calavera joined #gluster
22:17 elyograg joined #gluster
22:17 Pupeno joined #gluster
22:18 elyograg sanity check -- "gluster volume delete" does *not* delete any actual data, just the info that allows the disparate bricks to be treated as a single entity, right?
22:20 anrao joined #gluster
22:22 Sjors joined #gluster
22:22 kaushal_ joined #gluster
22:26 NTQ Which ports do I need to open in my firewall when using glusterfs?
22:27 ndevos ~ports | NTQ
22:27 glusterbot NTQ: glusterd's management port is 24007/tcp (also 24008/tcp if you use rdma). Bricks (glusterfsd) use 49152 & up since 3.4.0 (24009 & up previously). (Deleted volumes do not reset this counter.) Additionally it will listen on 38465-38467/tcp for nfs, also 38468 for NLM since 3.3.0. NFS also depends on rpcbind/portmap on port 111 and 2049 since 3.4.
22:31 NTQ I was just going though this quick start guide and had only port 24007 open. While creating a 10 MB file I lost ssh connection to my server and now it is no more "pingable". I guess the server has just crashed.
22:31 NTQ http://gluster.readthedocs.org/en/latest/Quick-Start-Guide/Quickstart/
22:39 NTQ What's about ports 832? There is glusterfs process listening on that port.
22:40 gildub joined #gluster
22:41 elyograg what is the commandline for that specific process?  I don't see 832 on mine.
22:44 elyograg there is a process listening on port 637.  it's the nfs process.
22:44 NTQ I just did 'netstat -tlnp'. Process 'glusterfs' is listening on ports 832, 2049, 38465, 38466, 38468 and 38469. The command line is "/usr/sbin/glusterfs -s localhost --volfile-id gluster/nfs -p /var/lib/glusterd/nfs/run/nfs.pid -l /var/log/glusterfs/nfs.log -S /var/run/xxxxx98a334e02d17d0b2aa6cc555ad6.socket"
22:44 natarej_ joined #gluster
22:45 elyograg yep, tht's the nfs process.  no idea why it needs a low port, or why your low port is different than mi9ne.
22:45 elyograg checked another server, and the low port is different again -- 806.
22:46 NTQ Yes, on the other server it is 717
22:47 elyograg i don't know why it opens a low port like that, or why the number is seemingly random.
22:47 NTQ I want to have static ports. Is this possible? Otherwise I can not configure the firewall.
22:48 Pupeno joined #gluster
22:48 Leildin joined #gluster
22:51 NTQ Do I need to open all these ports?
22:52 elyograg are you accessing gluster over nfs?  if not, that port won't matter.
22:52 elyograg if you are, then I don't know for sure whether it will matter.
22:52 dusmant joined #gluster
22:56 NTQ I am not sure if I am using nfs. :D These are my first steps with glusterfs
22:56 NTQ Do you mean the nfs filesystem? I am using ext4.
22:57 elyograg there are two main ways that you mount a gluster filesystem.  one is with the "native" fuse client, the other is nfs.
23:00 NTQ Okay. I seems that my 10 MB file I created on the first server was not transfered completely to the second server. It has only 7,6 MB on the other side. How can I synchronize the replicated volumes?
23:02 NTQ I mounted the the filesystem this way: 'mount -t glusterfs server1:/gv0 /mnt'
23:03 elyograg that's the native type.
23:03 elyograg using FUSE.
23:05 plarsen joined #gluster
23:08 NTQ After stopping and starting the volume it seems that all files are the same on all bricks
23:20 calavera joined #gluster
23:54 Debloper joined #gluster

| Channels | #gluster index | Today | | Search | Google Search | Plain-Text | summary