Perl 6 - the future is here, just unevenly distributed

IRC log for #gluster, 2013-12-18

| Channels | #gluster index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:02 Gilbs1 After a rebooting and brick replacement I'm running into :glusterd dead but pid file exists  --  Is it ok to delete the glusterd.pid in /var/run?
00:05 Gilbs1 Update -- I moved the file, restarted glusterd and am still stuck with: glusterd dead but pid file exists      Any ideas?  (looking at logs now)
00:05 MugginsM joined #gluster
00:07 MugginsM yay spending the day fighting with gluster :-/
00:11 JoeJulian Gilbs1: "glusterd --debug" is usually very helpful and quicker than scouring the logs when that kind of stuff happens.
00:12 Gilbs1 Ah -- input in flex scanner failed
00:13 mkzero Wow. Those processes seam to spawn DOA now. Rebooted a machine and it already got 20 webserver procs in D state. after a bit of digging logs it seems to be some kind of problem with the non-blocking self-heal happening/failing on those files
00:14 Gilbs1 is gluster's flux capacitor upset?
00:24 MugginsM so if my gluster servers are logging a "0-management: connection attempt failed (Connection refused)"  every 3 seconds, how should I try to work out what it's trying to connect to?
00:24 MugginsM just upgraded from 3.3 to 3.4.1
00:25 JoeJulian I would probably use strace: strace -f -e trace=network -p $(pidof glusterd)
00:25 JoeJulian Would be nice if it was more verbose about that though.
00:26 MugginsM oh good plan. it's a socket by the look.         /var/run/58597292e1a02d923f68f1f771d4ea31.socket
00:27 MugginsM lsof shows nothing has it open
00:28 MugginsM both servers (2 pair replica) independently failing on similar
00:30 asku joined #gluster
00:30 MugginsM mmm, wonder if it's  https://bugzilla.redhat.com/show_bug.cgi?id=976750
00:30 glusterbot Bug 976750: low, medium, ---, vagarwal, CLOSED CURRENTRELEASE, Disabling NFS causes E level errors in nfs.log.
00:33 Gilbs1 100% /  o_O;     There we go...
00:46 Gilbs1 log files gone wild -- deleted and glusterd back up.  Thanks JJ
00:47 JoeJulian Stupid log files...
00:48 Gilbs1 thought I would be ordering chinese and sleeping in the office again.
00:50 gdubreui joined #gluster
00:50 Gilbs1 will "gluster log rotate" rotate glustershd.log as well?
00:54 JoeJulian Dunno.... I don't use that, I use logrotate with copytruncate and compress.
01:08 theron joined #gluster
01:08 theron_ joined #gluster
01:15 psyl0n joined #gluster
01:34 bala joined #gluster
01:37 harish joined #gluster
01:41 flrichar joined #gluster
01:44 jiqiren joined #gluster
02:34 vpshastry joined #gluster
02:37 shubhendu joined #gluster
03:00 gdubreui joined #gluster
03:09 saurabh joined #gluster
03:19 kshlm joined #gluster
03:23 satheesh joined #gluster
03:41 bharata-rao joined #gluster
04:16 raghu joined #gluster
04:20 itisravi joined #gluster
04:21 yosafbridge joined #gluster
04:28 kdhananjay joined #gluster
04:29 raghug joined #gluster
04:58 lalatenduM joined #gluster
05:00 MiteshShah joined #gluster
05:10 DV_ joined #gluster
05:19 ndarshan joined #gluster
05:20 kanagaraj joined #gluster
05:23 prasanth joined #gluster
05:23 shylesh joined #gluster
05:23 nshaikh joined #gluster
05:25 mohankumar joined #gluster
05:31 psharma joined #gluster
05:34 ppai joined #gluster
05:43 aravindavk joined #gluster
05:49 ngoswami joined #gluster
05:56 badone joined #gluster
06:09 vpshastry joined #gluster
06:10 zwu joined #gluster
06:13 glusterbot New news from newglusterbugs: [Bug 1040275] Stopping/Starting a Gluster volume resets ownership <https://bugzilla.redhat.com/show_bug.cgi?id=1040275>
06:18 theron joined #gluster
06:23 meghanam joined #gluster
06:28 CheRi joined #gluster
06:29 hagarth joined #gluster
06:31 zeittunnel joined #gluster
06:45 overclk joined #gluster
06:54 hagarth joined #gluster
07:02 harish joined #gluster
07:13 glusterbot New news from newglusterbugs: [Bug 1044337] Error while executing action CommitRemoveGlusterVolumeBricks: Command execution failed return code: 146 <https://bugzilla.redhat.com/show_bug.cgi?id=1044337>
07:16 pravka joined #gluster
07:17 KORG joined #gluster
07:22 jtux joined #gluster
07:25 anands joined #gluster
07:28 bala joined #gluster
07:43 shapemaker joined #gluster
07:44 dneary joined #gluster
07:46 morse joined #gluster
07:50 ricky-ticky joined #gluster
07:51 morse joined #gluster
07:56 vpshastry1 joined #gluster
07:58 sankey what does it mean "failed to construct the graph" ?
07:58 sankey here's some basic information about my configuration: https://linux.ucla.edu/paste/ASPlv
08:00 morse joined #gluster
08:01 ctria joined #gluster
08:01 sankey i'm trying to mount a glusterfs volume, just like in the quickstart tutorial: http://www.gluster.org/community/documentation/index.php/QuickStart
08:01 glusterbot Title: QuickStart - GlusterDocumentation (at www.gluster.org)
08:04 morse joined #gluster
08:09 ndevos @later tell social I'm not really sure if a fuse-mount should fsync() on close(). After re-thinking, I don't think NFS does that either and the Linux-VFS is responsible for flushing dirty pages, latest on an umount.
08:09 glusterbot ndevos: The operation succeeded.
08:09 samppah ndevos: hmm, what are you referring to?
08:14 glusterbot New news from newglusterbugs: [Bug 1044352] [RFE] Exempting a list of client IPs or the RHS servers themselves from anonymous uid and gid feature and/or from root squashing <https://bugzilla.redhat.com/show_bug.cgi?id=1044352>
08:15 eseyman joined #gluster
08:17 ndevos samppah: social noticed yesterday that some files were inconsistent, his app closed a file, and the contents were not available to the other clients mounting the same volume
08:18 ndevos samppah: he was looking into adding fsync() before close() in the fuse-client, but I think an app should call fsync() itself (or mount with -osync)
08:18 samppah ndevos: ah, okay
08:21 tryggvil joined #gluster
08:24 jiqiren joined #gluster
08:27 itisravi joined #gluster
08:32 jag3773 joined #gluster
08:33 purpleidea ndevos: hey you still around?
08:33 solid_liq joined #gluster
08:33 solid_liq joined #gluster
08:34 ndevos purpleidea: still? its only 09:33 here
08:34 * ndevos is in .nl, near Amsterdam
08:34 purpleidea ndevos: ah, morning from Canada then. (3:34am)
08:36 ndevos purpleidea: thanks, and good night to you!
08:36 purpleidea ndevos: just fyi, in case it helps you out. i've been hacking on my puppet-gluster+vagrant thing, and in the end i made base images instead of using stock virtualbox ones which weren't working properly with libvirt...
08:36 purpleidea anyways, in the end i made them with virt-builder, and if you need to customize images based on that, you're welcome to jump off from there...
08:38 purpleidea i also made rpms for virt-builder on F19 because i'm still on that if you want them for your needs.
08:40 purpleidea ndevos: ^^^
08:42 rastar joined #gluster
08:44 ndevos purpleidea: oh, cool! I dont need the virt-builder rpms, but I'd like to check those images
08:47 purpleidea ndevos: currently i'm doing centos6.5, naked images, which are then provisioned with puppet of course. they're "insecure" and intended for vagrant usage, but if they fit your bill you could customize based on that if you wanted.
08:48 purpleidea (currently uploading one for you to download, please stand by - slow ass upload in canada... cables must be frozen)
08:48 hybrid5121 joined #gluster
08:49 ndevos purpleidea: that fits me very well!
08:49 ndevos purpleidea: what modification have you made, compared to the plain virt-builder ones?
08:50 purpleidea ndevos: lots basically... all the "prep" stuff that vagrant needs... i figured it would be more useful to use my rpm's (those were a pita to get together for F19)
08:51 purpleidea s/more useful to/more useful for you to/
08:51 glusterbot What purpleidea meant to say was: ndevos: lots basically... all the "prep" stuff that vagrant needs... i figured it would be more useful for you to use my rpm's (those were a pita to get together for F19)
08:52 purpleidea thanks glusterbot
08:52 purpleidea glusterbot: thanks
08:52 glusterbot purpleidea: you're welcome
08:52 ndevos purpleidea: hmm, not sure if those modification are needed for very simple testing...
08:53 purpleidea ndevos: maybe not! figured i'd mention it anyways. it'll all be blogged or github-ed soon so you can pick and choose then if you like.
08:53 ndevos purpleidea: also, I have access to hardware that I can provision with f20, at least temporary, so I can use virt-builder from f20 :)
08:53 ndevos purpleidea: thanks for letting me know, I'll surely follow your posts and progress!
08:54 purpleidea ndevos: awesome. F20 is out now! but i was working on this before that, so :P oh well. learned a bit i guess
09:00 purpleidea ndevos: is there a meeting today?
09:00 mbukatov joined #gluster
09:00 ndevos purpleidea: I assume so, but hagarth should know for sure
09:01 ndevos hagarth: is there a meeting later today?
09:01 * ndevos guesses he's having lunch
09:01 purpleidea ndevos: anyways, if you need the rpms: http://download.gluster.org/pub/gluster/purpleidea/virt-builder/f19/rpm/
09:01 glusterbot Title: Index of /pub/gluster/purpleidea/virt-builder/f19/rpm (at download.gluster.org)
09:02 ndevos purpleidea: thanks!
09:02 mgebbe_ joined #gluster
09:08 tqrst- joined #gluster
09:11 KORG|2 joined #gluster
09:12 PatNarciso joined #gluster
09:12 mikedep333 joined #gluster
09:12 delhage joined #gluster
09:12 sac`away joined #gluster
09:12 mattf joined #gluster
09:12 mkzero joined #gluster
09:12 johnmwilliams joined #gluster
09:12 Kins joined #gluster
09:12 VerboEse joined #gluster
09:12 fkautz joined #gluster
09:12 al joined #gluster
09:12 uebera|| joined #gluster
09:14 glusterbot New news from newglusterbugs: [Bug 969461] RFE: Quota fixes <https://bugzilla.redhat.com/show_bug.cgi?id=969461>
09:16 vimal joined #gluster
09:16 shubhendu joined #gluster
09:17 raghu joined #gluster
09:20 stickyboy joined #gluster
09:21 tryggvil joined #gluster
09:21 PatNarciso joined #gluster
09:21 mikedep333 joined #gluster
09:21 delhage joined #gluster
09:21 sac`away joined #gluster
09:21 mattf joined #gluster
09:21 mkzero joined #gluster
09:21 johnmwilliams joined #gluster
09:21 Kins joined #gluster
09:21 VerboEse joined #gluster
09:21 fkautz joined #gluster
09:21 al joined #gluster
09:21 uebera|| joined #gluster
09:29 ngoswami joined #gluster
09:49 overclk joined #gluster
09:50 badone joined #gluster
09:52 dneary joined #gluster
09:54 andreask joined #gluster
10:04 MiteshShah joined #gluster
10:07 jag3773 joined #gluster
10:08 ctrianta joined #gluster
10:28 ngoswami joined #gluster
10:33 hybrid512 joined #gluster
11:02 neofob joined #gluster
11:10 nullck joined #gluster
11:13 calum_ joined #gluster
11:22 fen_ joined #gluster
11:36 pureflex joined #gluster
11:38 hagarth ndevos: yes, the meeting is on later today. Just updated the agenda
11:39 ndevos hagarth: do you send a reminder to the mailinglists as well each week? that may attract more people
11:44 hagarth ndevos: sounds like a good idea. I sent a recurring invite earlier.
11:45 hagarth will send out with a link to the agenda
11:45 ndevos hagarth: yes, but the recurring event is only good for the people that add it to their calendar - I bet a lot will miss it
11:50 ppai joined #gluster
11:53 hagarth ndevos: agree, added a note in my calendar to send out pointers to agenda every wednesday
11:54 ctrianta joined #gluster
11:54 l0uis is there a recommended way to stop client glusterfs processes? after unmounting the fs, can they just be killed?
12:08 yinyin joined #gluster
12:10 andreask joined #gluster
12:15 CheRi joined #gluster
12:20 hagarth joined #gluster
12:21 ppai joined #gluster
12:22 Remco left #gluster
12:36 itisravi joined #gluster
13:03 bfoster joined #gluster
13:09 partner trying to remount with -o remount says "according to mtab, GlusterFS is already mounted on /mnt" - is there a way to really remount or do i need to umount+mount?
13:10 partner i seem to have 3.3.1 client here (with 3.3.2 on the server end)
13:11 partner and debian squeeze. googling around i've seen few similar questions without replies
13:13 zeittunnel joined #gluster
13:26 partner it seems the vim temporary files caused the stuff to pile up, lsof reports (deleted) (stat: Structure needs cleaning) and for each of those the load goes up by 1.00
13:37 diegows joined #gluster
13:40 davidbierce joined #gluster
13:45 jiphex_ joined #gluster
13:48 blook joined #gluster
13:51 B21956 joined #gluster
13:54 B21956 joined #gluster
13:58 psyl0n joined #gluster
14:08 bennyturns joined #gluster
14:11 sroy_ joined #gluster
14:24 Norky joined #gluster
14:26 Norky left #gluster
14:30 harish joined #gluster
14:31 dbruhn joined #gluster
14:35 vpshastry1 left #gluster
14:37 dbruhn joined #gluster
14:47 andreask joined #gluster
14:49 zaitcev joined #gluster
14:50 aixsyd joined #gluster
14:50 hagarth joined #gluster
14:50 aixsyd hey guys - I need some quick help with setting up a gluster system
14:50 dbruhn ok?
14:51 aixsyd i have two servers - both have a RAID1 500gb OS drive, and each have two 2TB drives. both 2TB drives are in a single drive RAID0 (PERC6i card wants em that way for indivule use)
14:51 wushudoin joined #gluster
14:52 aixsyd I'm looking to do a distributed stipe volume - where essentially each 2TB drive is a brick
14:52 jseeley_vit joined #gluster
14:52 aixsyd two bricks per server, two servers
14:52 itisravi joined #gluster
14:53 dbruhn ok? So... what's the question.
14:53 aixsyd none of the doc syntax addresses this scenario
14:54 aixsyd https://access.redhat.com/site/documentation/en-US/Red_Hat_Storage/2.0/html/Administration_Guide/sect-User_Guide-Setting_Volumes-Striped_Replicated.html
14:54 glusterbot Title: 8.7. Creating Striped Replicated Volumes (at access.redhat.com)
14:54 aixsyd this is the closest, but showing four servers
14:54 kdhananjay joined #gluster
14:54 dbruhn Yep, the syntax is the same though
14:55 aixsyd so i'd only liste the two servers instead of the 4 as in the example?
14:55 aixsyd *list
14:55 dbruhn Gluster doesn't really care about the servers
14:55 dbruhn it cares about the bricks.
14:55 shubhendu joined #gluster
14:55 aixsyd ahh
14:56 dbruhn so you just have your server1:/brick1 server1:/brick2 server2:/brick3 server2:/brick4
14:56 aixsyd gotcha gotcha - another question - i plan to do two sets of two clusters - would it be better to just group all four servers into one cluster than seperating them?
14:57 aixsyd thats 8 bricks of 2Tb, 4 servers
14:57 dbruhn Not sure what you're use case, what are you trying accomplish?
14:57 aixsyd i need 8tb total
14:57 aixsyd essentially running VM disks from the cluster
14:57 ira joined #gluster
14:57 aixsyd via Proxmox
14:58 dbruhn why did you want to break them up into two clusters?
14:58 aixsyd were replacing two NAS devices right now, ones a VM storage NAS, ones a file server NAS - so it was really a logical break up
14:58 pk joined #gluster
14:59 aixsyd im creating a new VM to replace the file server NAS, the disk will live on the cluster
14:59 dbruhn If it's logical in your use case I can't see why you wouldn't want to then
14:59 hagarth community meeting about to start in #gluster-meeting
15:00 aixsyd okay. thought as much
15:00 dbruhn How many vm's are you hosting on this system?
15:00 aixsyd one sec
15:00 badone joined #gluster
15:00 aixsyd 12 on one, 1 on the other
15:01 aixsyd 12 on one cluster, i should say, one on the other cluster
15:01 jseeley_vit Hello. I have a question about the mount option backup-volfile-servers. I'm using gluster-3.4 RPMs from 'download.gluster.org EPEL' repository. I keep getting "/usr/bin/fusermount-glusterfs: mount failed: Invalid argument" when I use this option. 1.) anyone know what's going on and 2.) is this option even necessary?
15:01 aixsyd right now, theyre all running on a RAID10 QNAP NAS.
15:01 dbruhn Seems like a low disk count for I/O for 12 vm's, obviously I am totally not clued into the usage characteristics
15:02 aixsyd limitation of funding, unfortunately...
15:02 kaptk2 joined #gluster
15:02 aixsyd i would have liked 4x 2TBs in each server, two sets of RAID1's.
15:08 ricky-ticky joined #gluster
15:08 aixsyd dbruhn: my main worry now is how data could be lost upon a drive failure
15:08 aixsyd according to this graphic: https://access.redhat.com/site/documentation/en-US/Red_Hat_Storage/2.0/html/Administration_Guide/images/Striped_Replicated_Volume.png
15:09 aixsyd Bricks 3 and 4 will be on server 2 of 2
15:09 aixsyd what happens if server 2 goes offline?
15:10 aixsyd i'd really like it to stripe within each server, then replicate that to server 2
15:10 bala joined #gluster
15:11 dbruhn You said you were looking for distributed and striped before?
15:11 aixsyd well, it seemed to fit the bill the closest
15:11 dbruhn why are you wanting to use striped?
15:12 aixsyd well... hmm
15:13 aixsyd essentially, i need 4TB usable, across two servers. theres 8TB worth of drives. I want to still be up and live if one physical server goes down
15:14 jseeley_vit For the record, I just tried the earlier version of the command "backupvolfile-server" (vs backup-volfile-servers) and that worked.
15:14 aixsyd would it be better to simply RAID0 each 2TB drive on a server and have a giant 4TB brick on each server, and just replicate?
15:14 bugs_ joined #gluster
15:15 aixsyd like a network level RAID 10
15:15 dneary joined #gluster
15:15 aixsyd that way, if one drive fails, yes, the whole server/RAID fails, but the second server takes over right away
15:17 aixsyd I think thats how it'll be done.
15:18 nocturn joined #gluster
15:18 aixsyd any updates on rdma/infiniband with GlusterFS?
15:19 nocturn Hi, I get an error mount my glusterfs volume after the client rebooted.  OS is Scientific Linux (RHEL) 6, the error is  0-mgmt: failed to fetch volume file (key:/labfs)
15:20 spechal joined #gluster
15:21 spechal I seem to be having an issue with CentOS 5 and the Gluster-EPEL repo ... http://fpaste.org/62903/13873800/ ... could someone provide me with guidance?
15:21 glusterbot Title: #62903 Fedora Project Pastebin (at fpaste.org)
15:22 aixsyd shit, wait a sec, that doesnt work, dbruhn - if its a hardware level RAID0 on each - and a drive fails, how would the raid be rebuilt? all 4TB would have to be resync'd, no?
15:25 jseeley_vit spechal: Try using this repo instead: http://download.gluster.org/pub/gluster/glusterfs/3.4/3.4.0/EPEL.repo/glusterfs-epel.repo
15:25 spechal never mind, I got it by removing the no arch repo as it doesn't exist
15:26 spechal [glusterfs-noarch-epel] doesn't exist from what I can tell
15:26 dbruhn aixsyd, sorry phone. Your idea of a "Raid10" isn't a bad idea, on the RDMA front still flakey, I would suggest running tcp/ip on your IB for now
15:26 spechal yeah, http://download.gluster.org/pub/gluster/glusterfs/3.4/3.4.1/EPEL.repo/epel-5/ ... no noarch
15:26 glusterbot Title: Index of /pub/gluster/glusterfs/3.4/3.4.1/EPEL.repo/epel-5 (at download.gluster.org)
15:27 dbruhn Yes it would have to be re-synched. Personally I run raid with protection on my systems, it's much easier to rebuild a raid than a brick right now
15:27 nocturn Server says: rpcsvc-auth.c:324:rpcsvc_authenticate] 0-rpc-service: Auth too wea
15:33 spechal Could someone please help me troubleshoot a failed mount?  It claims there is no fuse ... http://fpaste.org/62907/38738079/
15:33 glusterbot Title: #62907 Fedora Project Pastebin (at fpaste.org)
15:34 spechal This is on CentOS5 ... I have no issues doing the same thing on CentOS6
15:34 plarsen joined #gluster
15:35 dbruhn special, what version of server are you working against?
15:36 spechal 3.4.1-3.el6
15:36 lpabon joined #gluster
15:37 spechal There is in fact, no /dev/fuse on the CentOS5 box
15:38 spechal dbruhn: http://fpaste.org/62910/73811221/
15:38 glusterbot Title: #62910 Fedora Project Pastebin (at fpaste.org)
15:39 spechal I am trying to mount the el5 box to the el6 box
15:40 aixsyd dbruhn: i think i can make the case for 4x 2tbs in each server. ill steal 4x 2Tbs from cluster 2 and configure cluster 1 with 8x 2tbs ;)
15:41 dbruhn special, do you have fuse installed on the cent5 box?
15:42 aixsyd so that way if a drive fails on either server, everythings still okay - if two drives fail, the whole server fails, but still okay due to another completely OK server.
15:43 spechal Yes, glusterfs-fuse.x86_64                    3.4.1-3.el5                   installed ... but /dev/fuse does not exist
15:43 spechal I will try removing and reinstalling
15:43 ndevos spechal: you have to load the fuse module too
15:43 dbruhn special glusterfs-fuse and fuse are two different thing
15:43 dbruhn https://www.centos.org/forums/viewtopic.php?t=19042
15:43 glusterbot Title: CentOS View topic - Installing fuse on CentOS 5 (at www.centos.org)
15:44 spechal doh, I knew that and forgot
15:44 dbruhn aixsyd, sounds like you have a direction
15:44 ndevos spechal: there should be a file under /etc/sysconfig/modules.d/ , I think, that loads the module on boot
15:44 spechal and of course after I install fuse everything is great
15:44 spechal Thank you both
15:45 Gilbs1 left #gluster
15:45 dbruhn np, glad it worked out
15:45 l0uis are there any known dead lock cases w/ gluster? i'm running into a situation that appears to dead lock the client under heavy write/read load
15:46 l0uis still trying to debug what's happening, but it appears to be a write simply isn't completed, thus stalling the pipeline in my program
15:47 bstromski joined #gluster
15:47 l0uis and further, how would I go about debugging? just strace? anything else?
15:50 nueces joined #gluster
15:51 aixsyd dbruhn: if im using four drives in each, why not RAID10 at the hardware level, instead of 2x RAID1's
15:51 aixsyd then a simple replicate
15:52 jseeley_vit What's the best way to minimize or eliminate the time it takes for a client connected to a 2-node replicate cluster to connect to the other node in the cluster when the initial node the client is connected to goes down? For example, I have client1 connected to node1 through the native client. If node1 goes down, it takes a very noticeable amount of time for client1 to then connect to node2 since node1 went down.
15:53 nueces joined #gluster
15:53 aixsyd jseeley_vit: I'd like to know this, too
15:53 dbruhn aixsyd a raid10 on each server+ a replication would mean 4 copies of your data
15:53 aixsyd Yep. totally cool with that.
15:54 aixsyd the more copies, the better, says management
15:54 dbruhn hahah, until they have to pay for it.
15:55 aixsyd ino, rite?
15:56 jseeley_vit joined #gluster
15:57 aixsyd dbruhn: i figure the performance will be pretty decent from a RAID10, too
16:03 dbruhn RAID10 does provide a nice read boost for sure. Can take a bit of a hit on write depending on the controller used.
16:06 dbruhn l0uis, have you increased logging to debug level logging?
16:07 l0uis dbruhn: yes... unfortunately it didn't seem to happen when I had logging enabled. i let the program run about 5x as long as it takes for the dead lock to happen w/ logs off
16:08 l0uis dbruhn: not that it *won't* happen w/ logging on, just didn't see it
16:08 l0uis dbruhn: i saw a bug report about auxillary groups causing a dead lock. i'm trying w/ the suggested work around of diabling that w/ group timeout of -1
16:08 dbruhn Just a suggestion on something to look at.
16:09 xymox joined #gluster
16:10 l0uis it does happen if i enable the fuse state dump, but it takes longer than normal.
16:10 dbruhn jseeley_vit, how long are we talking? like 42 seconds?
16:11 jseeley_vit dbruhn: exactly, at least that's whats in the logs: Dec 18 11:05:25 centos-02 GlusterFS[17952]: [2013-12-18 16:05:25.065997] C [client-handshake.c:127:rpc_client_ping_timer_expired] 0-gv0-client-0: server 10.0.2.11:49152 has not responded in the last 42 seconds, disconnecting.
16:11 dbruhn http://joejulian.name/blog/keeping-your-vms-from-going-read-only-when-encountering-a-ping-timeout-in-glusterfs/
16:11 glusterbot Title: Keeping your VMs from going read-only when encountering a ping-timeout in GlusterFS (at joejulian.name)
16:11 dbruhn this blog entry will probably help you out
16:15 jseeley_vit dbruhn: Thank you for that blog post. This leads me to believe what's the harm in shortening the value of the timeout. I see it says "This is important because re-establishing FDs and locks can be a very expensive operation" though I really don't understand what the "expense" is.
16:16 dbruhn Processing, bandwidth, etc.
16:17 dbruhn What he is saying in that blog entry is if something were to respond after your time out setting, it would default to having to reconnecting, and going through all of the motions, if you shorten the value to much you will end up doing that a lot more, which could cause the system to do unneeded processes like establishing FD's and locks
16:18 dbruhn so it's a trade off
16:19 semiosis joined #gluster
16:19 jseeley_vit ah, ok.
16:21 jseeley_vit 42 seconds just seems too high in a simple replica setup As this person puts it, http://gluster.org/pipermail/gluster-users/2011-April/007242.html, "I would expect gluster to transparently fail in distributed replica? But it looks like if you lose a node you end having high response time which is determinental to performance when under high load and a gluster server crashes."
16:23 muhh joined #gluster
16:23 dbruhn Totally understand what you are saying, and that's why the option to tune it is in there. 42 seconds may seem unreasonable, until you end up with a 1GB pipe that saturated and you end up actually needing that 42 seconds for communication.
16:25 theron joined #gluster
16:26 semiosis joined #gluster
16:26 semiosis joined #gluster
16:29 FarbrorLeon joined #gluster
16:30 l0uis dbruhn: so far, disabling the auxillary groups by setting the gid timeout to -1 seems to help. hasn't locked up yet, but hasn't completed either. about 2x as far than normal lockup time.
16:39 JoeJulian jseeley_vit: If your servers are crashing so often that the 42 seconds is a problem, you have other problems.
16:40 jag3773 joined #gluster
16:45 aixsyd JoeJulian: what if youre running VM disks from the cluster? 42 secs of downtime will crash any/all VMs attached
16:50 zerick joined #gluster
16:52 l0uis sigh, well scratch that. just deadlocked again
16:54 l0uis I've got one gluster thread that is waiting on a request_wait() inside of fuse_dev_do_read()
16:55 l0uis Got another waiting on a pip
17:14 RicardoSSP joined #gluster
17:17 sroy_ joined #gluster
17:18 davidbierce joined #gluster
17:24 jbd1 joined #gluster
17:28 thogue joined #gluster
17:29 * semiosis waves to thogue
17:36 Mo_ joined #gluster
17:36 psyl0n joined #gluster
17:40 aixsyd infiniband - whats the difference between UD and CM modes?
17:51 _VerboEse joined #gluster
17:52 jseeley_vit JoeJulian: I agree with you that if servers are crashing that often, there are bigger problems. I am currently in the process of redesigning a system that uses 2 servers in a active/passive configuration where data is rsynced from one server to another. This system gets deployed in various client environments that do not always have redundant power feeds or if they do, they don't always hold up as they should.
17:54 sac`away` joined #gluster
17:55 l0uis Does anyone have any advice as to how to debug an apparent hang/deadlock in the gluster client?
17:55 mattf` joined #gluster
17:57 aixsyd which repos should I use for glusterfs 3.4.x for a debian server?
18:00 semiosis ,,(latest)
18:00 glusterbot The latest version is available at http://download.gluster.org/pub/gluster/glusterfs/LATEST/ . There is a .repo file for yum or see @ppa for ubuntu.
18:01 semiosis well, the LATEST symlink tracks the latest version
18:01 aixsyd :D
18:01 semiosis you probably want to use a specific version, rather than the variable.  how about http://download.gluster.org/pub/gluster/glusterfs/3.4/3.4.1/Debian/
18:01 glusterbot Title: Index of /pub/gluster/glusterfs/3.4/3.4.1/Debian (at download.gluster.org)
18:02 semiosis so you dont get upgraded accidentally
18:02 al joined #gluster
18:02 zeittunnel joined #gluster
18:03 lalatenduM joined #gluster
18:04 theron joined #gluster
18:09 mattf joined #gluster
18:10 nueces joined #gluster
18:12 jseeley_vit Is there any reason I should NOT mount a volume using the native client on the same server that is serving the volume?
18:13 PatNarciso joined #gluster
18:13 mikedep333 joined #gluster
18:13 delhage joined #gluster
18:13 mkzero joined #gluster
18:13 johnmwilliams joined #gluster
18:13 Kins joined #gluster
18:13 fkautz joined #gluster
18:13 uebera|| joined #gluster
18:13 semiosis jseeley_vit: it's really the only choice for a localhost mount.  nfs localhost mounts are dangerous & should be avoided.
18:14 semiosis jseeley_vit: however, be aware that split-brain is made possible by localhost mounts.  for example, say two servers also have client mounts, and they become disconnected from each other.  if the same file is written through both clients, that file is now split brained
18:16 aixsyd http://nawrot.psych.ndsu.nodak.edu/Courses/465Projects05/splitbrain/splitbrain4_files/image001.gif
18:16 jseeley_vit semiosis: Does that split-brain scenario apply to native localhost mounts as well?
18:17 semiosis jseeley_vit: native localhost mount is all i'm talking about
18:17 jseeley_vit ah
18:17 semiosis it's the only kind of localhost mount supported by gluster
18:17 semiosis nfs localhost mount is dangerous, not recommended
18:19 FarbrorLeon joined #gluster
18:21 semiosis using quorum on your volume can prevent this kind of split-brain
18:21 semiosis I'd recommended using replica 3 + quorum if you need localhost mounts
18:22 jseeley_vit semiosis: Thanks. NFS mounts were already probably not going to be an option for me, at least didn't look like I needed it. It appears the native mount will work just fine however my application will run on the same server(s) that run on the server(s) that host the gluster volumes
18:22 semiosis alternatively, if you can get away with having the localhost mounts read-only, that might be an option
18:22 semiosis although there was a bug which broke read-only mounts, let me check the status of that (been a while)
18:23 [o__o] joined #gluster
18:27 semiosis @read only
18:27 glusterbot semiosis: I do not know about 'read only', but I do know about these similar topics: 'read-only'
18:27 semiosis @read-only
18:27 glusterbot semiosis: read-only is broken in 3.3 -- https://bugzilla.redhat.com/show_bug.cgi?id=853895
18:27 semiosis glad we made that factoid, because i can never find what i'm looking for with bugzilla search
18:28 semiosis according to that bug, 3.4.0+ should support read only mounts (-o ro) again
18:28 badone joined #gluster
18:32 jseeley_vit Thanks. I don't think readonly mounts are an option. Since it seems like it's going to be a requirement for me to run an application on the same server that hosts the GlusterFS volume, the application will receive data and need to write that data to the GlusterFS volume.
18:37 semiosis then you definitely want to use quorum
18:37 semiosis and replica 3
18:37 semiosis with replica 2 the mounts will turn read only when one of the servers goes down
18:40 the-me joined #gluster
18:40 jseeley_vit ok - dumb question but I'm having trouble putting together what replica means. I understand what a brick is but not replica. Take for example a replica 2 vs replica 3 - what does that mean?
18:42 aixsyd anyone know why my directly connected 802.3ad bonding isnt working/starting? http://fpaste.org/62964/87391956/
18:42 glusterbot Title: #62964 Fedora Project Pastebin (at fpaste.org)
18:43 dbruhn joined #gluster
18:44 aixsyd dbruhn: know anything aboout 802.3ad?
18:45 flrichar joined #gluster
18:46 glusterbot New news from newglusterbugs: [Bug 1044648] Documentation bug for glfs_set_volfile_server <https://bugzilla.redhat.com/show_bug.cgi?id=1044648>
18:46 dbruhn Honestly no I don't
18:46 aixsyd shite.
18:46 dbruhn Everything I do is either 10GB or 40/56GB Infiniband
18:47 dbruhn if I need more than what a single 1GB nic can do for me
18:47 semiosis jseeley_vit: in a replicated or distributed-replicated volume, you set a replica count, then you have a multiple of that number of bricks.  for example, replica 2, you could have 2, 4, 6, ... bricks arranged in replica sets.  files will be distributed evenly over the replica sets
18:47 dbruhn You jus trying to bond a couple of nic's for gluster?
18:52 the-me joined #gluster
19:08 PatNarciso joined #gluster
19:11 smellis aixsyd: need help with bonding?
19:12 aixsyd i dooo
19:13 smellis aixsyd: you marked the interfaces as manual, meaning you have to manually ifup them, using static instead
19:13 aixsyd ahhhhhhhhh
19:14 smellis aixsyd: you may need to define the slave interfaces too, but I'm not sure, have used deb based distros in a while for that purpose
19:14 smellis aixsyd: they work when you ifup bond0 right?
19:14 solid_liq joined #gluster
19:15 aixsyd one sec, dell just showed up
19:15 solid_liq joined #gluster
19:15 aixsyd >.<
19:19 the-me joined #gluster
19:20 aixsyd smellis: even after ifup, the bond0 is still ni broadcast mode, not getting an IP
19:21 aixsyd *in
19:22 aixsyd also, slaves are defined
19:22 aixsyd this is the config
19:22 aixsyd http://fpaste.org/62964/87391956/
19:22 glusterbot Title: #62964 Fedora Project Pastebin (at fpaste.org)
19:25 FarbrorLeon joined #gluster
19:25 diegows joined #gluster
19:36 aixsyd GAHHH this is infuriating
19:37 aixsyd omg... that was dumb.
19:37 aixsyd iface bond0 inet manual
19:38 aixsyd should be iface bond0 inet static
19:38 aixsyd SLRGHLFG
20:19 smellis aixsyd: haha, that's what I saif
20:19 smellis said
20:19 smellis lol
20:21 aixsyd but een manually ifuping them didnt attach an IP
20:21 aixsyd *even
20:21 aixsyd oddly
20:26 aixsyd dbruhn: heres an interesting one for ya - i set up a cluster, and im transfering data to server 1. i have a dual gig nic set up on each as the in-between. data is being written to both via their network-facing ports, the interconnect dual gigs have zero activity between them any idea why?
20:29 dbruhn Are you using the FUSE client?
20:29 aixsyd i believe proxmox does
20:30 dbruhn the fuse client communicates with each server directly, the replication happens when the client writes, it writes to both servers.
20:30 dbruhn ls
20:30 dbruhn sorry wrong window
20:30 aixsyd so when is that interconnect utilized?
20:30 dbruhn self healing, and rebalance operations
20:30 aixsyd gotttcha
20:31 dbruhn You would be better off putting your bond outward facing, and if you need more bandwidth at the client bond there as well.
20:33 aixsyd well, i'll have a bond of 4x gig nics that are outward facing, and the IB cards directly conecting to eachother
20:33 aixsyd im waiting on the stupid IB cables, so im using a gig bond for testing now
20:33 dbruhn gotcha
20:34 aixsyd say i have a 4gb file being written to the cluster. half way in, i pull the cable from server 2 - server 1 keeps on writing.
20:34 aixsyd and before the transfer is done, plug in server 2
20:34 dbruhn yep
20:34 aixsyd splitbrain?
20:34 dbruhn yep
20:34 aixsyd something a self-heal couldnt fix?
20:34 semiosis not splitbrain
20:34 aixsyd ?
20:34 dbruhn 3.4 might keep it from writing with quorum, not 100% sure
20:35 aixsyd shall i try it?
20:35 aixsyd :P
20:35 dbruhn At this point I would say you are probably manually fixing it
20:35 aixsyd am i daring?
20:35 aixsyd hmmm...
20:35 aixsyd fuck it, lets try it
20:35 semiosis split brain is when the same file is written out of sync on two bricks.  if one brick is taken offline, it can't be split brain, because nothing is writing to it
20:36 dbruhn semiosis, he was talking about mid write
20:36 semiosis doesnt matter
20:37 dbruhn explain?
20:37 aixsyd k, brought back server 2. now what? how could i check for health?
20:38 semiosis aixsyd: look at client log file
20:39 aixsyd looking for...?
20:39 semiosis when client loses connection to a brick it will hang until ping-timeout, hoping the brick comes back online.  if you reconnected the cable before ping-timeout, client will just resume
20:39 aixsyd hm.
20:39 semiosis if ping-timeout occurs before you reconnect the cable, client will disconnect from the brick and continue working with the remaining replica
20:40 semiosis the file you're writing to on that replica will be marked as having changes not synced with the down replica
20:40 semiosis when the down replica returns to service, and the client reconnects to it, it will sync up the changes from the good replica
20:41 semiosis none of this is a split brain
20:41 aixsyd so, i see nothing on the client log
20:42 aixsyd i see a lot of readv failed no data available
20:42 semiosis either you reconnected the cable before ping-timeout (default 42 seconds) or you're looking in the wrong log file :)
20:42 aixsyd it was long after 42 seconds
20:42 MugginsM joined #gluster
20:43 aixsyd /var/log/glusterfs/mout-point-name.log
20:43 semiosis yes
20:43 aixsyd yepper
20:43 MugginsM is the volume rebalance status "size" field the total moved since the rebalance started?
20:44 MugginsM cos if so, rebalance is sllloooowwwww
20:45 l0uis Does anyone have any ideas/tips/pointers/thoughts on an apparently deadlock/hang in the gluster client? I have a workload that is opening and writing to a series of filles (about 7000). Proceeds fine for a period of time then hangs. The same program works fine on local disks.
20:45 l0uis In examining the open file descriptors of my process when the hang happens, there is one file open, the others have been closed. The writing happens from 8 forked children, so only 1 child has an open fd to the gluster mount.
20:46 l0uis Running under strace has not reproduced the issue, unfortunately. Without strace the issue happens consistently.
20:46 l0uis This is on 3.4.1
20:47 l0uis When things are hung. Fuse reports 0 waiting requests in /sys/fs/fuse/../waiting
20:50 l0uis Issue also occurs without multiple writers in a single threaded process, but takes longer.
20:50 MugginsM hmm, dunno
20:54 semiosis MugginsM: not sure about the size field, but yes rebal is slow
20:54 MugginsM I just added 4TB to a nearly full 4TB volume, and in 12 hours, rebalance has moved 31GB?
20:54 semiosis l0uis: no idea
20:54 MugginsM it's still writing lots to the almost-full bricks
20:55 l0uis semiosis: not sure if that should make me feel good or bad, honestly ;). on one hand, surely if such a race/deadlock existed others would have hit it by now? on the other... ugh :)
20:56 aixsyd semiosis: is there a way to see when all healing is completed?
20:56 semiosis l0uis: are you stracing your app or the glusterfs fuse process?
20:56 l0uis semiosis: gluster process
20:57 semiosis aixsyd: what version of glusterfs?
20:57 aixsyd 3.4.1
20:57 aixsyd latest
20:57 aixsyd i guess thats .1-1
20:57 aixsyd er .1-2? whatever the latest one is.
20:57 semiosis gluster volume heal info, iirc?
20:58 aixsyd http://fpaste.org/63016/00319138/
20:58 glusterbot Title: #63016 Fedora Project Pastebin (at fpaste.org)
20:58 aixsyd what does this output mean?
21:00 semiosis i presume those are files that need to be healed?
21:00 semiosis not sure tho
21:00 aixsyd cause if i do gluster volume heal info healed
21:00 aixsyd http://fpaste.org/63019/13874004/
21:00 glusterbot Title: #63019 Fedora Project Pastebin (at fpaste.org)
21:01 aixsyd so its saying that file healed - okay - then why is it also listed under just heal info?
21:01 semiosis idk, maybe logs have more info... there should be a shd (self heal daemon) log
21:02 aixsyd shd.. okay
21:04 aixsyd aaaaaaaand now that file is not there for standard heal info
21:04 aixsyd okay.
21:04 aixsyd so it just takes a bit of time
21:04 aixsyd got it now ^_^
21:05 jseeley_vit I've got a question about the timestamps for gluster events. I'm seeing timetamps for gluster events that do not match the timestamps of the server. http://fpaste.org/63021/
21:05 glusterbot Title: #63021 Fedora Project Pastebin (at fpaste.org)
21:05 aixsyd and thats awesome, when its done healing, the bond interconnect stops transfering/no activity lights
21:05 jseeley_vit The gluster events are seemingly happening in the future...
21:06 semiosis jseeley_vit: timezone difference?
21:06 MugginsM are your times on clients/servers synchronized?
21:06 jseeley_vit MugginsM: Yes, they are.
21:06 jseeley_vit It seems like a timezone difference but where would that difference be? Doesn't gluster get its time from the server?
21:07 semiosis apparently not?
21:07 jseeley_vit ha. hmm..
21:07 jseeley_vit even the timestamps of the logs in /var/log/glusterfs are that way too
21:07 MugginsM when we upgraded from 3.3 to 3.4 it started logging in UTC rather than localtime. seemed a bit random :)
21:09 jseeley_vit https://bugzilla.redhat.com/show_bug.cgi?id=958062
21:09 glusterbot Bug 958062: high, medium, ---, rhs-bugs, ASSIGNED , logging: date mismatch in glusterfs logging and system "date"
21:10 MugginsM aha, that'd explain it
21:16 l0uis semiosis: do you have any thoughts on how i could debug this? anything at all? I'm sort of at a loss as to how to proceed since it doesn't happen under strace (so far).
21:16 FarbrorLeon joined #gluster
21:16 semiosis strace the bricks?  strace the app?
21:22 l0uis semiosis: i'll try to strace the app. re the bricks, i've got 10, so that will be a pain but i guess is the next step
21:22 semiosis pain is inevitable. suffering is optional.
21:24 l0uis semiosis: pain is weakness leaving the body? :)
21:24 theron joined #gluster
21:25 MugginsM anyone got hints for recovering from a failed rebalance? :-/
21:27 dbruhn joined #gluster
21:27 dbruhn semiosis, sorry didn't mean to bail after asking that question earlier, laptop battery died, and I forgot my extra charger.
21:28 semiosis no worries :)
21:29 MugginsM heal-failed on 1023 gfids *sigh*
21:32 yinyin_ joined #gluster
21:36 neofob left #gluster
21:36 dbruhn Hey the link to the IRC logs on http://www.gluster.org/interact/log-details/ is linking to the old logs. What's the new log URL?
21:36 glusterbot Title: Chat Archive Log Details | Gluster Community Website (at www.gluster.org)
21:41 johnmark dbruhn: http://irclog.perlgeek.de/gluster/
21:41 JMWbot johnmark: @3 purpleidea reminded you to: thank purpleidea for an awesome JMWbot (please report any bugs) [269974 sec(s) ago]
21:41 JMWbot johnmark: @5 purpleidea reminded you to: remind purpleidea to implement a @harass action for JMWbot  [198738 sec(s) ago]
21:41 JMWbot johnmark: @6 purpleidea reminded you to: get semiosis article updated from irc.gnu.org to freenode [103268 sec(s) ago]
21:41 JMWbot johnmark: Use: JMWbot: @done <id> to set task as done.
21:41 glusterbot Title: IRC logs - Index for #gluster (at irclog.perlgeek.de)
21:41 semiosis wow
21:42 semiosis johnmark: so the case study says the #gluster channel is on irc.gnu.org, but i've never heard of that irc server/network
21:43 badone joined #gluster
21:49 yinyin_ joined #gluster
21:51 johnmark semiosis: it's the same IRC network
21:51 semiosis huh, ok
21:55 yinyin_ joined #gluster
22:03 marcoceppi joined #gluster
22:03 marcoceppi joined #gluster
22:08 MugginsM right, so this week's lesson is "don't do a rebalance on 3.3.1 when a brick is full"
22:14 Alex Tell me more!
22:14 Alex As I've currently got a volume where one brick is still full even after a rebalance
22:14 Alex and no matter how much I stare angrily at it, it's not rebalancing.
22:15 MugginsM I'm on 3.4.1 now, and it's rebalancing at a speedy 1GB/hour
22:15 MugginsM while I fix hundreds of permission errors and duplicate files :-/
22:15 Alex Heh
22:15 MugginsM unfortunately the users are writing to the full bricks faster than it can rebalance
22:15 Alex https://gist.github.com/bc234dbf0d9f2793d3c0 <- this is 3.3.1.
22:15 glusterbot Title: gist:bc234dbf0d9f2793d3c0 (at gist.github.com)
22:15 Alex now, I am worried :)
22:16 MugginsM I think it's mostly fine, just seems to be an occasional unlucky one
22:16 andreask joined #gluster
22:19 MugginsM of course it has to be Nagios that tells me the users are suddenly uploading 1TB of data, not the users...
22:24 Alex So this is odd. The rebalance log says: "data movement attempted from node (shared-replicate-0) with higher disk space to a node (shared-replicate-2) with lesser disk space" -- but shared-replicate-0 is at 100% disk usage for both subvolumes, whereas shared-replicate-2 isn't.
22:24 yinyin joined #gluster
22:30 spechal joined #gluster
22:30 semiosis s/space/usage
22:30 semiosis Alex: ^
22:30 Alex Mm?
22:31 semiosis the statement is less odd if you replace space with usage
22:31 Alex sorry, I may have the dumbs by now. In this case - https://gist.github.com/alexjs/bc234dbf0d9f2793d3c0 - data1 == 'shared-replicate-0', data3 == 'shared-replicate-2'
22:31 glusterbot Title: gist:bc234dbf0d9f2793d3c0 (at gist.github.com)
22:31 semiosis bbiab, coffee time
22:32 Alex Agreed, it is less odd that way :) I'm just not sure why Gluster isn't doing the migrating, as moving from a node with high disk usage to a low disk usage node seems sensible
22:33 ira joined #gluster
22:34 diegows joined #gluster
22:36 JoeJulian Alex: It's hashing the filename, applying the hash map, and determining that the file it's looking at belongs on the drive that's full. The default rules prevent that move and it informs you of that.
22:36 mypetdonkey joined #gluster
22:37 Alex JoeJulian: Interesting, for some reason I thought the default rules would prefer uniform utilisation - but a dangerous assumption, I concede :)
22:37 mypetdonkey semiosis: see? this is irc.gnu.org :)
22:37 Alex Are the hashmaps not updated as part of the fix-layout to enforce that?
22:37 * semiosis pulls back mypetdonkey's mask to reveal johnmark
22:37 mypetdonkey LOL
22:37 semiosis i never doubted you, bro
22:38 mypetdonkey oh I just wanted to make sure
22:38 semiosis cool
22:38 mypetdonkey semiosis: you may not have doubted me, but I did :)
22:38 JoeJulian The hash-maps are reallocated to allocate a numerically balanced set of hashes to each dht subvolume. If you have a perfectly random set of hashes, that should work out to uniform usage.
22:39 JoeJulian I had to read that three times to make sure I said what I thought I said, so hopefully it's not too cryptic.
22:40 mypetdonkey left #gluster
22:40 MugginsM I guess the problem comes when a brick is full and it has to move a bunch off before it can move new ones on
22:40 MugginsM which it won't know to do
22:40 Alex OK, that makes sense. And, statistically it's unlikely we have a perfectly random set, but I'm concerned then that the usage is quite so much higher - a smaller variance I could expect, but one at 100% while the others sit at 25 - 50%, does seem troublesome
22:41 JoeJulian After the first rebalance completes, it should be more uniform.
22:41 JoeJulian "should" being the operative word...
22:41 Alex *nod*
22:41 Alex This is, er, the 4th rebalance :/
22:42 Alex ("TRY IT AGAIN! IT MIGHT WORK THIS TIME!")
22:42 JoeJulian Is this production?
22:42 Alex It is.
22:42 JoeJulian Can you use the qa release - at least until you have a successful rebalance?
22:42 Alex And given the extra bricks are, fundamentally, quite roomy, it's not service impacting -- but it's troubling me
22:43 JoeJulian I think there's bugs that have been addressed, though I haven't actually looked at the patch log.
22:43 Alex Will have a dig through too, yeah. It's Really Very Unlikely that I can risk migrating these boxes, but it's certainly the next thing I'll try in a more controlled environment.
22:44 MugginsM at this rate my first rebalance will be done some time mid-Feb :-/
22:44 MugginsM lets hope it speeds up a bit
22:45 Alex M'off. Cheers for the help JoeJulian, semiosis.
22:46 MugginsM yeah, you guys rock
22:53 l0uis :(
22:53 sankey i don't know if this is is a common mistake, but it set me back a full day!!!
22:53 sankey when mounting a gluster volume, the path part of the mount device (after the colon) should just be /<volume> and NOT the full path to the mounted brick
22:54 sankey i kept doing mount -t glusterfs localhost:/full/path/to/brick /mnt/gluster0
22:55 sankey and hitting my head on the wall as the log just kept saying "failed to get the 'volume file' from server"
23:00 gdubreui joined #gluster
23:02 jbd1 sankey: that seems like it should be obvious-- what would you do if a host had multiple bricks?
23:09 sankey jbd1: of course it didn't make sense! but that's not what was going through my mind when i was working with one brick on each host
23:18 JoeJulian Glad you got it figured out.
23:19 JoeJulian You're not alone. There's a lot of people that have trouble separating the concept of a brick from a volume.
23:23 rotbeard joined #gluster
23:35 yinyin joined #gluster
23:45 B21956 joined #gluster

| Channels | #gluster index | Today | | Search | Google Search | Plain-Text | summary