Camelia, the Perl 6 bug

IRC log for #gluster-dev, 2012-11-12

| Channels | #gluster-dev index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
01:51 sghosh joined #gluster-dev
01:58 maxiz joined #gluster-dev
02:21 maxiz joined #gluster-dev
02:22 sunus joined #gluster-dev
03:01 bala joined #gluster-dev
03:28 hagarth joined #gluster-dev
04:11 sripathi joined #gluster-dev
05:40 shireesh joined #gluster-dev
05:46 sripathi joined #gluster-dev
06:58 mohankumar joined #gluster-dev
07:02 deepakcs joined #gluster-dev
07:12 lkoranda joined #gluster-dev
07:13 lkoranda joined #gluster-dev
07:24 maxiz joined #gluster-dev
07:29 sripathi1 joined #gluster-dev
07:48 sripathi joined #gluster-dev
08:07 lkoranda joined #gluster-dev
08:26 inodb joined #gluster-dev
08:48 sunus hi, i want to make sure one thing, is the last(bottom) volume section is the top of a graph?
09:02 1JTAAQD6F joined #gluster-dev
09:10 kd1 joined #gluster-dev
09:47 kd joined #gluster-dev
09:56 sripathi joined #gluster-dev
10:45 kd left #gluster-dev
10:53 20WABJLOG joined #gluster-dev
11:13 puebele1 joined #gluster-dev
11:33 inodb_ joined #gluster-dev
12:18 puebele joined #gluster-dev
12:41 maxiz joined #gluster-dev
12:48 puebele joined #gluster-dev
13:16 puebele1 joined #gluster-dev
13:35 puebele1 joined #gluster-dev
14:00 bulde joined #gluster-dev
14:19 inodb_ Hello, i would like to know what is the best way to _maintain_ a specific user defined layout
14:19 inodb_ do i have to run setfattr on a periodic basis on the layout ?
14:20 inodb_ i mean on newly created directories the layout will remain the "normal" glusterfs layout
15:13 wushudoin joined #gluster-dev
15:19 lkoranda joined #gluster-dev
15:56 jbrooks_ joined #gluster-dev
16:17 jdarcy inodb_: Since e066a5fea7bdaa5da78e49c9a5bf344af2f33d3c (March 23) GlusterFS will preserve user-set layouts.  Unfortunately seems to have been passed over for 3.3 though.  :(
16:21 inodb_ jdarcy, do you mean that in 3.3.1 your patch is not applied ?
16:25 jdarcy inodb_: Sadly, that is correct.
16:31 inodb_ jdarcy, that's strange. Anyway i'm experimenting with a git version a the moment
16:31 inodb_ which includes your patch :)
16:36 jdarcy inodb_: Just make sure you set the layout type to 1, and we should leave it alone.
16:36 jbrooks_ joined #gluster-dev
16:39 inodb_ jdarcy, do you mean on setting value of trusted.glusterfs.dht ?
16:40 jdarcy inodb_: Yes.  If you're setting layouts yourself, I'm guessing you've read http://cloudfs.org/2011/04/gl​usterfs-extended-attributes/
16:41 jdarcy Also http://hekafs.org/index.php/2011/​04/glusterfs-extended-attributes/ since something seems to have happened to cloudfs.org
16:42 jdarcy In any case, the part I refer to as a "format designator" needs to be a 32-bit (big-endian) one, not zero.
16:50 semiosis jdarcy: cloudfs.org expires dec. 1
16:51 inodb_ jdarcy, ok i was not sure how exactly this was used. In your script rebalance.py it's "0x0000000200000000%08x%08x"
16:58 davdunc joined #gluster-dev
16:58 davdunc joined #gluster-dev
17:00 jdarcy Looks like it was an error in the link, not the site.
17:00 jdarcy http://hekafs.org/index.php/2011/​04/glusterfs-extended-attributes/
17:02 jdarcy Hm, I think that should be 0x0000000100000000%08x%08x
17:05 inodb_ i was thinking of using : 0x0000000100000001%08x%08x
17:09 JoeJulian jdarcy: You haven't implemented gf_dm_hashfn in python, have you?
17:09 JoeJulian Oh, even better... can I access that from glpy?
17:10 JoeJulian s/...
17:22 JoeJulian Oh, nevermind. I see how that works.
17:22 JoeJulian Cool! I can now calculate the hash of a given filename.
17:26 johnmark mohankumar: ping
17:29 jdarcy JoeJulian: You don't even need glupy.  Just ctypes.
17:30 JoeJulian Yeah, I'd never done that before. It's nicely simple.
17:33 jdarcy JoeJulian: Almost feels like cheating, doesn't it?
17:34 JoeJulian It does, really. On friday I toyed with writing that algorithm in python. In 3 lines I can just use the library.
17:35 JoeJulian Could do it in 2 but I prefer readability.
17:35 jdarcy "Wait, you mean I can just reach in and call somebody else's functions?  From another language?  How the hell does that work?"
17:36 jdarcy And the answer is that it's *really* ugly on the inside, but you don't need to know that.
17:46 JoeJulian hehe
17:48 JoeJulian Is trusted.glusterfs.dht a mask, range, or what?
17:49 johnmark wow, this sounds pretty awesome
17:50 johnmark jdarcy: any word on whether user-defined layouts can mke it into 3.4?
17:52 jdarcy No idea.
17:53 semiosis while we're all here... important question:
17:54 nick5 joined #gluster-dev
17:54 semiosis what's gluster's position on maintaining stable releases?
17:54 semiosis you know, 3.4 is coming out soon, 3.3.1 is barely out the door
17:54 semiosis how long will 3.3 be maintained before we have to switch to 3.4... and will that switch require downtime like the 3.2->3.3 switch did?
17:56 semiosis and if there's no easy answer from "upstream" -- does anyone know if redhat has a position on this w/r/t RHS?
18:06 inodb_ JoeJulian: trusted.glusterfs.dht *includes* a range of hash values that a directory handles
18:07 inodb_ jdarcy explains on his blog thatseems that glusterfs by default
18:08 inodb_ distributes this hashes without knowing brick sizes
18:08 inodb_ which is a problem on setups with different sizes
18:10 inodb_ i am currently trying to modify the layout as jdarcy does in his rebalance.py script
18:10 inodb_ but it seems that if i create a new directory
18:10 inodb_ on a distributed setup
18:11 inodb_ this directory is created with the gluster calculated range
18:12 JoeJulian I was in the middle of writing a blog article with practical examples of dht but found inconsistencies with how I thought it worked.
18:12 jdarcy All new directories will be created with the default (non-size-weighted) layout.  It's not inherited from the parent or anything (I'd like it to be).
18:13 johnmark semiosis: great question. we should consider this from the board's PoV and work out a solution there
18:14 johnmark semiosis: from the RHS perspective, it depends on what types of changes go in-between versions
18:15 johnmark but I know that easier rolling upgrades is somethign they want to enable
18:15 johnmark we should also have an easier upgrade path. it's one of my pet peeves
18:15 semiosis johnmark: +1 to all
18:16 JoeJulian Since this is considered an enterprise level tool I would like to lean toward matching enterprise life-cycles. Prior to 3.1 I don't think that was possible.
18:16 semiosis enterprise life-cycles?  that's scary coming from a guy who still runs fc4/cobol ;P
18:16 johnmark LOL
18:16 JoeJulian lol
18:18 semiosis even if the answer is '6 months' -- or whatever -- i think it would be helpful to let people know up front so they can plan their upgrades accordingly
18:18 JoeJulian Referring more toward RHEL and whichever canonical distro is supposed to be considered enterprise.
18:20 JoeJulian https://access.redhat.com/su​pport/policy/updates/errata/
18:23 semiosis while that would be great for RHS i think it's a bit unrealistic for the upstream glusterfs open source project
18:23 JoeJulian avati once talked about wanting to do devel/stable versions.
18:52 puebele joined #gluster-dev
18:52 the-me semiosis: hey :)
18:53 semiosis the-me: hi :)
18:53 the-me how you are?
18:53 semiosis very well, thanks.  and you?
18:54 the-me I am just back from vacation and now I am on moving, but fine :)
18:54 jdarcy I would definitely favor a six-month "as close as possible to what's in master at the time" kind of cycle.
18:54 semiosis i have a commit for you... changing glusterfs-server "suggests" nfs-common to "depends-on"
18:55 bulde joined #gluster-dev
18:55 the-me why a hard dependency?
18:57 semiosis because glusterfs-server will by default autmatically share glusterfs volumes over nfs, and it must register them with portmapper to do so
18:57 semiosis if portmapper is missing glusterfs thinks volumes are shared over nfs but nfs clients wont be able to connect
18:57 semiosis nfs-common depends on portmapper
18:58 semiosis in short, a hard dep so glusterfs' nfs feature will work out-of-the-box
18:58 the-me but if the administrator does not want to have portmapper/nfs on this machine? suggest is very weak yeah, but if glusterfs still works without it, recommends may be the better choice
18:59 the-me recommends are installed per default like dependencies, but experienced users disable this "install recommends like depends"-feature, to keep there systems clean and as minimal as possible.
19:00 semiosis ah, ok, then recommends sounds good
19:00 the-me fine I will commit
19:00 semiosis oh thanks!
19:00 the-me just wanted to have a look on it because of http://bugs.debian.org/cgi-b​in/bugreport.cgi?bug=691595
19:01 kkeithley fwiw, I'll be adding a dep on portmapper in the Fedora and EPEL packaging. Since I just spun new releases though I'm going to hold off for a while and see if anything else turns up that should be in.
19:03 semiosis the-me: i will try to reproduce that bug and fix
19:06 the-me semiosis: looking at the code => reproduceable at brain :-)
19:08 the-me but uhm, it looks like it also applies to testing 3.2.x :/
19:13 the-me semiosis: hmm now I am confused about your init script
19:15 semiosis the-me: yeah it's not obvious what's going on there
19:15 the-me semiosis: http://nopaste.linux-dev.org/?67292 : on line 9 it is started correctly, line 11 is "duplicated"
19:15 semiosis the-me: iirc even though the variable is unset there is a built-in default for the location, which is /etc/glusterfs/glusterd.conf
19:15 the-me but with an non-existing configuration
19:16 the-me yep, removing line 11+12 would be enough
19:16 semiosis i'm not sure this problem is 100% reproducible
19:17 the-me with 3.2.x the init script still works
19:17 semiosis hmm
19:18 semiosis brb 30, going to get some lunch
19:27 inodb joined #gluster-dev
19:27 inodb left #gluster-dev
19:43 semiosis back
19:57 the-me semiosis: everything done now :)
19:58 the-me fixed in unstable + experimental and just filled an unblock request ;)
19:58 semiosis great!  thank you
20:00 the-me semiosis: so other open todos?
20:02 semiosis well generally, is there anything like launchpad ppas for debian?  build-servers for debian releases, hosted repos with package signing, etc?
20:02 davdunc joined #gluster-dev
20:02 davdunc joined #gluster-dev
20:02 semiosis kkeithley set up a repo but it was a very manual process, building vms, etc
20:03 the-me johnmark: semiosis: ah because of the buildd topic. I do not think that you (you => gluster/redhat) realy need it, since most up to date versions are quickly uploaded by me to debian and they are also backported to the offical backports repository, if we are not in a freeze like now
20:04 the-me semiosis: no
20:05 the-me and I think before anyone is wasting time in doing duplicate work we should just concentrate on testing/unstable and so on also backport the testing versions. this is the offical by debian supported way
20:08 the-me security support, upgrade support, release cycle life support, signed by debian and up to date; on this debian administrators swear, additional repositories will just introduce new bugs, work and confusion
20:08 inodb joined #gluster-dev
20:09 the-me I am just not able/allowed to upload 3.3.x to squeeze-backports, since we are in freeze, so most of time users are also able to use the most recent glusterfs on their debian stable, with support
20:45 semiosis got pulled into a meeting, back now
20:45 semiosis so, the freeze happens for new releases, which is every two years right?
20:46 semiosis and how long should the freeze last?  seems like it has been a long time already
20:46 the-me in the last years it was something like a 2 year cycle to the next freeze, yeah
20:46 semiosis and outside of that freeze time, how long will it take to get a new release into -backports for stable?
20:47 the-me and the freeze is over if all release critical bugs are fixed and testing will become the new stable release => when it is done
20:48 the-me but there are _again_ good signs that it is done at all in 6 months (from the beginning of the freeze, so on => january, but no guarante)
20:50 the-me semiosis: not very long. outside of the freeze redhat would e.g. release 1.2.3, one day later e.g. (normaly I am quick) I am uploading 1.2.3-1 to unstable. if no critical bugs appear it will normaly migrate to testing after ten days. if it is migrated to testing I am allowed to upload the testing one backported to the offical backports archive
20:52 the-me so it may need until two weeks under normal concirmuations; but users installing it from backports (gluster/redhat customers) could be sure (or a little bit more sure) that this new release will not fuck their business down
20:52 the-me so it is bleeding edge with some more testing
20:52 semiosis well that sounds great
20:53 the-me gluster just should add installation instructions to the webpage
20:53 the-me and for this: http://backports-master.debian.org/Instructions/
20:54 semiosis ah good to know
20:55 the-me in the current stable release squeeze 3.0.5-1 is available, but if you want/need 3.2.7-2~bpo60+1 is available currently in squeeze-backports.
20:57 the-me johnmark: semiosis: do not understand me wrong, I welcome your decision to support Debian, but I think everyone will benefit (users, maintainers, developers) if we are working on the offical ones and do not "fork" the packaging.
20:59 semiosis i understand
20:59 the-me and if we are at this topic..
20:59 semiosis johnmark: ping -- you around?
21:00 the-me .. would be a good decision to warn about the ubuntu packages, they are neglected as most packages there..
21:01 semiosis yes i know :(
21:01 the-me general ubuntu problem
21:01 johnmark semiosis: pong
21:02 semiosis johnmark: just wanted to see if you were around & following this
21:02 * johnmark reads backscroll
21:03 semiosis the-me: the problem with ubuntu is upstart... which means the package must be forked from debian, a "merge" in ubuntu terms, not a simple sync
21:03 johnmark ok
21:04 semiosis the-me: and the problem with that is it is very time consuming for me to maintain the fork within ubuntu's release process
21:04 johnmark so the issue is... what exactly? that our packaging is different from official packaging?
21:04 semiosis the-me: however i have been maintaining the fork in my own ppa, which is extremely easy
21:04 johnmark the-me: and you're recommending that we merge in some way?
21:04 the-me semiosis: upstart is "simply" changing the init parts, I mean the whole bugreports, the responsivness, syncing, security support etc..
21:05 the-me johnmark: I didn't had a look at your packaging
21:05 johnmark the-me: that would mean that we need an "official" packager working on debian packages, yet?
21:05 johnmark the-me: ok
21:05 johnmark just trying to understand what we should do
21:05 the-me johnmark: I am an offical Debian Developer, what else do you need? ;-)
21:05 semiosis johnmark: the-me is our official debian packager :D
21:06 semiosis the issue is...
21:06 semiosis that debian's release process may at times (freeze) delay people from getting new glusterfs releases
21:06 semiosis roughly 6 months out of 24
21:07 semiosis besides for that issue i agree it would be beneficial for glusterfs and debian to not maintain two forks, but focus our efforts on one official debian package
21:08 semiosis i knew backports existed but i did not know that glusterfs packages went there from testing to stable automatically, or how to install from backports
21:08 the-me .. but do not forget, that the debian using audience of glusterfs mostly knows how to use the packaging systems, so I think most users - if they want to use the new release within a freeze - know how to install/build it from the offical (debian) sources
21:08 semiosis that is very good to know :)
21:09 semiosis i have to disagree there... i think most people (like myself) only know how to apt-get install
21:09 the-me semiosis: testing => stable is not automagic, I have to backport it manualy
21:09 the-me and I am doing this work.
21:09 semiosis oh, well you make it look like magic :)
21:10 the-me rpm building is magic for me, debian packaging is as easy as walking for myself ;-)
21:11 semiosis in my experience most people new to glusterfs just want to install the latest package, and if they have to build it they will give up
21:12 semiosis many people apt-get install from the main repo and think 3.0.5 is the latest
21:13 semiosis i think giving them backports instructions would be OK, but asking them to build a package from source would be a blocker
21:13 the-me johnmark: as I said I didn't had a look at the gluster deb packaging, so I can not say that it is wrong, faulty, correct, beatiful, better, whatever. But IMO it would be the best for *everyone* to bundle the ressources at one point and which point is better than the offical one?
21:14 semiosis the-me: let me clear up a bit of recent history you missed while on vacation
21:14 semiosis before glusterfs 3.3.0 there were "unique" glusterfs deb packages hosted on the gluster FTP site, these were totally independent of the debian packages
21:15 semiosis gluster is no longer producing those "unique" packages since 3.3.1
21:16 the-me semiosis: glusterfs was in the last months in the most representative linux/opensource printmagazins available, but mostly I were shocked since they had got a todo "how to install"-from-source on a debian system, where apt-get install were enough.. exactly the same versions..
21:16 semiosis just now since 3.3.1 kkeithley set up a repo with squeeze & wheezy packages which are clones of the official debian packages, but built and hosted independently
21:17 the-me do not know how this could happened, normaly they only recommend this step if they want to present a quite newer version
21:17 semiosis maybe the magazines did not know about backports?
21:17 the-me they do
21:18 the-me I know some of those authors personaly.
21:19 semiosis well seems there is no way right now to install glusterfs 3.3.1 using apt-get on stable... because of the freeze... could that be the reason?
21:19 the-me but this is more a marketing topic, before doing any marketing the product has to be ready and available, it was, but the marketing failed
21:19 the-me no that were about the 3.2.x ones
21:19 semiosis oh
21:20 the-me I didn't saw any printed article yet about the 3.3.x branch, but it is quite new
21:20 semiosis johnmark: did you know glusterfs was being talked about in print magazines? :)
21:20 semiosis i didn't know that
21:20 semiosis pretty cool imo
21:21 semiosis though sad about the difficult install instructions
21:21 the-me semiosis: I do not know if it was one of the "Linux Magazin" or "Admin" (german), but it also was a cover topic on one of these
21:23 the-me semiosis: yeah okay, it is not hard to install something about the well known three-steps, but - for myself - in the hard business you have to use well known and tested software. so if a magazine e.g. publishs an article about the great software xyz...:
21:23 the-me ...: and it is not installable about well known package managers, it may be to dangerous
21:24 the-me .. dangerous, not enough tested, no stable variant, no (security) support, you have to administrate everything yourself..
21:25 semiosis yes I agree.  packages are the way to go
21:25 the-me my main job is administrating and leading hundred of linux servers, software which isn't available in distro repositorys is a no-go.
21:26 johnmark the-me: good points
21:26 johnmark yup, definitely re: "software which isn't available in distro repositorys is a no-go."
21:26 the-me .. nice to know that new cool technology is available, but that is maybe considered for the future then
21:29 the-me johnmark: yeah I also read in some print magazines about ceph, also sounds very good, but not more, it is not available in any offical release, yet, so nice to read, not more
21:32 the-me so it is much more important to have a as good as possible packing in the offical respositories (debian, suse, redhat, whatever), so potential users just could install and use it as they are using every other software on their servers, without adding unoffical repositories, adding some unknown not trusted gpg keys, compiling it for themselve, and what most people forget:
21:33 the-me .. so they do not have to investigate in bug/security fixes thereselve
21:34 the-me ... or that debian users could continue to report bugs to the debian bugtracker and do not have to register somewhere else, same for suse, redhat whatever users.
21:34 the-me and if those things are working well, marketing is done by itself
21:36 the-me then also nobody has to fear to test and implement it, also users need less time to evaluate and deploy it
21:38 the-me johnmark: semiosis: so the most important thing is, that everyone is working together :)
21:39 semiosis sounds good to me
21:40 semiosis only thing i would like resolved is how to support users who want to apt-get the latest glusterfs version during a freeze
21:40 semiosis any ideas on best way to do that?
21:40 semiosis i dont want to tell them to wait for the freeze to be over :/
21:40 the-me depends
21:40 semiosis i mean, i *am* a user and i would not like to wait for the freeze to be over :)
21:41 the-me currently users just could fetch the debs from experimental
21:41 semiosis dependency issues on stable
21:42 semiosis i mean, experimental builds would depend on newer versions of libs than are available in stable
21:42 the-me eh yeah sorry, to testing, yeah
21:42 the-me to stable the package has to be rebuilt
21:42 semiosis :(
21:43 the-me which could be quite easy if there wouldn't be any lib conflicts
21:43 semiosis yes, and even with lib conflicts, that is usually easy to fix in debian/control by adjusting the numbers
21:44 the-me http://nopaste.linux-dev.org/?67301 that is e.g. the source difference (as minimal as needed) between testing and squeeze backports, which was required so that the package builds
21:46 the-me this is the only source difference, which was required. so it is important to gain compatibility.
21:48 johnmark the-me: can you shepherd the process so that we can get into -testing and -unstable?
21:48 semiosis johnmark: he already does :)  things are just frozen right now
21:49 the-me johnmark: I am doing so since beginning of 2012 :) http://packages.debian.org/changelogs/pool/​main/g/glusterfs/current/changelog#version2.0.9-1
21:52 the-me ehh  s/2012/2010/
21:52 Technicool joined #gluster-dev
21:53 johnmark the-me: heh heh :)
21:53 johnmark please forgive my ignorance
21:54 semiosis Technicool: http://irclog.perlgeek.de/glu​ster-dev/2012-11-12#i_6145874
21:55 the-me semiosis: ;)
21:56 the-me http://www.admin-magazin.de/Das-Hef​t/2012/02/Das-verteilte-Dateisystem​-GlusterFS-aufsetzen-und-verwalten
21:56 the-me but it also was a top topic at the german linux magazine
21:56 inodb joined #gluster-dev
21:56 the-me and if I remember correctly also at the famous iX
22:00 johnmark the-me: ah, nice
22:00 johnmark ausgezeichnet!
22:05 the-me johnmark: :)
22:05 the-me so sorry now I have to go to bed, different timezone here in germany and I have to work tomorrow, it was nice to talk to you both :)
22:05 semiosis the-me: thanks for everything!
22:05 semiosis we'll talk again soon :)
22:08 inodb joined #gluster-dev
22:51 johnmark the-me: indeed. thank you for everything
23:38 JoeJulian If there's a developer in the house, could you take a look at http://joejulian.name/blog​/dht-misses-are-expensive/ for accuracy please?
23:41 semiosis my 2c: in the So What? section, add a quick tip to optimize your include path, which is a production php best practice anyway.  http://framework.zend.com/manual/1.​12/en/performance.classloading.html
23:48 a2 another thing you can "do about it" is to set 'lookup-unhashed off'
23:49 a2 this way if the linkfile is missing, dht will assume the file does not exist anywhere else (in an unhashed manner) in the volume

| Channels | #gluster-dev index | Today | | Search | Google Search | Plain-Text | summary