Perl 6 - the future is here, just unevenly distributed

IRC log for #puppet-openstack, 2013-10-20

| Channels | #puppet-openstack index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:26 tnoor1 joined #puppet-openstack
00:27 tnoor2 joined #puppet-openstack
00:39 bodepd_ I prefer inifile. I think the implementation is much cleaner.
00:41 dachary bodepd_: +1
02:30 bodepd_ ceph.conf is a basic ini file, right?
02:32 bodepd_ ls
02:33 openstackgerrit Dan Bode proposed a change to stackforge/puppet-ceph: Add project files  https://review.openstack.org/52791
02:51 e1mer joined #puppet-openstack
03:18 e1mer joined #puppet-openstack
03:31 e1mer joined #puppet-openstack
06:12 tnoor1 joined #puppet-openstack
06:49 tnoor1 joined #puppet-openstack
06:58 dalgaaf bodepd_: I have to take a look at the inifile and how it really works. especially the issues (as far as I understood it) with comments and ordering within the files may makes it hard for me to use. I really like the approach to have a template file and to fill it depending on the parameter/variables set. It makes the file much easier to read.
07:02 dalgaaf Not sure how the sharing of information between nodes work with inifiles ... e.g. adding information about the different MONs back to the other configs without the need to have to keep all the info in a central place ... which is really easy with concat
07:02 e1mer joined #puppet-openstack
07:32 xarses dalgaaf if any one uses ceph-deploy it will strip all comments and create a none deterministic order
07:32 xarses just as a fyi
07:35 xarses as far as i can tell inifile will replace the parameter in the current file where ever it lies
07:35 xarses if its currently in a file
07:41 dafter joined #puppet-openstack
07:46 e1mer joined #puppet-openstack
07:49 e1mer joined #puppet-openstack
08:15 dafter joined #puppet-openstack
08:15 dafter joined #puppet-openstack
09:06 dalgaaf xarses: hm ... that behavior somehow sucks ... produces inifile a diff I something changes in the config file? This is something I really like about concat ... I always get a diff displayed on the puppet run and can then easily see what changed
09:17 marun joined #puppet-openstack
09:57 e1mer joined #puppet-openstack
10:32 e1mer joined #puppet-openstack
11:05 dafter|afk joined #puppet-openstack
11:08 dafter joined #puppet-openstack
11:08 dafter joined #puppet-openstack
11:11 dafter|afk joined #puppet-openstack
11:44 marun joined #puppet-openstack
12:45 MxG joined #puppet-openstack
13:27 qba73 joined #puppet-openstack
13:46 dachary dalgaaf: ping
14:01 dalgaaf dachary: sorry ... wasn't that much online this weekend
14:01 dachary no worries :-)
14:01 dachary I'm reading your additions to the blueprint
14:01 dachary and answering
14:02 dachary I +1 https://review.openstack.org/52791 but I guess there should be people able to +2 to make it happen
14:05 dalgaaf dachary: I didn't have the time to add comments to all sections ... I'll read the rest later and leave more comments ...I'm later on the road and will be online ... we can discuss then the other issues like e.g. to start with which classes
14:06 dalgaaf dachary: I read currently the review...
14:06 dachary dalgaaf: regarding ceph versions support, it would make sense to use branches the same way openstack does : stable/cuttlefish stable/dumpling .
14:06 dachary dalgaaf: cool.
14:08 dachary since we're in the context of puppet-openstack, following the path chosen by other modules such as puppet-nova or puppet-cinder seems an easy way to deal with things like package management, versioning, maintenance of stable versions, ...
14:10 dalgaaf hm ...about the branching: yes in general ... but for now I would try to develop it in a way it would work for all ceph versions starting with cuttlefish ... I would not target bobtail and argonaut.
14:11 dalgaaf and as soon as it's completely working for cuttlefish and newer versions we could think about adding new releases which may depend on newer ceph versions
14:11 dalgaaf and features
14:12 dalgaaf what do you think?
14:14 dalgaaf btw, there is already a proposal for relicense the existing puppet-ceph module under Apache v2 and we (DTAG) would also do the same ... this would allow us to use already existing code from there to prevent reinventing the wheel (I don't say that we should use everything, but it would be a good place to start with implementation of it)
14:24 tnoor1 joined #puppet-openstack
14:25 tnoor2 joined #puppet-openstack
14:33 tnoor1 joined #puppet-openstack
14:36 tnoor1 joined #puppet-openstack
14:44 dachary dalgaaf: " it would work for all ceph versions starting with cuttlefish ... I would not target bobtail and argonaut." => makes sense to me.
14:46 dachary dalgaaf: although it was decided to re-start from scratch, it would be helpfull to cherry-pick the parts that can be reused instead of re-coding them, I could not agree more ;-)
14:50 dmsimard joined #puppet-openstack
14:53 dachary dalgaaf: https://wiki.openstack.org/wiki/Puppet-openstack/ceph-blueprint#Support_Ceph_versions_from_cuttlefish and https://wiki.openstack.org/wiki/Puppet-openstack/ceph-blueprint#Module_versioning
15:15 tnoor1 joined #puppet-openstack
17:36 ari joined #puppet-openstack
18:59 tnoor1 joined #puppet-openstack
19:01 tnoor2 joined #puppet-openstack
19:03 tnoor1 joined #puppet-openstack
19:19 tnoor1 joined #puppet-openstack
19:23 tnoor1 joined #puppet-openstack
19:41 francois1 joined #puppet-openstack
19:47 MxG joined #puppet-openstack
19:50 tnoor2 joined #puppet-openstack
20:32 bodepd_ dalgaaf: the main motivation for converting the openstack modules from templates to inifile was that with templates, we had to manage the entire file, so it was difficult to accept package defaults
20:33 bodepd_ dalgaaf: we basically had to rewrite the templates for every openstack release
20:34 bodepd_ dalgaaf: inifile shows you the diffs (just not in unified diff format)
20:34 bodepd_ dalgaaf: for every change, the resource shows old_value -> newvalue
20:37 mjblack if its an ini file I prefer to use inifile instead of template
20:38 bodepd_ dalgaaf: I fee like I should drop out of this. I'm pretty biased, b/c I helped write inifile
20:39 bodepd_ dalgaaf: or maybe that makes me the perfect person to argue it :)
20:39 mjblack bodepd_: I think in this case there isnt a conflict of interest since the file is ini based :D
20:42 bodepd_ I do understand the concern, and I know b/c I argued the same thing for converting to swift
20:42 dachary bodepd_: hi. I sent a mail to become part of https://launchpad.net/~puppet-ceph . Is there something else I should do ?
20:43 dalgaaf bodepd_: as far as I understood it there is no ordering with inifile ... correct me if I'm wrong. Each change could change the whole ordering in the file (or section) .... and how is the ordering of the sections? Is there any way to make sure the file keeps some kind of odering in there?
20:44 dachary let say we have three mons, one on each node. The /etc/ceph/ceph.conf "mon host" = will need to have all three IP addresses. And this conf file will need to be written on all nodes participating in the ceph cluster. How should that be done in your opinion ?
20:44 dalgaaf what about indentation?
20:45 bodepd_ I need to send an email about to the list to kick things off
20:45 dachary ok
20:45 bodepd_ mgagne created the group, let me see if I'm an admin
20:45 dalgaaf with concat this was no problem at all ... each node writes their config and all ends up with the next run (or ever earlier) on the config for all nodes
20:46 bodepd_ dalgaaf: can you be more spefic about: keeps the same kind of ordering
20:46 bodepd_ dalgaaf: inifile does not change the order of the file everytime
20:47 bodepd_ if an entry alreay exists, it updates that line
20:47 dalgaaf and each other node got the [global] [mon] [osd] and [mon] generic sections
20:47 bodepd_ if a commented out version of something exists
20:47 bodepd_ , it updates that line
20:47 bodepd_ if an entry does not exist (for a setting),it adds it to the end
20:47 bodepd_ when multiple settings are created and did not have previous entries
20:47 bodepd_ they are written at the end of the section in a random order
20:48 dalgaaf bodepd_: so I have to provide some kind of template to it to have e.g. all keys beginning with 'auth' in one block?
20:48 bodepd_ so, it doesn't shuffle around files on update
20:48 bodepd_ does the default file provided by the package not already set some of this up?
20:48 dalgaaf is inifile able to handle keys with spaces in it?
20:48 dalgaaf there is no default file
20:48 bodepd_ either by providing default values
20:49 mjblack are we arguing over human readability here?
20:49 dalgaaf at least not installed
20:49 dalgaaf mjblack: yes ... partly
20:49 bodepd_ then for that case, the only way to control the order in which the file is written is with require/before
20:50 bodepd_ require/before is a bit odd b/c it only guarentees order on write
20:51 bodepd_ dachary: you need to support duplicate keys?
20:51 dachary no
20:51 bodepd_ dachary: or it accepts a comma delimited string?
20:51 dachary yes
20:51 bodepd_ an array value will convert into comma delimited string
20:51 dachary mon host = 1.2.3.4,1.2.3.5
20:52 dachary hum
20:52 dalgaaf hm ... this would be part of which section?
20:53 dachary $mon_host = [ '1.2.3.4', '1.2.3.5' ] and added to global
20:53 dachary say mon1.domain.name has ip 1.2.3.4
20:53 dachary then it would also have something like : ceph::mon { ip_address = '1.2.3.4' }
20:54 dachary and mon2.domain.name has ip 1.2.3.5
20:54 dachary then it would also have something like : ceph::mon { ip_address = '1.2.3.5' }
20:54 dachary is there a way to only write that
20:54 dachary /mon1.domain.name/ {  ceph::mon { ip_address = '1.2.3.4' } }
20:54 dachary /mon2.domain.name/ { ceph::mon { ip_address = '1.2.3.5' } }
20:55 dalgaaf dachary: this is only under the assumption there is no special naming of the MONs
20:55 dachary and as a side effect the mon host line of ceph.conf would be set with those ?
20:55 dachary dalgaaf: yes, indeed
20:55 bodepd_ dachary: can you paste me an example of what the config file would look like?
20:56 dachary # cat /etc/ceph/ceph.conf
20:56 dachary [global]
20:56 dachary fsid = 2ce3e8df-0c31-4ffa-b1c0-be43ff8586ba
20:56 dachary mon_initial_members = bm0012, bm0014, bm0015
20:56 dachary mon_host = 188.165.194.113,188.165.223.112,188.165.223.81:6803
20:56 dachary bodepd_: ^
20:58 bodepd_ that is easy enough to model
20:59 bodepd_ (either with erb templates or inifile
20:59 bodepd_ )
20:59 bodepd_ dachary: I am not a manager of the puppet-ceph group
20:59 bodepd_ we have to ask mgagne on Monday
20:59 dachary bodepd_: cool. Is there an example that does that kind of thing in an openstack module by any chance ?
20:59 dachary bodepd_: ok, there is no rush ;-)
21:00 bodepd_ glance_api_servers
21:00 dachary cool
21:00 bodepd_ https://github.com/stackforge/puppet-nova/blob/master/manifests/init.pp#L188
21:01 bodepd_ there is nothing special about it. You just pass a value like normal
21:02 openstackgerrit Dan Bode proposed a change to stackforge/puppet-ceph: Add project files  https://review.openstack.org/52791
21:02 dalgaaf dachary: I would like to still use the documented syntax for the config files and this would always use space instead of '_'
21:03 dalgaaf since this is what all the documentation says
21:04 dachary dalgaaf: I have no opinion really. Whatever works best for everyone ;-)
21:04 dalgaaf it's simply because none of the documentations use the this kind of syntax for the keys (not on the website nor the code )
21:06 dachary bodepd_: how will https://github.com/stackforge/puppet-nova/blob/master/manifests/init.pp#L188  behave if there is more than one nova node is a glance API server ? will the last one win or will they both be listed ? What we want for mons is that it makes a list.
21:06 bodepd_ if it is passed a string, it will be x=y
21:07 bodepd_ if it is passed a list, it will be x=y1,y2,y3
21:08 dachary ok. So there is no way around it, the IP will have to be listed twice. Once as part of "$mon_host =  [ '1.2.3.4', '1.2.3.5' ]" and once as an argument ceph::mon { ip_address = '1.2.3.4' } ? I'm not sure it matters, I was just wondering if that kind of redundancy is something to worry about or not.
21:09 tnoor1 joined #puppet-openstack
21:10 ari joined #puppet-openstack
21:11 tnoor2 joined #puppet-openstack
21:11 dachary dalgaaf: I updated https://wiki.openstack.org/w/index.php?title=Puppet-openstack/ceph-blueprint#osd to describe the osd creation logic
21:11 dalgaaf you don't need the a mon_host line if you add a [mon.$name] section for each mon
21:11 dalgaaf if that is what you mean
21:12 dachary oh, that's a nice idea dalgaaf :-)
21:12 dachary and each mon host can handle its own [mon.$name] section
21:12 bodepd_ dachary: inifile settings with spaces work
21:13 dachary bodepd_: cool
21:13 bodepd_ I had to test it, I had no idea ;)
21:13 bodepd_ nope, I spoke too soon
21:13 bodepd_ there is a bug
21:14 bodepd_ it will write, but it does not properly detect if the lines are there
21:14 dalgaaf not good
21:14 * dachary updating https://wiki.openstack.org/w/index.php?title=Puppet-openstack/ceph-blueprint#mon to reflect dalgaaf  suggestion
21:15 xarses joined #puppet-openstack
21:16 bodepd_ my bad. it works
21:16 bodepd_ I had an old version
21:17 bodepd_ Notice: /Stage[main]//Nova_config[section/foo bar]/value: value changed 'baz' to 'bla'
21:17 dalgaaf then we need to know which version works
21:17 dalgaaf to make sure only this version is used
21:17 bodepd_ the latest should, let me track down the revision
21:19 dalgaaf btw. I would add the auth class to the ceph::conf class or let be it a subclass of this class ... but I would prefer it's part of ceph::conf since this is a significant part of the global section which would be written by ceph::conf I assume
21:20 dalgaaf and by default cephx should be enabled
21:22 dachary how would you create a default key without requiring manual intervention from the user dalgaaf ?
21:23 dachary default mon. key that is
21:23 dalgaaf dachary: I really hope the 'osd tell' works  .... I never tested it ... I could imagine strange situations if e.g. some daemons are not update at the same time because e.g. a puppet run happens later on one machine
21:24 dachary in the spirit of sensible default : i.e. if the user does nothing and by default it does not work, that's not desirable.
21:24 tnoor1 joined #puppet-openstack
21:24 dachary let me rephrase
21:25 dalgaaf you could make this key mandatory until cephx is disabled or not?
21:26 dachary if a user writes /node1/ { include ceph::mon; include ceph:osd; } it must do something useful. As is setup a one osd cluster with a pool size of one and allow the user to ceph -s it right away.
21:26 dalgaaf dachary: cephx is the default if the key is not set btw.
21:26 dachary oh, ok. I'll fix this.
21:27 dalgaaf dachary: from the ceph code side
21:27 bodepd_ dachary: 1.0 release should be fine
21:27 dalgaaf and there os no way to set a cluster up this way ... since the default replica level is 2 if IIRC
21:27 dachary dalgaaf: fixed
21:28 dachary dalgaaf: there is a way : check http://dachary.org/?p=2374. Simple things should be simple.
21:29 dachary dalgaaf: that said, it could also be achieved with cephx enabled and a default key
21:29 dalgaaf I don't say that it wouldn't be possible ... but is it really a good idea to redefine the default values of ceph?
21:30 dalgaaf to get the cluster with  puppet-ceph up and running?
21:30 dalgaaf I would not recommend this
21:30 dachary I would not recommend to put that in production of course :-)
21:31 dalgaaf we could add an example for that small case, but by default I would expect to have a normal ceph cluster
21:31 dachary My goal is simple : if there is a way someone can create a cluster by copy / pasting something a change absolutely nothing else, it must be done.
21:31 dalgaaf with the sane default values from ceph
21:31 dalgaaf and this would be: 1 mon (at least) and 2 OSDs
21:32 dachary ok, for 1 mon and 2 osds. And cephx with a default key ? So the first time user does not need to understand the concept before getting a cluster running ?
21:33 dalgaaf btw. how should that example from above ever work? you need to define at least network addresses
21:33 dalgaaf and disks for osds
21:33 dachary it can use directories, same as the quick start manual http://ceph.com/docs/next/start/quick-ceph-deploy/
21:34 dachary and use the first ip address returned by facter or even localhost
21:35 dachary $::ipaddress
21:36 dalgaaf huh ... not sure if we should do all the 'create a dir in tmp' and so on ... since this will never be used by anyone who tries to deploy a usable cluster and this code would be then pretty useless
21:36 dalgaaf isn't that a workflow ceph-deploy is used for?
21:37 dalgaaf but back to the fault key ... but at the end it's the same disussion ...
21:38 dachary are you suggesting that  include ceph::mon; include ceph::osd; does not work by default ?
21:38 dalgaaf you could add some default key ... but from my perspective I recommend to have a mandatory parameter to parse a value ... shouldn't be that hard for a user
21:39 dachary dalgaaf: I'm phrasing this in a provocative way in order to try to make my point about sensible defaults ;-)
21:39 dalgaaf good question ... never used a puppet module without setting some parameter
21:40 dachary I understand you're not the audience for sensible defaults because you tend to explore and read before trying.
21:41 dalgaaf I understand your point of view ... but may see it different ... since I see it from the perspective of production systems ... and I see the risk to have some values which may get overseen and end up in production
21:41 dachary I'm very pleased when I can just copy / paste, get something to work. And then only explore new options.
21:42 dalgaaf you could generate a key if none is set, but then you need exported resources which would prevent to start up mons in parallel
21:42 dachary dalgaaf: there is no protection against that, IMO. I mean that if we were to set a goal for the module to be moron proof and protect the careless sysadmin from doing mistakes, that would be a formidable undertaking ;-)
21:43 xarses joined #puppet-openstack
21:43 dachary dalgaaf: or have a function (runs on the puppetmaster) that only creates it if not already there ( stored in a file ? )
21:43 tnoor1 joined #puppet-openstack
21:44 dachary I'm thinking something like http://redmine.the.re/projects/l2mesh/repository/revisions/master/entry/lib/puppet/parser/functions/tinc_keygen.rb but I'm not sure it's a good practice.
21:46 dachary essentially the puppetmaster could generate the mon. key + boostrap-{mds,osd} keys and distribute them where needed without the need for the user to intervene at all.
21:47 dalgaaf whatever works for you ... have to think about the security impact and the fact that this would mean to install ceph on the puppet master
21:49 dachary dalgaaf: do you need ceph for that ?
21:49 mjblack does ceph need to generate the key or can it be done external?
21:50 dachary mjblack: that's my thinking, checking
21:50 dalgaaf you could write a function to rebuild the logic to generate the key in your own code
21:50 mjblack It doesnt look like something specific to ceph, cant think of the word, but it might be something similar along the lines of generating a uuid
21:51 dalgaaf it's something different
21:51 dachary https://github.com/ceph/ceph-deploy/blob/v1.2.7/ceph_deploy/new.py#L20
21:51 dachary dalgaaf: what am I missing ?
21:52 dalgaaf ?
21:52 mjblack right so you dont need puppet to generate the key
21:54 dalgaaf If you don't want to install ceph on the puppet master, then you need puppet (in what way ever) to generate the key
21:54 dalgaaf or keys
21:55 mjblack but why would you want puppet to generate it for you? wouldnt it be better if the user generated the keys?
21:55 dachary dalgaaf: yes and it seems fairly easy to do so, don't you think ? https://github.com/ceph/ceph-deploy/blob/v1.2.7/ceph_deploy/new.py#L20
21:55 dalgaaf mjblack: that's no problem for me ... I like the idea to have some mandatory parameter like this one
21:56 dalgaaf but dachary has a different approach
21:56 mjblack when we automated rbd with cinder, it wasnt optimal to generate a new uuid for virsh every time
21:56 mjblack so we use the same uuid for all computes
21:57 dalgaaf what I don't want is some kind of default key that may end up in any ceph cluster visible to the world
21:57 mjblack dont give it a default value
21:57 dalgaaf you need first a key for the admin and the mons
21:57 dalgaaf the rest could be automated
21:58 dalgaaf since the osds generate their own keys
21:58 mjblack it could but the problem is retreiving the key for use elsewhere, it would require storeconfigs which not everyone uses
21:58 dalgaaf and if you use a MDS you need also key for that one
21:58 dachary dalgaaf: what about this : cephx is by default, if the mon key is not set, a function generates ( and persists ) a key ( admin and mon ) that is localy generated ( no need to install ceph ). Would that match your expectations ?
21:59 dalgaaf locally generated means? On the puppet master?
22:00 mjblack so when you mean generates and persists a key, what do you have in mind?
22:01 dachary dalgaaf: yes. A function such as http://redmine.the.re/projects/l2mesh/repository/revisions/master/entry/lib/puppet/parser/functions/tinc_keygen.rb runs on the puppetmaster, not on the node
22:01 dachary mjblack: that's what I have in mind http://redmine.the.re/projects/l2mesh/repository/revisions/master/entry/lib/puppet/parser/functions/tinc_keygen.rb
22:02 mjblack thats a problem if we're talking about writing files on the puppet master without the users knowledge
22:02 mjblack it would be better to have them pass in the key to be used
22:03 dachary pass how ?
22:03 mjblack if I had to build my puppet master for whatever reason, it caught fire from some mythical beast, the last thing I want to worry about is if my ceph cluster fails because a new key was generated
22:03 dachary copy / pasting the result of a command ?
22:03 mjblack yes
22:03 mjblack $ceph_admin_key = "<key>"
22:04 mjblack we currently pass to our cinder/glance systems the key that we store in hiera data
22:05 dalgaaf that's also my problem ... I prefer to have some defaults that area given to puppet ... in whatever way ... but simply make some parameter mandatory
22:05 dachary ok
22:05 mjblack if I can provide ceph with pre-generated keys in a similar fashion I would rather do that instead of a generator, not that doesnt mean we could not include a puppet function to generate a key and use it but that would be up to the user to use it
22:05 dalgaaf as e,g. tge keys
22:08 mjblack I'm all for defaults where they are best suited and mandatory parameters, without defaults, as well
22:08 xarses joined #puppet-openstack
22:08 dachary ceph::mon would have a default key ? as in ceph::mon(key => '019834091843') ?
22:08 mjblack no
22:08 mjblack user provided value
22:08 xarses joined #puppet-openstack
22:09 dachary ( I agree with not using a function to generate a key on the puppetmaster, mjblack  dalgaaf  . I'm convinced.)
22:09 dalgaaf dachary: We have to think about the order of implementation: it should be 1) ceph::conf, ceph::package and ceph::key 2) ceph::mon 3) ceph::osd ...
22:09 dalgaaf since we need these 3 classes/parts to start with the MONs
22:10 dachary dalgaaf: not if auth = cephx is not enabled as a first step ( and then it becomes the default )
22:10 dalgaaf 4) would be ceph::pool and then it doesn't matter that much
22:11 dachary mjblack: ok
22:11 xarses joined #puppet-openstack
22:11 dalgaaf the key class is very simple ... we could take the class over from the puppet-ceph package (or I can rewrite it from scatch in some hours ;-) )
22:12 dalgaaf and as I said: cephx is the ceph default ... you would need to change the default to none (which is not recommended)
22:14 ari_ joined #puppet-openstack
22:14 dachary so the simples setup would become something like :
22:15 dachary pasting the following in one node:
22:15 dalgaaf and one question about the inifiles: how get's the part from ceph::conf shared between the nodes? Does each node need to call ceph::conf with a long list of possible parameter ? or would they need to inherit from some default ceph node to share the config options?
22:16 dachary ceph::conf::auth { enable: false }; ceph::mon { key: '01983241092834 }; ceph::osd { key: '9843029843' } ?
22:17 dalgaaf dachary: but use some real keys
22:17 dachary that's not as sexy as ceph::mon; ceph::osd; but it would work.
22:17 dachary dalgaaf: yes,  '01983241092834' being a real key
22:18 dalgaaf ceph::conf::auth { enable: false }; ceph::mon { admin_key: '01983241092834 }; ...
22:18 dachary because no disk is provided to ceph::osd, it would just create the directory in /var/lib/ceph/osd/ceph-0/
22:18 dalgaaf and then ceph would get the whole disk?
22:18 dachary no just the directory
22:19 dachary and yes, it would be in a position to use the whole disk if needed
22:19 dalgaaf or at least what's the place provided to /var?
22:19 dachary yes
22:19 dalgaaf okay .. with auth=none it would be the mon key ...
22:20 dalgaaf for the osds: does this only work if you give a key for bootstrap?
22:20 dachary ceph::conf::auth { enable: false }; ceph::mon { admin_key: '01983241092834, mon_key: '0283' }; ceph::osd { boostraph_key: '9843029843' }
22:21 dachary with auth = none, the keys won't matter at all, they will be useless. But since they are required arguments they must be provided.
22:21 dachary or they could be required arguments only if auth is not none, right ?
22:21 dalgaaf not sure if puppet can handle these cases ...
22:22 dachary dalgaaf: not in the prototype probably not, but in the body of the class it can right ?
22:22 dachary ceph::conf::auth { enable: false }; ceph::mon; ceph::osd;
22:22 dalgaaf I mean: parameter X in class Z should be only mandatory if parameter Y is set in class W
22:23 dachary ceph::conf::auth { enable: false }; ceph::mon; ceph::osd; ceph::client;
22:24 dachary /node/ { ceph::conf::auth { enable: false }; ceph::mon; ceph::osd; ceph::client;  }
22:24 dachary and then, from node, you would be able to ceph -s
22:24 dalgaaf and in case the mandatory parameter isn't set, when would puppet fail? before starting the run or in the middle of the run?
22:24 dachary I don't know :-)
22:25 dalgaaf IIRC if you set currently a class parameter to default it would fail before the puppet run on the node and would cause no harm
22:26 dalgaaf not sure if you handle the mandatory in the class body
22:26 dachary /node/ { ceph::conf::auth { enable: false }; ceph::mon; ceph::osd { '/srv/osd1' }; ceph::osd { '/srv/osd2' }; ceph::client;  }
22:26 dalgaaf it would be: /node/ { ceph::conf; ceph::conf::auth { enable: false }; ceph::mon; ceph::osd; ceph::client;  }
22:26 dachary ok
22:27 dachary /node/ { ceph::conf; ceph::conf::auth { enable: false }; ceph::mon; ceph::osd { '/srv/osd1' }; ceph::osd { '/srv/osd2' }; ceph::client; }
22:27 dalgaaf this example would only work under the assumption /srv is there and the dir is there?
22:27 dachary setting up 2 osds as you suggested since pools are created with replica 2 by default
22:27 dalgaaf jep
22:28 dachary ceph-disk has a logic to handle directories instead of disks but I did not look into it precisely
22:29 dalgaaf I mean we could setup whatever simple example we want to put to the doc including some simple instructions ... or what?
22:29 dachary I know they end up being symlinked to /var/lib/ceph/osd/ceph-X so that there is no complexity in handling path in unusual locations
22:30 dachary dalgaaf: yes, that's the idea. Trying to figure out the simplest use case possible and making sure it can be done simply is helpful I think.
22:31 dachary if it ends up being a) install puppet, b) paste this in site.pp and replace /node/ with the name of your current node, c) puppet apply site.pp , d) ceph -s and see that it works
22:31 dachary it will be extra easy for anyone to get started
22:34 dalgaaf or even: /node/ { ceph::conf {auth_enable: false }; ceph::mon; ceph::osd { '/srv/osd1' }; ceph::osd { '/srv/osd2' }; ceph::client; }
22:34 dalgaaf so we don't need ceph::auth at all?
22:35 dalgaaf btw. what would ceph::client do exaclty?
22:35 dachary https://wiki.openstack.org/wiki/Puppet-openstack/ceph-blueprint#Provide_sensible_defaults
22:36 dachary dalgaaf: ceph::client would install the ceph command ( ceph-common ) and setup /etc/ceph/ceph.conf with the IP of the monitors
22:36 dachary sorry
22:37 dachary the sections [mon.X] for each monitor
22:37 dachary so you can ceph -s
22:37 dalgaaf this section would come from ceph::mon
22:37 dachary although, since you're doing all on the same host it's going to be installed already...
22:37 dachary dalgaaf: yes
22:40 dalgaaf confused about ceph::client
22:40 dachary let say we split the use case
22:41 dachary /node/ { ceph::conf; ceph::conf::auth { enable: false }; ceph::mon; ceph::osd { '/srv/osd1' }; ceph::osd { '/srv/osd2' }; }
22:41 dalgaaf ceph::client would only work for ceph -s if you have the client.admin key
22:41 dachary /client/ { ceph::client; }
22:41 dachary dalgaaf: for now let's say auth is disabled
22:41 dalgaaf missed that
22:41 dachary it needs the ip of the mon and nothing else
22:41 dachary hum
22:42 dachary /client/ { ceph::client { mons = [ $ip_of_node ] }; }
22:43 dalgaaf that was my question some pages before
22:43 dalgaaf how do we share the config information between the nodes ?
22:43 dachary we don't
22:43 dachary we duplicate
22:44 dachary $mons = [ mon1, mon2, mon3 ]
22:44 dachary /client/ { ceph::client { mons = $mons }; }
22:44 dalgaaf via /ceph-default/ { ceph::conf } ; /node/ inherits ceph-default {...} ; /client/ inherits ceph-default
22:45 dalgaaf ....
22:45 dachary yes
22:46 dachary and if you want to change an argument for a given node ?
22:46 dalgaaf with concat it was no problem to have these parts shared (global, mon, osd, mds, mon[1-3] ...) ;-)
22:46 dachary I guess that's where https://github.com/bodepd/scenario_node_terminus comes into play
22:48 dachary can you show me an example dalgaaf ?
22:48 dalgaaf is there a case where you wan't to change the config of one single node (in meaning of changing one of the default global sections)?
22:49 dalgaaf not the mon.[1-n] or osd.[0-n] sections of a node
22:49 dachary probably not
22:49 dalgaaf I would say it would be a bad idea
22:49 dalgaaf ;-)
22:49 dachary :-)
22:49 dalgaaf example for what?
22:49 dachary shared parts using concat
22:51 dalgaaf the puppet-ceph module does it ... the global sections are shared (since all inherit from ceph::conf) and the mon.x sections are set by each mon , but end up on all nodes in the config
22:51 dachary oh, but with exported resources, right ?
22:51 dachary @@concat::fragment { "ceph-mon-${name}.conf":
22:51 dachary target  => '/etc/ceph/ceph.conf',
22:51 dachary order   => '50',
22:51 dachary content => template('ceph/ceph.conf-mon.erb'),
22:51 dachary }
22:52 dalgaaf yes ... that's the only place where it's done :-(
22:53 dalgaaf here is a more complex example site.pp: https://github.com/TelekomCloud/puppet-ceph/blob/rc/eisbrecher/examples/example-site.pp
22:54 dachary dalgaaf: I agree with the use of inheritance to reduce duplication where possible. And I've recently come to appreciate a more powerfull way to compose the components with https://github.com/bodepd/scenario_node_terminus in the context of https://github.com/CiscoSystems/openstack-installer/.
22:54 dalgaaf how will we share the mon sections without the need to duplicate config informations?
22:57 dachary I think we just duplicate and leave it to the user to decide which strategy to use to reduce this duplication. But mgagne & bodep know better, I'm just expressing a vague understanding, not an opinion.
22:58 dachary in openstack you have many classes that require an internal IP
22:58 dalgaaf what about ceph::auth? should we drop this class and simply make this part of ceph::conf?
22:59 dalgaaf since both would write in the same sections
22:59 dachary https://github.com/CiscoSystems/openstack-installer/blob/master/data/data_mappings/common.yaml#L51 factorizes all in one parameter
23:00 dachary dalgaaf: ceph::auth or ceph::conf::auth is a helper that translates a boolean into 4 lines. Don't you think it's handy ?
23:01 dalgaaf no since it could also translate to the same in ceph::conf (especially since it make no sense to set these 4 keys to different values)
23:01 dalgaaf I would tend to drop the class and add the parameter to ceph::conf
23:02 dalgaaf what do you think?
23:03 dachary how would that look ?
23:04 dalgaaf ceph::conf {auth_enable = false/true} ... and in the code it would set/unset the same 4 keys
23:04 dachary ah !
23:04 dachary excellent
23:04 xarses joined #puppet-openstack
23:05 dalgaaf okay ... then i drop it from the blueprint and add a comment to ceph::conf
23:05 dachary cool
23:07 xarses joined #puppet-openstack
23:09 xarses1 joined #puppet-openstack
23:10 dachary dalgaaf: regarding packages, why is it necessary ? Users are advised to add the ceph.com repositories by what ever method they usual use for that ( it's very diverse ). And then packages {} is used.
23:10 dachary https://wiki.openstack.org/wiki/Puppet-openstack/ceph-blueprint#package
23:10 tnoor1 joined #puppet-openstack
23:10 dachary I see how something like https://github.com/stackforge/puppet-nova/blob/master/manifests/params.pp#L8 is useful
23:10 dachary beyond that I have the feeling that it's outside of the scope
23:12 dachary I don't think the puppet-openstack modules provide a mean to select the repository from which packages are downloaded
23:13 dachary I would move to change this part just to say that it's up to the user to get the right repository and that the README.md should clearly point to the ceph.com repo + provide an apt {} based exemple to use it
23:13 xarses joined #puppet-openstack
23:14 dalgaaf ceph::conf would do the same as params.pp (+ writing the config)
23:15 dalgaaf sorry ...  I meant this in meaning keeping the config values
23:15 dachary ah :-)
23:15 dalgaaf yes we would need some kind of mapping for the OS and package names
23:15 dalgaaf since they also differ between rpm bases distros and e.g. debian bases distros
23:15 dachary yes
23:16 dachary so no need for ceph::package ?
23:16 dalgaaf they even may differ between S.u.S.e. and RedHat
23:16 dachary yes
23:19 dachary dalgaaf: I updated https://wiki.openstack.org/wiki/Puppet-openstack/ceph-blueprint#package to mention it's user responsibility
23:20 dachary I'll get some sleep now :-)
23:20 dalgaaf cu
23:23 tnoor2 joined #puppet-openstack
23:28 tnoor1 joined #puppet-openstack
23:56 dmsimard joined #puppet-openstack

| Channels | #puppet-openstack index | Today | | Search | Google Search | Plain-Text | summary