Perl 6 - the future is here, just unevenly distributed

IRC log for #puppet-openstack, 2013-06-14

| Channels | #puppet-openstack index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:01 marun joined #puppet-openstack
00:05 marun_ joined #puppet-openstack
00:06 marun_ bodepd: I tried running 'rake spec' but get an error 'gem_original_require'
00:06 marun_ bodepd: I'm afraid I'm not much of a ruby dev
00:07 openstackgerrit Simon Sellar proposed a change to stackforge/puppet-glance: Check if auth_admin_prefix is defined  https://review.openstack.org/32838
00:23 openstackgerrit Maru Newby proposed a change to stackforge/puppet-glance: Fix misnamed protocol param in glance-api.conf  https://review.openstack.org/32977
01:45 michchap_ joined #puppet-openstack
03:18 openstackgerrit Simon Sellar proposed a change to stackforge/puppet-glance: Allow blank keystone flavor in glance configs  https://review.openstack.org/32835
03:34 bodepd marun: did you sort it out?
03:35 bodepd in the wiki, there should be docs about how to run them from buldler
03:44 openstackgerrit A change was merged to stackforge/puppet-openstack_dev_env: clean up the dev_env  https://review.openstack.org/32958
05:26 openstackgerrit Simon Sellar proposed a change to stackforge/puppet-openstack: Storage node now uses local zone variable name.  https://review.openstack.org/32996
07:46 michchap joined #puppet-openstack
07:53 openstackgerrit A change was merged to stackforge/puppet-glance: Check if auth_admin_prefix is defined  https://review.openstack.org/32838
08:06 derekh joined #puppet-openstack
11:02 dprince joined #puppet-openstack
11:53 dmsimard joined #puppet-openstack
11:58 michchap joined #puppet-openstack
12:02 michchap joined #puppet-openstack
12:40 markvoelker_ joined #puppet-openstack
12:47 markvoelker joined #puppet-openstack
13:04 dprince joined #puppet-openstack
13:20 prad_ joined #puppet-openstack
13:31 prad_ joined #puppet-openstack
14:37 michchap joined #puppet-openstack
14:57 openstackgerrit Maru Newby proposed a change to stackforge/puppet-glance: Fix misnamed protocol param in glance-api.conf  https://review.openstack.org/32977
14:59 openstackgerrit A change was merged to stackforge/puppet-glance: Fix misnamed protocol param in glance-api.conf  https://review.openstack.org/32977
15:04 openstackgerrit A change was merged to stackforge/puppet-openstack: Storage node now uses local zone variable name.  https://review.openstack.org/32996
15:27 otherwiseguy joined #puppet-openstack
15:38 michchap_ joined #puppet-openstack
16:10 michchap joined #puppet-openstack
16:37 done__ left #puppet-openstack
16:43 derekh joined #puppet-openstack
16:47 michchap joined #puppet-openstack
16:49 Trozz joined #puppet-openstack
17:06 openstackgerrit A change was merged to stackforge/puppet-openstack: Refresh openstack::all  https://review.openstack.org/32389
17:22 Trozz joined #puppet-openstack
17:24 michchap joined #puppet-openstack
17:33 openstackgerrit A change was merged to stackforge/puppet-openstack: Adding an "openstack::repo" class which automatically sets up repos  https://review.openstack.org/29977
18:03 michchap joined #puppet-openstack
18:41 michchap joined #puppet-openstack
19:13 dprince joined #puppet-openstack
19:20 michchap joined #puppet-openstack
19:36 otherwiseguy joined #puppet-openstack
19:58 otherwiseguy joined #puppet-openstack
19:58 michchap joined #puppet-openstack
20:24 openstackgerrit loganmc10 proposed a change to stackforge/puppet-openstack: Add the ability for the Controller to run cinder-volume  https://review.openstack.org/33106
20:37 michchap joined #puppet-openstack
20:40 openstackgerrit Emilien Macchi proposed a change to stackforge/puppet-nova: Change default libvirt_vif_driver when using Quantum  https://review.openstack.org/33109
21:13 marun bodepd: ping
21:15 michchap joined #puppet-openstack
21:24 bodepd marun: pong
21:25 marun bodepd: I'm afraid I'm new to puppet.
21:25 marun bodepd: What's the best way to provide a value in a manifest that is used on creation but not used to update a resource if it exists (because it's read-only post-creation)?
21:26 marun bodepd: this is in reference to the quantum_network provider btw
21:26 marun bodepd: (pointers to relevant docs welcome)
21:27 marun bodepd: do I just not define a mutator in the provider?
21:29 bodepd it should be a parameter and not a propery
21:29 bodepd property
21:30 bodepd in the type interface, this is the difference between the methods:
21:30 bodepd newparam and newproperty
21:30 marun ok
21:31 bodepd marun: you may just want to google for puppet types and providers
21:31 bodepd marun: there are some docs at puppetlabs.com that shoud be sufficient for getting started
21:31 marun bodepd: I was just reading the guide on custom types
21:32 marun bodepd: should provider/type code use the ruby style guide, or is there something puppet specific
21:32 marun ?
21:33 marun bodepd: ah, so properties can be used in resource creation but don't require synchronization if the resource exists?
21:34 dontalton joined #puppet-openstack
21:37 bodepd ruby style
21:37 bodepd which is more or less the same as puppst style
21:38 bodepd nah, those are the parameters.
21:38 bodepd the properties are synced for resources that already exist
21:38 marun bodepd: oops, got it
21:38 bodepd for getting started, it's easier just to make everything a parameter
21:38 hogepodge joined #puppet-openstack
21:38 bodepd then just update things to be properties (and add the relevent getter/setter methods on the provider when you need them)
21:40 marun bodepd: ok, I'll keep that in mind.
21:41 marun bodeped: Thanks for the help! :)
21:54 michchap joined #puppet-openstack
21:55 openstackgerrit joined #puppet-openstack
21:56 openstackgerrit loganmc10 proposed a change to stackforge/puppet-openstack: Add the ability for the Controller to run cinder-volume  https://review.openstack.org/33106
22:14 marun bodepd: so, a less newbish question...
22:14 marun bodped: ids for openstack resources are set on creation
22:19 marun bodepd: if i want to use, say, the id of a given resource that will be created during a run in creating a different resource, is my only option doing so at the provider level?
22:20 bodepd marun: you have to assume another id as the unique identifier
22:20 bodepd marun: and enforce it
22:20 marun bodepd: for example, if I want to use the id of an openstack tenant, I'd have to pass the tenant name to the provider and ahve it look it up (something like model.catalog.resource("Keystone_tenant[#​{resource[:tenant_name]}]").provider.id)
22:21 bodepd marun: you may want to look at how I did this with the keystone native types
22:21 bodepd but yes, you can look things up with their ids
22:22 bodepd but I would prefer that you ask quantum
22:22 bodepd b/c the resource may not be in the catalog
22:22 bodepd so it's up to you to figure out how to lookup ids from names
22:22 marun bodepd: I can lookup ids from names
22:22 marun but names are not unique in the openstack world
22:22 marun Are you familiar with how this is handled in devstack?
22:22 bodepd the best thing to do is, that if you find duplicates: fail
22:23 bodepd fail("Entities must have unique names if you want to manage them with puppet")
22:23 marun bodepd: fair enoug
22:23 bodepd guids are just shit for managing things
22:23 bodepd and unique predictable names are already a requirement for the other puppet types
22:23 marun bodeped: sure, but in a distributed system, guaranteeing name uniqueness otherwise is worse
22:24 openstackgerrit loganmc10 proposed a change to stackforge/puppet-openstack: Add the ability for the Controller to run cinder-volume  https://review.openstack.org/33106
22:24 bodepd guids are freaking just develop laziness
22:24 marun bodepd: So, how do you guarantee uniqueness for distributed writes?
22:24 marun bodepd: I know the problem that guid's solve, i'm less sure of the alternatives
22:24 bodepd so do I, but I think it creates another fundamental problem
22:25 bodepd if I have a region called us-ca-1
22:25 bodepd then I have two of them
22:25 bodepd which one am I referring to?
22:25 marun bodepd: that's not relevent, I'm afraid
22:25 marun bodepd: we're talking about user-managed vs system-managed resources
22:25 marun a region is system managed and can be trivially name-unique
22:25 bodepd or even worse (from a puppet perspective) how can you initialize the configuration of them
22:26 marun Don't get me wrong, I can't defend the way openstack manages consistency
22:26 bodepd marun: it's also kind of a theoretical debate
22:26 marun guids are kind of besides the point if a db is used to maintain consistency
22:27 marun But I don't think this problem is going away and it would be helpful if puppet could deal with it better rather than punting
22:27 bodepd marun: the downside of guids is that they make things extremely hard to manage
22:27 bodepd and populate
22:27 bodepd how would it solve it?
22:27 marun Sure, if the tool used to manage doesn't fit the problem being solved
22:27 marun I'm not saying I have a solution - it's a hard problem
22:27 marun I just think it's one worth solving
22:28 marun (eventually) :)
22:28 openstackgerrit Donald Talton proposed a change to stackforge/puppet-cinder: added ceph backend support  https://review.openstack.org/33132
22:28 marun i'm too new to puppet to have much insight beyond 'this is a bit hackish'
22:28 bodepd I understand the usage of guids for consistency, but I consider that to be punting on the problem instead of solving it :)
22:29 openstackgerrit A change was merged to stackforge/puppet-swift: Add swift::storage::disk to support disk device.  https://review.openstack.org/26723
22:29 marun well, given that every distributed system i've encountered uses guids for one reason or another, i'm not sure you can expect their use to disappear anytime soon
22:29 marun regardless of their wisdom...
22:29 marun anyway, thanks for the help.
22:30 bodepd I guess that in my interaction with distributed systems, I always wind up picking a second id (that can be predicted) and mapping it to the guids
22:30 bodepd no worries.
22:30 marun hmmm
22:31 marun bodped: in terms of looking up ids, i may have an issue...
22:31 marun bodepd: i need quantum to be able to lookup the tenant id from keystone
22:31 marun bodeped: (or i can cheap out and use the catalog)
22:32 marun bodepd: The latter approach would seem much simpler (and less coupled), what are your thoughts
22:32 marun ?
22:33 michchap joined #puppet-openstack
22:35 bodepd hmm. trying to think about which objects have access to which other objects and scopes :)
22:36 bodepd I think is reasonable to look it up in the catalog
22:36 bodepd you may want to look it up via:
22:37 bodepd Puppet::Type(:keystone_tenant).instances
22:37 bodepd to ensure that it always rebuids the memoized representation of the objects
22:37 bodepd otherwise, you would wind up with more complicated logic of
22:38 bodepd if the thing is in the catalog, get it from there, otherwise, look it up directly
22:38 bodepd I do remember that they were some performance issues related to the way some things in keystone were retrieved.
22:38 bodepd I keep meaning to look into it, but it never quite gets prioritized
22:39 bodepd let me see how many api calls that methods makes
22:39 marun ok
22:39 bodepd marun: by the way. seriously awesome questions.
22:39 bodepd marun: it's good to get more developers in the code
22:39 marun bodepd: Seriously awesome to have someone to answer my questions! :)
22:39 bodepd I'm the only one that knows way too much of this stuff, and some of it was written in haste
22:40 bodepd (esp the types and providers)
22:40 bodepd you should just be able to run Puppet::type(:keystone_tenant).instnaces
22:40 bodepd in irb and introspect the results to see what the output looks like.
22:40 bodepd if you set Puppet[:debug] = true
22:40 bodepd you can also see how many system calls it makes to look everything up.
22:41 bodepd please open a ticket if it's an unreasonable numbr
22:41 bodepd and I can try to find my way back to that part of the code
22:41 marun ok
22:41 marun I'm afraid, though, that i'm a ruby newb
22:42 marun if i run irb do i have to import puppet (or something) before invoking that command?
22:42 bodepd require 'puppet'
22:42 marun ok
22:42 bodepd depending on the version, you may have to run some initialization of Puppet
22:42 bodepd I think it's Puppet.parse_config
22:42 marun are there docs somewhere?  i really am starting from zero
22:43 bodepd send me an email: bodepd@gmail.com and I'll see what I can send you
22:43 marun ok, will do
22:45 marun invoking Puppet::type(:keystone_tenant).instances, I get ArgumentError: wrong number of arguments (1 for 0)
22:55 openstackgerrit Maru Newby proposed a change to stackforge/puppet-quantum: Add quantum_network custom type.  https://review.openstack.org/31087
23:02 marun bodepd: Ah, I see the problem.  Relying on the catalog assumes that the host running puppet is running those manifests.
23:03 marun bodepd: This is problematic in a multi-node openstack deployment, since the providers all assume that the service config will be locally available.
23:06 marun bodepd: My brain hurts - if I need to upload a glance image on the glance server, and create networks on the quantum server, and then lookup the ids of the resources created to configure tempest on a 3rd server, what is going to make sure this happens in order?
23:11 marun bodepd: ah, I guess packstack will have to take care of that detail.
23:11 marun bodepd: I'll need to modularize the tempest class to allow for this, but that's easy enough.
23:12 michchap joined #puppet-openstack
23:19 bodepd marun: I would like for there to just be a role of machine that can handle all of this kind of stuff
23:19 bodepd something that can just auth through keystone and take care of the management of these things
23:20 marun bodepd: But don't the providers require local conf files at present?
23:20 bodepd yep.
23:20 bodepd and interact with command line utilities
23:20 bodepd this is more of a rewrite of the providers.
23:20 marun bodepd: The way the packages work I don't think those conf files get installed without installing the services :(
23:20 bodepd I was looking into fog
23:20 marun bodepd: ah, fair enough
23:20 bodepd and also some ruby client bindings from redhat
23:21 bodepd I have some examples that I wrote last year that use fog to implement resources for cloudstack
23:21 marun bodepd: I think that will have to be round 2
23:21 bodepd I agree.
23:22 marun bodepd: the immediate need is for tempest testing of single-node packstack.
23:22 bodepd I tried to talk mgange into doing that
23:22 bodepd ah. gotcha.
23:22 marun bodepd: and i can focus on multi-node immediately after
23:23 bodepd if you want to do the absolute minimem :)
23:23 marun bodepd: In that case, is it ok that I take the lazy road for this first pass of the quantum providers?
23:23 bodepd https://github.com/stackforge/puppet-open​stack/blob/master/manifests/test_file.pp
23:23 bodepd https://github.com/stackforge/puppet-openstac​k/blob/master/templates/test_nova.sh.erb#L40
23:24 bodepd I have a script that does some basic quantum setup.
23:24 bodepd I've just been testing ovs GRE tunnels, so it may not work for anything else
23:24 marun bodepd: ah, not sure why i'm going through the trouble of working on quantum providers then :(
23:24 bodepd if it's just for testing, then this is sufficient.
23:25 bodepd these resources are for the actual configuration to bootstrap networking components
23:25 bodepd did you look at tempest?
23:25 marun bodepd: that is my target
23:26 bodepd oh, tempest does not setup the quantum bits
23:26 bodepd ?
23:26 marun bodepd: How about I create another version of this script that targets quantum?
23:26 bodepd you know this exists? https://github.com/stackforge/puppet-tempest
23:26 marun bodepd: yes
23:26 bodepd marun: it has a flag for quantum.
23:26 marun bodepd: it doesn't support quantum
23:27 bodepd marun: but this script is menat to be for quick and dirty testing
23:27 marun bodepd: because it doesn't do the base setup
23:27 bodepd the assumption is that you configure the machiens into their roles.
23:27 bodepd use opensack::auth_file and openstack_test_file
23:27 bodepd to deploy the openrc and this test script.
23:27 bodepd then run it from /tmp/test_node.sh
23:27 bodepd test_nova.sh
23:28 bodepd for the integration into basic setup for tempest, I would much prefer native resources
23:28 marun bodepd: That's what I'm aiming for
23:28 bodepd ok. good to understand :)
23:29 marun bodepd: the glace fix was to allow the tempest module to upload an image
23:29 marun bodepd: and i'm working on the quantum providers so that the tempest module could leverage them
23:29 marun bodepd: I'm hoping I can get it done fast enough to be useful - I have until Tuesday to have something working
23:29 marun bodepd: For single-node, at least, this should work
23:30 marun bodepd: For multi-node, maybe the quick-n-dirty script needs to be extended to support tempest until a provider rewrite can happen.
23:31 bodepd ah, the protocol fix?
23:31 bodepd well, let me know if you want to do the types/providers.
23:31 bodepd from my perspective this is hte right way
23:32 bodepd but I understand it will be more work
23:32 bodepd do you need to support per tenant resources initially?
23:33 marun not sure what you mean?
23:34 marun bodepd ^
23:34 bodepd you were talking about retrieving tenant objects from keystone to configure quantum.
23:34 marun right.
23:34 bodepd I assume that is b/c you need network objects assigned to tenants?
23:35 marun yes
23:35 bodepd that is a requirement for quantum?
23:35 bodepd gotcha.
23:35 marun It's not strictly a requirement for getting tempest working
23:35 marun I should relax that requirement for now
23:35 marun it's more for creating an environment where a user can immediately boot a vm post-boot, no extra setup required
23:36 marun i.e creating a tenant-owned net and subnet to support this
23:37 marun So given the right credentials fog can replace the openstack clients?
23:38 marun dodepd: And how do you imagine a given provider will acquire the necessary config/credentials for fog to work?
23:40 bodepd for cloudstack, I created a file called ~/puppet/transport.yaml
23:40 bodepd which help the necessary credentials.
23:40 bodepd for comes with its own config file for credntials as well
23:40 bodepd ~/fog.conf or something...
23:42 marun bodepd: ok, so a file written to disk?
23:43 marun bodepd: this may be the way to go sooner than later
23:44 marun bodepd: it would allow the tempest module to target an already deployed cloud provided it had the right credentials
23:46 marun bodepd: in any case, what do I need to add in the way of spec tests?  The quantum_network type is pretty simple...
23:51 michchap joined #puppet-openstack
23:58 bodepd marun: I would love to find time to work on this. It's been in the back of my mind as something that needed to be done.
23:58 bodepd I started a prototype somewhere...
23:58 bodepd that I need to dig up.

| Channels | #puppet-openstack index | Today | | Search | Google Search | Plain-Text | summary