Perl 6 - the future is here, just unevenly distributed

IRC log for #puppet-openstack, 2014-01-27

| Channels | #puppet-openstack index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
23:26 ryanycoleman joined #puppet-openstack
23:39 xarses Is there a method in puppet where we can stop the agent/apply in the event of a resource failed and is causing other deps to be skipped?
23:41 michchap joined #puppet-openstack
00:33 werebutt joined #puppet-openstack
00:33 werebutt left #puppet-openstack
01:37 openstackgerrit Mathieu Gagné proposed a change to stackforge/puppet-ceilometer: Deprecate ceilometer::keystone::auth params in favor of *_url  https://review.openstack.org/69006
01:53 openstackgerrit Mathieu Gagné proposed a change to stackforge/puppet-ceilometer: Deprecate ceilometer::keystone::auth params in favor of *_url  https://review.openstack.org/69006
02:10 xarses joined #puppet-openstack
02:13 starmer joined #puppet-openstack
02:32 morazi joined #puppet-openstack
02:40 ryanycoleman joined #puppet-openstack
02:46 ilbot3 joined #puppet-openstack
02:46 Topic for #puppet-openstack is now Place to collaborate on Puppet/OpenStack tools: logs at http://irclog.perlgeek.de/puppet-openstack/today
03:09 ryanycoleman joined #puppet-openstack
03:43 saneax joined #puppet-openstack
07:17 starmer joined #puppet-openstack
07:27 dgollub joined #puppet-openstack
08:09 EmilienM good morning o/
08:44 werebutt joined #puppet-openstack
08:44 werebutt left #puppet-openstack
09:18 derekh joined #puppet-openstack
09:23 piliman974 joined #puppet-openstack
09:39 mattymo hey EmilienM got any experience using postgres for deployment?
09:59 beddari .. a conversation and question I'd love to listen in on, I _want_ to use PostgreSQL (as that is what we got inhouse knowledge about) but I haven't spent time on that yet
10:04 michchap I ran a Folsom release cloud on pgsql back in the Folsom days. I never noticed anything not working as expected, if that's what you're worried about.
10:05 mmagr joined #puppet-openstack
10:28 EmilienM mattymo: not yet, sorry. How could I help?
10:32 BroadcastStorm joined #puppet-openstack
10:32 mvwensveen joined #puppet-openstack
10:32 mattymo michchap, cool
10:33 mattymo we have a request to support postgres
10:33 mattymo galera is great except when anything happens to the network or if all servers get powered off at the same time
10:33 EmilienM indeeed
10:34 mattymo we spend a lot of our support time working on correcting galera after these two incidents
10:35 EmilienM mattymo: dummy question, but does pgsql support multi master ?
10:37 mattymo pgpool, we plan to use http://www.pgpool.net/mediawiki/index.php/Main_Page
10:38 EmilienM mattymo: interesting, thank you
10:39 beddari well PostgreSQL won't solve or better any of those problems you have with Galera I'd think
10:39 mattymo we need it for some other projects aside from using it as a DB backend for openstack, but it's just good to know who else has experience
10:39 mattymo beddari, care you elaborate?
10:39 mattymo s/you/to
10:39 beddari not really, I just know that those exact problems (network outages, etc) are hard also with PostgreSQL :)
10:40 beddari and the best you can do is to try and never have them hehe, e.g not a two-node cluster, always use three nodes at least and three locations
10:40 mattymo we had a really awesome learning experience last week with OVS creating multiple switch loops across vlans and taking down the switches
10:41 mattymo and how galera prefers to react
10:41 beddari sure, but forgive me for asking, you DID have three nodes, three locations, good connectivity .. so as to hopefully never lose quorum vote
10:42 mattymo 3 nodes with quorum. these hosts had great connectivity (2x 10G nics with lacp bonding)
10:42 mattymo but the switch was brought down by 4 other nodes on the same switch
10:43 beddari so my (stupid) answer, "not enough separation"
10:43 mattymo I am inclined to agree
10:43 beddari the point is I don't think PostgreSQL will help (much) with these ..
10:44 beddari unless you are better at stiching data with it than with Galera
10:44 beddari PostgreSQL-XC is a multi-master variant … but I think you'd always have issues like this using anything multimaster really :)
10:44 mattymo we have a network for provisioning of nodes (as opposed to galera/storage/API traffic). we're thinking of trying to attach the cluster on both IPs so that it can fall back to the slower network if one goes down
10:44 mattymo I don't know if it's supported though
10:45 beddari is it possible to set up "dummy" vote-only nodes with Galera?
10:45 beddari that could make it easier to be more resilient
10:45 mattymo I think that's another great option
10:48 mattymo beddari, does that ever solve the all 3 nodes shut down at once solution?
10:48 beddari well pgpool isn't really mulitmaster is it? the guys that are writing postgresql-xc says that trying to id write queries are actually unsolvable
10:48 beddari I don't know
10:48 mattymo beddari, multimaster is just a convenient solution to minimize convergence when a node dies
10:49 beddari sure ..
10:49 mattymo oh, who is going to FOSDEM this week?
10:49 beddari only cfgmgmtcamp :)
10:50 mattymo aaah I didn't see this
10:50 mattymo I'm speaking about Fuel at CentOS Dojo
10:53 beddari yeah you are producing code quicker than I can follow haha
10:55 mattymo we're still not merged to puppet-openstack though :( I started work on it on Dec
10:56 beddari I wrote a line about something similar and deleted it again hehe
10:56 beddari not being backed by any vendor I'm hoping eventually provisioning and deploy will be general openstack projects
10:56 mattymo so Ironic has a lot of promise
10:56 beddari issue #1 would be contibuting back to what exists already
10:56 beddari yeah
10:57 beddari but puppet-openstack is deprecated afaik, so I don't know if that is of much use
10:57 mattymo I meant puppet-*
10:57 beddari yeah :)
10:58 mattymo nova, glance, cinder, ceph, swift, neutron...
10:58 beddari getting to know them better know and well they're not exactly best practice
10:58 beddari e.g missing anchoring
10:59 beddari so I can't always say in a profile class that I want to depend on this and that already being done
11:00 mattymo we have a lot of stylistic demands in this group, so any changes require really hardcore tuning. I think it's too late to add anchors and get our core reviewers to consider it
11:00 beddari sad
11:02 beddari the real value are trying to get the types and providers in the best possible state, the rest … well ;)
11:02 beddari writing more tests
11:02 beddari neutron needs a LOT more
11:02 beddari more idempotence testing
11:02 beddari ++ :)
11:03 beddari but as you said, this is neigh on impossible to do and keep backwards compatibility
11:04 beddari hopefully there will be a provisioning + deployment track at the upcoming summits
11:06 mattymo at summit we said backwards compatibility isn't critical, but versioning needs to reflect minor/major changes
11:06 beddari yeah, agree
11:06 mattymo so if you break backwards compatibility, you just have to increment to a new major version
11:11 michchap beddari: The deployment track is tripleo, it's unlikely to have any puppet content
11:12 mattymo I sat in some of their sessions. They argued that they won't pick puppet (or chef) because any choice will alienate the other group(s)
11:12 beddari kind of strange, right? as the core puppet modules are a vital component for lots of projects
11:12 mattymo therefore, enormous linear bash scripts will do all last-minute customizations via heat
11:12 michchap beddari: The summits sessions are voted on and the short answer is that despite puppet-openstack having the most users bodepd wasn't able to get a session for any of the puppet-openstack work
11:12 beddari :(
11:12 beddari so strange
11:12 mattymo so the biggest opponent to puppet-openstack is Crowbar project from Dell
11:13 michchap so we will have the same problem next time. I talked to some people about using puppet to do the config and standard tripleo to do the deployment, but the work hasn't been done. Derek Higgins from RedHat was trying it
11:18 mattymo yes, that approach involves using whatever tools you want to prepare images
11:18 mattymo and puppet fills out config files with placeholder values
11:18 beddari oh no :P
11:19 beddari my thinking does not lend itself easily to image-backed approaches
11:21 beddari an "image" for the core OS, sure .. but then I'd possibly run the imaging tool at pxe time and build it do disk, no ? ;-)
11:22 mattymo beddari, there are a few approaches on the table here
11:22 mattymo imagine you have 1 provisioning master... or even 100
11:22 mattymo imagine you want to deploy 10,000 hosts and you know exactly what config goes on every single one... and for the majority, the configs are identical except for hostname
11:23 mattymo you can prepare as many or as few images that are pre-configured that just need to be uploaded
11:23 mattymo a direct image can be sent faster than OS install + puppet run
11:24 mattymo those images could be prepared with a base core OS image, mount it in a VM, run puppet on it, then record that new image
11:24 beddari well I'm not buying the immutable server dream, but yeah I get what you are saying, that is valid
11:28 beddari "They argued that they won't pick puppet (or chef) because any choice will alienate the other group(s)" <- that is just unpalatable
11:29 mattymo go ahead and try to fix it :)
11:29 fvollero EmilienM: Yep :) What a nice sunday right? :)
11:30 beddari mattymo: technically it would be easy to make a pluggable solution
11:32 beddari mattymo: that single decision may actually kill the whole idea I think
11:36 michchap If you write a pluggable solution and implement puppet as an image element, the patch will not be opposed.
11:36 michchap (to tripleo)
11:39 michchap The image based approach avoids some of the complications of keeping updates in sync across large clusters. I like it. Puppet as a deployment tool for packages is not ideal, pushing them out as part of images ensures that every node is identical even if a package mirror is updated mid-push.
11:40 mattymo I wish puppet would let you do bulk package installs
11:41 michchap I think puppet can add value after the image has been rolled out to ensure that the config is set correctly in a sane way, rather than expecting file templating to be adequate for the complexity of config options across a real openstack cluster
11:41 mattymo calculate all the packages required in the catalog and do them all in one operation
11:41 michchap mattymo, like for image building?
11:41 mattymo in general
11:41 michchap you can query the catalog
11:41 michchap extract all the package resources
11:43 beddari yup ...
11:43 michchap oh I see what you're asking...that's different
11:44 mattymo for yum, you're wasting 3-5 seconds for every yum call
11:44 beddari well it is not very hard to write a simple type/provider to do that
11:44 fvollero fc__: ping
11:44 michchap actually there's a couple of things I was thinkign about along those lines...you could in theory split package operations into download and install, and do the download in parallel across many threads, and then install one by one as they come in
11:44 beddari I think actually there is one already mattymo
11:44 mattymo beddari, if you can find it, that would be great
11:48 beddari mattymo: I might, there's examples around that code in the types and providers book
11:49 beddari mattymo: but you'd need to do some hacks to have it batch yum calls
11:50 mattymo https://projects.puppetlabs.com/issues/2198 looks like they started work on it but stopped
11:50 mattymo they have so many tickets that get left to die
11:51 mattymo Hunner promised to get someone to look at this one: https://tickets.puppetlabs.com/browse/PUP-897
11:51 mattymo it's a one line fix
11:52 mattymo patch here: http://projects.puppetlabs.com/issues/5456
11:52 beddari yeah thanks :) I've read those earlier
11:52 beddari their migration to Jira is messy ..
11:54 beddari hehe there WAS a patch yeah, and I've seen it floating around in some modules as a custom yum provider
12:12 EmilienM fvollero: o/
12:13 fvollero EmilienM: So, Emilien, I covered your reviews or still someone out ? :)
12:14 EmilienM fvollero: i think that's ok :-)
12:14 fvollero EmilienM: Ok, waiting for my bottle of Calvados :)
12:14 EmilienM fvollero: sure, but where I come from, it's more white wine ;-)
12:15 fvollero EmilienM: Lol :) Ok, so you will find something suitable :)
12:32 gcha joined #puppet-openstack
12:35 openstackgerrit A change was merged to stackforge/puppet-ceilometer: Remove log_dir from params and make logs configurable in init  https://review.openstack.org/68202
13:14 bcrochet joined #puppet-openstack
13:16 morazi joined #puppet-openstack
13:16 markvoelker1 joined #puppet-openstack
13:20 csschwe joined #puppet-openstack
13:23 rcrit joined #puppet-openstack
13:24 rcrit left #puppet-openstack
13:35 britthouser joined #puppet-openstack
13:59 finnx_ joined #puppet-openstack
14:05 dprince joined #puppet-openstack
14:13 piliman974 joined #puppet-openstack
14:13 finnx_ joined #puppet-openstack
14:27 dgollub joined #puppet-openstack
14:54 prad_ joined #puppet-openstack
15:02 morazi_ joined #puppet-openstack
15:03 otherwiseguy joined #puppet-openstack
15:05 frankbutt joined #puppet-openstack
15:05 frankbutt left #puppet-openstack
15:09 fvollero dgollub: hi :) how things going in d-tel ? :)
15:13 dgollub good. dealing with lots of performance challenges. how are things at RH?
15:15 pzimmer joined #puppet-openstack
15:16 rwsu joined #puppet-openstack
15:30 fvollero dgollub: Usual :) Trying to make packstack working and getting ready to sync our puppet stuff with foreman :)
15:36 mattymo fvollero, are you also working on Astapor?
15:36 fvollero mattymo: i'm working on openstack-puppet-modules with mmagr
15:37 fvollero mattymo: Astapor is morazi_ domain :)
15:38 mattymo oh ok
15:43 mgagne joined #puppet-openstack
15:47 xarses joined #puppet-openstack
15:50 fvollero mattymo: you're working on astapor or you know someone that work on it ?
15:52 badiane_ka joined #puppet-openstack
15:57 angdraug joined #puppet-openstack
15:58 mgagne joined #puppet-openstack
16:03 mattymo fvollero, no :) Just ran into it by accident
16:03 mattymo it's quite similar to Fuel (which I work on) and I'm just curious to learn a little more about its strategy
16:03 mattymo and I'm interested to know why RH wants to develop 3 tools for deploying OpenStack
16:04 dmsimard joined #puppet-openstack
16:12 fvollero mattymo: which 3 tools ? :)
16:13 fvollero mattymo: packstack it happened to be just a PoC that later become the fastest way to deploy openstack, but I think gonna be replaced in the future with foreman
16:14 mattymo tripleo (and its related projects), packstack, and astapor
16:14 fvollero mattymo: and tripleo is not yet production ready afaik, morazi, please correct me if i'm wrong
16:14 fvollero astapor == foreman
16:14 mattymo its description says it's for installing openstack via foreman
16:21 fvollero mattymo: exactly, it's a set of puppet modules to work like packstack
16:24 openstackgerrit Emilien Macchi proposed a change to stackforge/puppet-neutron: lbaas: use stdlib to manage haproxy package  https://review.openstack.org/69399
16:24 EmilienM fvollero: one interesting review if you have time ^
16:24 EmilienM mgagne: sorry, I just read your comment on the bug. Could you check if my patchset is valid then? If not, let me know.
16:26 fvollero EmilienM: ack :) take a look in a while
16:28 mgagne EmilienM: IMO, openstack puppet modules shouldn't manage resources that could be handled by other modules as it could result in conflicts. ensure_packages uses ensure => present. it could conflict with someone using ensure => latest somewhere else.
16:29 EmilienM mgagne: so wat? I should delete the HAproxy resource check everywhere? In that case, we should do the same for dnsmasq, VPN, etc
16:29 mgagne EmilienM: what do you mean?
16:30 EmilienM mgagne: HAproxy is an example, but vpnaas, dhcp agents also ensure some 3rd party packages are installed
16:30 mgagne puppetlabs-haproxy doesn't allow you to specify latest, so might not be that big of an issue
16:31 mgagne EmilienM: that idea is that our modules should try to be as minimal as possible to avoid those conflicts.
16:31 mgagne EmilienM: you can declare a dependency and not declare the package resource.
16:31 EmilienM mgagne: make sense. My concern is having dupplicate resource when declaring HAproxy class & neutron lbaas on the same node. Just let me know what we need to do
16:32 EmilienM ok
16:32 EmilienM mgagne: make sense, thank you. I'll do another patchset
16:32 EmilienM fvollero: WIP
16:34 mgagne EmilienM: alternative solution: add a parameter: install_haproxy/manage_haproxy_package/whatever and default it to true so people could decide if they want lbaas.pp to manage the package or not.
16:34 EmilienM mgagne: declare a dependency and not declare the package resource seems to be better, isn't it ?
16:35 mgagne EmilienM: depends on what experience we want to give to the end user.
16:37 EmilienM mgagne: we are end users, that's why I like discuss here
16:38 mgagne EmilienM: yes and no, we are advanced end users =)
16:38 EmilienM at least we try!
16:39 mgagne EmilienM: we know (I suppose) the inner working of puppet and how it can be used/tweaked. Other end users reading puppetlabs blog about the new release of whatever puppet modules for openstack might not.
16:41 EmilienM mgagne: i don't want that end user would have to take care of HAproxy package. I think the stdlib solution is not so bad. I may be wrong. Did you look at my patch already?
16:42 mgagne EmilienM: yes
16:42 mgagne EmilienM: ensure_packages should be ok for now. puppetlabs-haproxy doesn't allow you to use ensure => latest anyway.
16:45 openstackgerrit Emilien Macchi proposed a change to stackforge/puppet-neutron: lbaas: use stdlib to manage haproxy package  https://review.openstack.org/69399
16:45 EmilienM fvollero: mgagne ^ open for reviews :-)
17:09 openstackgerrit Emilien Macchi proposed a change to stackforge/puppet-neutron: lbaas: use stdlib to manage haproxy package  https://review.openstack.org/69412
17:09 EmilienM mgagne: and the backport ^
17:09 mgagne k
17:12 rmoe joined #puppet-openstack
17:13 morazi joined #puppet-openstack
17:13 mgagne fc__: how should we proceed to deprecate the version parameter?
17:14 tnoor joined #puppet-openstack
17:14 bauzas joined #puppet-openstack
17:15 starmer joined #puppet-openstack
17:15 fc__ mgagne: well … just set it to undef as the others and add it to the url ? do you see any other option ?
17:16 mgagne fc__: I see challenges. If the guy is already providing public_url and didn't append the version to the URL, how do we inform the user that he needs to append the version to the url?
17:16 tnoor1 joined #puppet-openstack
17:17 fvollero mgagne: fc__ : about the 68433 the hash comment was half right :)
17:18 fc__ mgagne: indeed … didn't see this one coming
17:18 mgagne fvollero: what is wrong with it?
17:19 mgagne fc__: should we grep the url for the version?
17:21 openstackgerrit A change was merged to stackforge/puppet-neutron: lbaas: use stdlib to manage haproxy package  https://review.openstack.org/69399
17:21 mgagne fc__: or add a new parameter: append_version with true as default value. the guy would have to set it to false once the version is added to the public_url parameter.
17:21 mgagne fc__: and add a warning if version is provided
17:22 fc__ mgagne: then what if "the guy" hides keystone behind some reverse proxy https://identity.example.com/ that redirects to https://identity.internal/v2.0 ? The endpoint should be without a version in that case
17:23 fc__ mgagne: I'm not sure if keystoneclient wouldn't complain though …
17:25 mgagne fc__: that's why I suggest using append_version instead.
17:26 fc__ mgagne: yep, should be a good option
17:26 mgagne fc__: and later on then: remove version parameter and noop append_version with a warning
17:27 fvollero mgagne: i am talking about my comment, that it will return an array and not an hash, so need to be wrapped inside Hash[]
17:27 mgagne fvollero: =)
17:29 fvollero mgagne: nobody should think about work on sunday :)
17:31 mgagne =)
17:33 openstackgerrit A change was merged to stackforge/puppet-neutron: lbaas: use stdlib to manage haproxy package  https://review.openstack.org/69412
17:39 marun joined #puppet-openstack
18:03 thumpba joined #puppet-openstack
18:04 Hunner mattymo: :(. I've pinged people 3 times about that :/
18:06 mgagne I suggest we rename the connection parameter for database_parameter here: https://github.com/stackforge/puppet-neutron/blob/master/manifests/server.pp#L121
18:06 mgagne to be consistent with other modules
18:07 tnoor joined #puppet-openstack
18:23 mgagne anyone had this problem with Horizon? 'AnonymousUser' object has no attribute 'backend'
18:25 mgagne found it, forgot to update python-keystoneclient
18:56 marun joined #puppet-openstack
19:29 benh57 joined #puppet-openstack
19:30 benh57 howdy
19:32 benh57 has anyone made a version of puppetlabs-nova which works with rabbitmq > 3.0.0 ?
19:32 dmsimard1 joined #puppet-openstack
19:33 hogepodge joined #puppet-openstack
19:34 mgagne benh57: I wrote my own manifest for that one, much more flexible than the one provided by puppet-nova. =)
19:34 dmsimard joined #puppet-openstack
19:35 mgagne benh57: and mostly stripped it done to: include rabbitmq and user/permissions. rabbitmq class is configured through hiera.
19:37 benh57 hmm, ok
19:37 benh57 being both a puppet and openstack newbie, trying to use as much out of the box stuff as i can, but that may not be possible.
19:38 mgagne benh57: I see where you are coming from =)
19:38 benh57 looks like it might not be too hard for me to modify puppetlabs-nova to work with rabbitmq > 3.
19:40 angdraug joined #puppet-openstack
19:40 mgagne benh57: we had some challenge moving to rabbitmq > 3. puppetlabs-rabbitmq got refactored multiple times in backward incompatible ways which made it hard/impossible for us to upgrade without breaking stuff.
19:41 benh57 i'd happily use the old one but pre-existing internal modules depend on the latest rabbitmq :/
19:43 mgagne benh57: yha, that's the problem with puppet modules... they need to upgrade all together to the latest version of the dependencies or you end up in this mess that is conflicting dependencies.
19:59 benh57 joined #puppet-openstack
20:01 mjblack joined #puppet-openstack
20:01 benh57 joined #puppet-openstack
20:06 openstackgerrit joined #puppet-openstack
20:29 dsockwell joined #puppet-openstack
20:39 dmsimard joined #puppet-openstack
21:27 hogepodge joined #puppet-openstack
21:28 openstackgerrit Emilien Macchi proposed a change to stackforge/puppet-swift: dispersion: add endpoint_type parameter  https://review.openstack.org/69472
21:38 dmsimard1 joined #puppet-openstack
21:45 prad joined #puppet-openstack
21:54 ianw joined #puppet-openstack
21:56 mjblack ugh
21:56 mjblack https://github.com/stackforge/puppet-horizon/commit/fae0d04b82867dab3c355e5f1d5a9b2d0054f949
21:56 mjblack without that it looks like horizon has css issues
21:58 dmsimard joined #puppet-openstack
22:06 starmer joined #puppet-openstack
22:11 ianw joined #puppet-openstack
22:16 mgagne mjblack: which platform?
22:16 mjblack ubuntu
22:17 mgagne mjblack: I had the same issue today. This review should fix it: https://review.openstack.org/#/c/68977/
22:18 mjblack interesting, do we know why  its only needed for ubuntu?
22:18 starmer joined #puppet-openstack
22:18 mjblack lol
22:19 mjblack I probably should have read the commit message before hitting enter
22:35 hogepodge_ joined #puppet-openstack
22:49 mgagne rspec tests for horizon are a mess :'(

| Channels | #puppet-openstack index | Today | | Search | Google Search | Plain-Text | summary