Perl 6 - the future is here, just unevenly distributed

IRC log for #puppet-openstack, 2013-07-18

| Channels | #puppet-openstack index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:11 digicloseup joined #puppet-openstack
00:45 dmsimard joined #puppet-openstack
02:33 openstackgerrit Yu Xingchao proposed a change to stackforge/puppet-swift: Add configurable log for storage server  https://review.openstack.org/34319
03:00 openstackgerrit Dan Bode proposed a change to stackforge/puppet-openstack: forward additional tempest parameters  https://review.openstack.org/37611
03:21 michchap joined #puppet-openstack
06:25 marun joined #puppet-openstack
07:14 openstackgerrit Michael Chapman proposed a change to stackforge/puppet-quantum: Support Cisco quantum plugin  https://review.openstack.org/36830
07:28 otherwiseguy joined #puppet-openstack
07:44 michchap joined #puppet-openstack
08:12 derekh joined #puppet-openstack
08:37 koudaddy_ joined #puppet-openstack
08:40 koudaddy_ joined #puppet-openstack
10:09 mkollaro joined #puppet-openstack
11:00 openstackgerrit Francesco Vollero proposed a change to stackforge/puppet-swift: Added the allow_versions flag to enable/disable object versioning in swift object container  https://review.openstack.org/37493
11:02 dprince joined #puppet-openstack
13:00 dprince joined #puppet-openstack
13:24 prad joined #puppet-openstack
13:35 dmsimard joined #puppet-openstack
13:46 dmsimard1 joined #puppet-openstack
14:01 dmsimard joined #puppet-openstack
14:06 derekh joined #puppet-openstack
17:07 openstackgerrit joined #puppet-openstack
17:42 blkperl bodepd: so installing openstack from puppet doesn't seem to setup the AMQP server correctly
17:42 blkperl errr or maybe those are old messages...
17:42 blkperl either way I'm missing the ability to create instances
17:44 * blkperl goes to debug rabbit
18:26 bodepd blkperl: can you be more specific
18:26 bodepd blkperl: there are always some rabbit failures b/c the packages start the services
18:27 bodepd blkperl: feel free to write a logstack module when you realize how much of a pain openstack it to debug
18:36 blkperl bodepd: :( somethign about 2013-07-18 10:43:25.592 21334 ERROR nova.openstack.common.rpc.common [-] AMQP server on 131.252.208.124:
18:36 blkperl 5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 1 seconds.
18:37 bodepd blkperl: those erros don't necessarily mean anything
18:37 bodepd actually, given that uses the ip address, it is not the usual error
18:38 bodepd check netstat -npalt on the controller and verify  that port 5672 is open
18:38 bodepd and verify it has open connections from the host where you see this failure
18:38 bodepd also check memory consumption
18:38 bodepd when memory is low, rabbitmq is hte first thing to fall over
18:38 bodepd blkperl: ^^
18:40 blkperl looks open
18:40 blkperl memory looks ok
18:41 blkperl so my problem is "instances error" on creation so maybe its not rabbitmq related
18:41 blkperl but those are the only errors in the nova logs
18:42 bodepd do a nova list/nova show
18:42 blkperl I also love how horizon is unable to get logs for the instance
18:42 bodepd what is the error?
18:42 bodepd yeah, it needs log aggregation
18:43 bodepd for instance failure: I always check: nova-scheduler logs, then nova-compute (which often leads to libvirt logs)
18:43 blkperl err I bet I shoudn't have put that special charcater in the keystone token
18:44 blkperl nova list/show just says Status: Error
18:44 blkperl oh theres a fault line
18:44 blkperl | fault                               | {u'message': u'NoValidHost', u'code': 500, u'details': u'No vali
18:45 blkperl d host was found.  |
18:45 blkperl bodepd: ^
18:46 bodepd blkperl: yep, that is the line
18:46 bodepd run: nova service-list
18:46 bodepd or nova service list
18:46 bodepd (some of the CLI stuff is inconsistent and hard to remember)
18:47 bodepd also, check the scheduler.
18:47 bodepd either, your compute is not registered, or is out of resources (memeory, cpu, etc)
18:47 bodepd you can also see the services from horizon
18:48 blkperl says they are all enbaled
18:48 bodepd why does the scheduler say it could not schedule?
18:48 bodepd and does it say there is a nova-compute service up?
18:49 blkperl no
18:49 blkperl only 4, cert,conductor,auth,scheduler
18:49 bodepd yep, not regiseterd compute host
18:49 bodepd check the nova-compute logs on teh compute host
18:49 blkperl there are none? :)
18:50 bodepd which node is your compute host?
18:50 blkperl oh does openstack::controller not come with computer?
18:50 blkperl compute*
18:50 bodepd no
18:50 blkperl :)
18:50 bodepd that is all-in-one
18:50 blkperl that makes more sense
18:50 bodepd so, all of the errors make sense now :)
18:50 mgagne bodepd: and disk space, there's a low water mark
18:51 mgagne bodepd: oops, scroll was jammed
18:51 bodepd esc-n :)
18:51 bodepd mgagne: ^^ I make that mistake all the time :)
18:52 blkperl bodepd: so if you have 4 nodes, does it make sense to have openstack::all on one, and openstack::compute on the other 3
18:52 bodepd I havn't testedit
18:53 bodepd but I know people wanted that model to work
18:53 bodepd (start with an all-in-one, then scale out compute nodes)
18:53 * blkperl continues with the test then :)
18:53 bodepd it *should* work
18:53 blkperl we should make a vagrant file to do this :)
18:54 bodepd https://github.com/CiscoSystems/openstack-installer/pull/49
18:54 bodepd we are in process for adding all-in-one to CI
19:02 bodepd shit. those last test failures suck.
19:02 bodepd I think we have to stop testing against master :(
19:02 bodepd of modules from puppetlabs
19:03 blkperl that may be wise
19:03 blkperl since they change so often
19:05 blkperl bodepd: http://pastebin.com/NzGBvFrQ - switch controller to all
19:05 blkperl err switching*, causes these errors
19:07 bodepd I seriously doublt you can convert to an all-in-one
19:07 blkperl :) ok ill reload
19:13 otherwiseguy joined #puppet-openstack
19:28 openstackgerrit Chris Hoge proposed a change to stackforge/puppet-quantum: documentation Updated README.md for module release  https://review.openstack.org/37753
19:53 mgagne bodepd: ping
19:54 bodepd mgagne: dude
19:54 bodepd can you tell from my email that I'm in a bad mood :)
19:55 mgagne bodepd: I just think about a conversation we had before that was related to this particular subject
19:55 bodepd mgagne: I've see a while string of issues with rabbitmq
19:55 mgagne bodepd: Modulefile, Puppetfile, .fixtures.yml =)
19:55 bodepd in the last week
19:55 bodepd the all have different purposes
19:55 mgagne bodepd: it's true
19:56 mgagne bodepd: they however don't have the same supported sources
19:56 bodepd (well arguably fixtures/Puppetfile have the same purpose)
19:56 mgagne bodepd: but that wasn't the question I wanted to ask :P
19:56 bodepd oh, it's just on the top of my mind
19:57 bodepd mgagne: let me ask you one thing;
19:57 bodepd should I pin to the same version as the modulefile or the last known working revision?
19:58 mgagne bodepd: I can say that it would have 2 different purposes/results
19:58 bodepd what about forking?
19:59 mgagne bodepd: forking should be a last resort I would say as it would hide an underlying problem
20:00 mgagne bodepd: my question: How can I mirror/clone apt.puppetlabs.com ?
20:00 bodepd ha!
20:00 mgagne bodepd: reprepro cannot reproduce the layout used by puppetlabs
20:00 bodepd ask in #puppet
20:00 bodepd ask haus
20:00 mgagne bodepd: alright, rsync is cloning all (hidden) snapshots :-/
20:01 bodepd and if you fix the repo so that I can pin to 3.x or 3.2.x let me know
20:01 bodepd and I will use yours :)
20:01 mgagne bodepd: you mean debian packages?
20:02 bodepd any of them
20:02 mgagne bodepd: https://gist.github.com/mgagne/3af85296ef8ff8bba577
20:02 mgagne bodepd: that's what I do
20:07 mgagne bodepd: about pinning or not puppet modules
20:07 mgagne bodepd: we have to test against trunk in some ways to avoid bad surprise if trunk gets packaged and released to forge
20:08 mgagne bodepd: otherwise we will be playing catchup
20:15 prad joined #puppet-openstack
20:39 bodepd I've just seen some really bad indicators this week
20:39 bodepd 1. complete rewrite of the module without any communication
20:40 bodepd there are even people from puppetlabs that are working on openstack, but little coordination between them and the module team
20:40 bodepd 2. I saw a patch get merged, then half of it's node removed without any notification
20:40 bodepd s/node/code/
20:41 mgagne bodepd: although they probably have good intention, it's poorly executed in regard to the impacts it has
20:41 bodepd no question, the intention is to make things better
20:41 bodepd if the trade-off is stability/reliability
20:42 bodepd then it's hard to follow master which makes it a littl harder to consume as an upstream
20:42 mgagne bodepd: wut... https://github.com/puppetlabs/puppetlabs-rabbitmq/commit/7d5398a2bddeaefb331b0fe17c8b68ab143724d3
20:42 bodepd the same could be said for the stackforge mdodules
20:43 bodepd mgagne: yep,that is why stackforgeis broken :)
20:43 bodepd the whole thign got refactored
20:43 bodepd kind of on a whim
20:44 bodepd I hope you understand that I don't explore the possibility of forking things on a whim
20:44 mgagne bodepd: limbo state until it's released to forge I would say
20:44 bodepd imagine if that kind of change were to happen every week to every module
20:44 bodepd for the forseeable future
20:45 mgagne bodepd: I guess I won't run puppet-librarian anytime soon
20:45 bodepd why?
20:45 bodepd oh ,b/c you can't get master anymore?
20:46 bodepd what if we just forked rabbit
20:46 bodepd ?
20:46 mgagne bodepd: you can if you are crazy
20:46 bodepd mgagne: fair enough. sometimes I just need someone to talk me down from a ledge :)
20:46 mgagne bodepd: oh, I guess I guarded myself against such changes :D https://gist.github.com/mgagne/6e83e5bc49332708b1bf
20:47 bodepd yep. pinning is smart
20:47 bodepd but also a pain to maintain
20:47 mgagne it sure is
20:48 mgagne bodepd: we should then make sure to impose an upper bound to dependency versions in Modulefile of our modules
20:49 mgagne bodepd: so people don't end up with a combination of versions we didn't test
20:49 bodepd I think we need to be strict about versions
20:49 bodepd on uppper/lower bounds for release (forge)
20:55 blkperl | fault                               | {u'message': u'ConsoleTypeInvalid', u'code': 400, u'details': u'Invalid console type spice-html5', u'created': u'2013-07-18T20:41:43Z'} |
20:55 blkperl bodepd: ^ new problem :)
20:59 bodepd why are you deploying spice and not novnc ?
21:00 bodepd there was a patch for spice support earlier, but I don't thikn it got merged
21:00 blkperl im not? im just using the default test/site.pp in puppet-openstack
21:15 mgagne bodepd: could we leverage tags in .fixtures.yml?
21:16 mgagne bodepd: or can we do something completely new: improve how fixtures work and try to find the latest "stable" versions within a certain range and test against that?
21:17 mgagne bodepd: this would "force" us to "force" upstream to release more often and tag releases appropriately.
21:17 mgagne bodepd: https://github.com/puppetlabs/puppetlabs-rabbitmq/releases I did the last release and it was 2.1.0. tag for it is not there.
21:24 openstackgerrit Dan Bode proposed a change to stackforge/puppet-openstack: Pin rabbitmq module  https://review.openstack.org/37782
21:26 bodepd Iam resonding to asp's email atm
21:26 bodepd I am asking these kinds of things
21:30 mgagne bodepd: tags will be added =)
21:36 openstackgerrit A change was merged to stackforge/puppet-openstack: Pin rabbitmq module  https://review.openstack.org/37782
21:37 bodepd mgagne: yep. honestly, I would much rather that everyone just work together and we continue to target trunk
21:37 bodepd and released module versions from the forge
21:38 blkperl bodepd: so I have 2 hypervisors connected, via ::all and ::computer but they can't launch instances :)
21:38 blkperl ::compute
21:38 blkperl cinder and glance seem to be working fine though
21:39 bodepd blkperl: can't b/c of the vnc issue?
21:42 koudaddy1981 joined #puppet-openstack
21:44 bodepd blkperl: are you getting the wrong version of teh packages?
21:44 bodepd maube you are installing havana
21:51 openstackgerrit A change was merged to stackforge/puppet-openstack: Adds support for RabbitMQ Clustering  https://review.openstack.org/36716
21:56 blkperl bodepd: nope grizzly, yes vnc issue
21:57 bodepd and it says spice?
21:57 bodepd https://review.openstack.org/#/c/26852/
21:57 bodepd that is confusing, it you have vnc enabled, it shoudl be set to novnc
21:58 bodepd that is for the all-in-one?
22:03 bodepd blkperl: check the vnc settings in nova.conf on the compute host
22:07 bodepd mgagne: so, ever since July 11th, the rabbitmq installs the latest version of rabbitmq
22:08 bodepd I need to update that commit to here: 43a000b95b13c62e4fbea14e31bc61929bda1b23
22:08 bodepd I am SOOOO lucky my unit tests for nova happen to catch that
22:08 bodepd just b/c it was not installing the apt module
22:09 blkperl bodepd: http://pastebin.com/u2aELB6H
22:10 blkperl yes all-in-one
22:10 blkperl i don't see mention of spice
22:10 bodepd you may need to do some google search ing and check that patch
22:11 bodepd spice is supported for vnc, but shoudl not be the default
22:11 bodepd maybe just disable vnc for now?
22:11 blkperl yeah i can try that
22:11 bodepd I haven't read too much about spice
22:12 openstackgerrit Daneyon Hansen proposed a change to stackforge/puppet-cinder: Remove Unnecessary mysql::server Dependency  https://review.openstack.org/37791
22:14 openstackgerrit Daneyon Hansen proposed a change to stackforge/puppet-quantum: Remove Unnecessary mysql::server Dependency  https://review.openstack.org/37792
22:14 openstackgerrit Daneyon Hansen proposed a change to stackforge/puppet-glance: Remove Unnecessary mysql::server Dependency  https://review.openstack.org/37793
22:21 openstackgerrit Dan Bode proposed a change to stackforge/puppet-openstack: update rabbitmq reference  https://review.openstack.org/37795
22:29 openstackgerrit Dan Bode proposed a change to stackforge/puppet-nova: Update rabbitmq dependency  https://review.openstack.org/37797
22:38 openstackgerrit Daneyon Hansen proposed a change to stackforge/puppet-keystone: Remove Unnecessary mysql::server Dependency  https://review.openstack.org/37800
23:53 openstackgerrit Dan Bode proposed a change to stackforge/puppet-cinder: Update revision for rabbitmq  https://review.openstack.org/37811

| Channels | #puppet-openstack index | Today | | Search | Google Search | Plain-Text | summary