Perl 6 - the future is here, just unevenly distributed

IRC log for #puppet-openstack, 2013-10-06

| Channels | #puppet-openstack index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
07:41 qba73 joined #puppet-openstack
09:52 * dachary reading openstack-installer/data/README.md
09:55 dachary I'm trying to run basic_test.sh with repos_to_use not set meaning I fall in https://github.com/bodepd/openstack-installer/blob/master/Puppetfile#L39 and I'm not sure if my chances of success are good. I assume https://github.com/bodepd/openstack-installer/blob/master/Puppetfile#L20 (i.e.if ENV['repos_to_use']  == 'downstream' ) are cisco specific and not good for external use but I may be wrong.
11:35 * dachary reports success on test_type=openstack_multi test_mode=basic_tests operatingsystem=ubuntu openstack_version=grizzly bash -x tests/basic_test.sh
11:35 dachary \o/
12:57 mmagr joined #puppet-openstack
13:37 dachary bodepd: I documented my modest success at http://dachary.org/?p=2352 and then wondered if it would be repeatable in a month from now. As far as I can see most modules do not depend on a specific commit, rather on a specific branch.
13:38 dachary is there a convenient way to say : "this combination of modules worked for me, I want to remember their commit hash" ?
13:44 * dachary trying
13:44 dachary test_type=openstack_multi test_mode=tempest operatingsystem=ubuntu openstack_version=grizzly bash -x tests/basic_test.sh
13:46 * dachary reading http://openstack-jenkins.cisco.com/job/coi_ci/250/consoleFull
13:47 dachary bodepd: I see that it's running basic_test ( + [[ basic_tests == tempest* ]] ) and I wonder what are my chances of success with tempest
15:33 ari_ joined #puppet-openstack
15:57 ari_ joined #puppet-openstack
16:27 ari_ joined #puppet-openstack
17:02 ari_ joined #puppet-openstack
17:05 qba73 joined #puppet-openstack
17:54 ari joined #puppet-openstack
19:42 dafter joined #puppet-openstack
20:19 dafter joined #puppet-openstack
21:04 ari joined #puppet-openstack
21:32 bodepd dachary: very unlikely atm :)
21:33 ari joined #puppet-openstack
21:33 dachary bodepd: :-) good to know, saves me time.
21:33 bodepd dachary: a lot of it is unfortunately things that worked once, but I haven't been able to maintain
21:34 bodepd I've been struggling a bit with the weight of the whole thing
21:34 dachary I'm not surprised.
21:35 bodepd I added instructions for installing using the module in a way that is decoupled from vagrant/basic_tests
21:35 bodepd dachary: there is not a way to save the commits.
21:36 bodepd but that is a pretty reasaonble feature request for librarian-puppet-simple
21:36 bodepd dachary: it's all targeted at branches b/c it's intended to run as a testing system
21:36 bodepd dachary: there is conditional logic where I wanted to add the capability to be able to install from modules
21:36 bodepd dachary: did all of the data stuff make sense?
21:37 dachary bodepd: yes, it makes perfect sense.
21:37 bodepd (it still needs to be optimized by movign all of the hash merge logic to the ENC)
21:37 bodepd Cisco gave me some really specific problems to solve
21:37 bodepd mainly about how to encode multiple reference architectures into a CD system
21:38 bodepd I solved their issues, but I've been a bit worried its a little complicated
21:38 bodepd the rest of my work will be helping them encode their supported references architectures
21:38 bodepd and helping write debugging tools and things to simplify and hopefully optimize the system
21:39 dachary applying the system will allow you to tune it
21:39 bodepd how do you mean?
21:39 dachary encoding the reference architecture will demonstrate what's difficult and what's easy, what works and what does not
21:40 bodepd it also demonstrates a lot of inconsistencies in the modules
21:40 bodepd and places where the defaults are not sufficient
21:41 bodepd next week, we will look at adding ceph
21:41 dachary I suspect the difficult part will be ( for me and others ) to understand how a given value relates to a given action. That mapping without an understanding of the implementation requires a good documentation. Or reading the sources.
21:41 bodepd so that when you include ceph as either a cinder and glance backend
21:41 bodepd it will become a role
21:41 bodepd how was the README in the data directory?
21:42 dachary the readme was an excellent introduction
21:42 bodepd I was told that it is useful only if you have a lot of existing contect
21:42 dachary I wonder how much work it would be to write the reference manual
21:42 bodepd s/contect/context/
21:43 bodepd it's not just the ping test
21:43 dachary ceph has changed *a lot* since the puppet module was written. I suspect things should be done very differently now.
21:43 bodepd it also runs ssh -i /tmp/id_dsa cirros@<floating_ip> host
21:43 bodepd hostname
21:43 bodepd cisco is rewriting it to use the ceph-deploy tool
21:43 bodepd that is what will be integrated in
21:44 bodepd they weren't happy with the enovance module
21:44 bodepd b/c it required too much coordination of actions
21:44 dachary interesting. Is there a URL for that effort somewhere ?
21:44 bodepd let me see if I can find it
21:44 bodepd https://github.com/CiscoSystems/puppet-ceph/tree/grizzly
21:44 bodepd I'm guessing that is it
21:45 dachary ok. I think ceph-deploy is probably not the right tool to rely on because it vastly overlaps with puppet.
21:46 dachary it's the right source to figure out how things should be done though
21:46 bodepd I think from cisco's perspective, it all comes down to effort
21:46 bodepd they want to focus on network plugins and let someone else worry about installation
21:47 bodepd did you see that I wrote those bootstrap scripts?
21:47 bodepd for the build server and for getting puppet 3.3 installed?
21:47 bodepd I know Robert already used them to deploy a cluster from scratch
21:47 dachary I noticed but did not look yet.
21:48 * dachary checking
21:48 bodepd I think Robert has some outstanding patches that clarify things like how to pick the deployment scenario
21:48 ari left #puppet-openstack
21:48 bodepd and understanding how roles map to certnames
21:48 dachary install-scripts/setup.sh ?
21:48 bodepd yeah
21:49 bodepd that bootstraps machiens with Puppet 3.3
21:50 bodepd https://gist.github.com/bodepd/6859538
21:50 dachary that's a sexy onliner ;-)
21:50 bodepd I like the curl-bash
21:53 dachary https://github.com/CiscoSystems/openstack-installer/raw/master/install-scripts/setup.sh
21:53 dachary is the URL though
21:53 dachary hum
21:54 dachary no
21:54 dachary https://raw.github.com/CiscoSystems/openstack-installer/master/install-scripts/setup.s
21:54 dachary is
21:54 dachary (missing master)
21:55 bodepd I just merged a commit that I believe fixes that
21:55 dachary bash <(curl -fsS https://raw.github.com/CiscoSystems/openstack-installer/master/install-scripts/setup.sh)
21:56 dachary I rarely get to use the <() but find it cute ;-)
21:56 bodepd can you check that latest update?
21:57 bodepd if you want to implement a freeze call on librarian-puppet-simple that captures all of the commits, that seems like a pretty reasonable feature
21:58 bodepd I merged those fixes, so you can update the blog post
21:59 dachary bodepd: the /master/ must be after openstack-installer
21:59 dachary https://raw.github.com/CiscoSystems/master/openstack-installer/install-scripts/setup.sh
21:59 dachary is in README.md
21:59 bodepd oh, was robert's patch no good?
21:59 bodepd https://github.com/CiscoSystems/openstack-installer/commit/c7ea63e00d193468204f19e51d0b8c85461aa0e3
22:00 dachary indeed
22:00 bodepd was that not correct?
22:00 dachary try curl'ing the URL
22:00 bodepd I could do that :)
22:00 dachary I'll fix it
22:01 bodepd I verified that it was indeed incorrect
22:05 bodepd dachary: you already fixed my README typos :)
22:05 bodepd dachary: I was just about to have a look at those. I wrote that doc in a hurry :)
22:05 bodepd dachary: thanks!
22:05 dachary I'm glad to be marginaly useful ;-)
22:14 dachary bodepd: I assume a tenant is defined by default and that the user / password ( for basic_test.sh ) is to be found in the data directory somewhere. This is the kind of thing a reference manual would cover ?
22:16 dachary ( I figured it out btw ;-)
22:17 bodepd the way the data is supposed to work is:
22:18 bodepd - most data represent defaults that are determines by the global_params
22:18 bodepd - the user data is user.common (which will probably eventually be deprecated)
22:18 bodepd - user data is user.%{scenario}
22:18 bodepd for the most general case, you should only have to look in these files to know what to override
22:19 bodepd for developers, it is much more cimplicated
22:19 bodepd b/c they have to understand potentially how to find all possible data values
22:19 dachary and the associated documentation that explains what the parameters does and how it relates to the rest ?
22:19 dachary yes
22:19 dachary the user has to understand the semantic of the value which is less complicated
22:19 bodepd that explanation needs to be the next revision
22:20 bodepd my idea was:
22:20 bodepd for a user:
22:20 bodepd select a scenario
22:20 bodepd potentially update global params
22:20 bodepd override user data
22:20 dachary how would the user figure what global params mean ?
22:20 bodepd it needs to be documented
22:20 dachary ok
22:20 bodepd it is documented in the README
22:20 bodepd (which I just wrote yesterday)
22:20 dachary oh, just those you mean ?
22:21 dachary hum
22:21 bodepd but, I need different docs for user vs. developer
22:21 bodepd s/yesterday/friday/
22:21 dachary by developer you mean someone writing  a scenario, right ?
22:21 bodepd yes
22:21 bodepd the README is focused on that
22:21 bodepd I need more user facing docs elsewhere
22:21 dachary ok
22:21 bodepd one things that may not be clear
22:21 bodepd this was intended to be a back-end
22:22 bodepd that would interface with a front-end
22:22 bodepd take foreman for example
22:22 bodepd as a user:
22:22 bodepd 1. pick the scenario
22:22 bodepd 2. select global data
22:22 dachary I would be a developer then because the global params won't cover what I suspect I'll need to do
22:22 bodepd 3. run a script (that does not exist yet) that takes that data and builds user intefaces in foreman
22:22 bodepd dachary: it does not cover your use case?
22:23 dachary it covers my first use case, yes
22:23 dachary the one I'm focused on right now
22:23 bodepd dachary: what is your second use case?
22:24 dachary from the top of my head, tweak the ceph configuration
22:24 bodepd timing is good then :)
22:25 bodepd b/c that is what I am working on next week for Cisco (getting the ceph model's in their reference architecture)
22:25 bodepd there is one feature of the model that still needs to be implemented
22:25 dachary :)
22:25 bodepd the roles for the scenarios should be driven by selection of global params
22:25 bodepd ie: some global params include new class_groups/roles/classes
22:25 bodepd others can exclude them
22:26 bodepd I'm defintely happy to have your opinion on this. Making complex things usable is always tricky
22:27 bodepd did you figure out to use the hiera common line tool?
22:27 bodepd that is the best way to understand how data got/gets its value
22:27 dachary bodepd: not yet.
22:27 bodepd if you run it with --debug
22:27 bodepd the thing that is painful about it, is that you have to set all globals from the command line
22:28 dachary I'm happy that I got basic_test.sh to work today. But I broke it later in the afternoon and can't figure out why.
22:28 bodepd ie: hiera --debug --config /etc/puppet/hiera.yaml operatingsystem=redhat scenario=2_role
22:28 * dachary trying
22:28 bodepd oh
22:28 bodepd hiera --debug --config /etc/puppet/hiera.yaml operatingsystem=redhat scenario=2_role nova::verbose
22:28 bodepd (you need to specify a fully qualified variable to lookup)
22:29 bodepd you have to pass the params it needs from the CLI
22:29 bodepd which makes it hard to use for debugging though
22:29 bodepd that is the tool I will build next week
22:29 bodepd is something that wraps hiera, but consumes the data files to setup the right global variables
22:29 dachary vagrant@build-server:~$ hiera --debug --config /etc/puppet/hiera.yaml operatingsystem=redhat scenario=2_role nova::verbose
22:29 dachary WARN: Sun Oct 06 22:29:05 +0000 2013: Cannot load backend data_mapper: no such file to load -- hiera/backend/data_mapper_backend
22:29 dachary nova::verbose
22:29 bodepd ah, yeah
22:29 bodepd export RUBYLIB=/etc/puppet/modules/hiera_data_mapper/lib
22:30 bodepd (b/c the plugin is delivered as a module, and hiera is not using Puppet's plugin libs)
22:30 dachary works :-)
22:30 dachary vagrant@build-server:~$ hiera --config /etc/puppet/hiera.yaml operatingsystem=redhat scenario=2_role nova::verbose
22:30 dachary nova::verbose
22:31 bodepd you have to set al globals that hiera uses manually though
22:31 bodepd globals hiera uses: facts, and variables set by the env
22:31 bodepd enc
22:31 bodepd which is what I'm fixing tomorrow is that you should be able to run something that says:
22:32 bodepd given how my master is currently configured, tell me how this is set
22:32 bodepd what is the basic_tests.sh failure?
22:32 bodepd I have 5 minutes before I go for a bike ride :)
22:33 dachary | fault                               | {u'message': u'NoValidHost', u'code': 500, u'details': u'No valid host was found.                               |
22:33 dachary |                                     |   File "/usr/lib/python2.7/dist-packages/nova/scheduler/filter_scheduler.py", line 97, in schedule_run_instance |
22:33 dachary |                                     |     raise exception.NoValidHost(reason="")                                                                      |
22:33 dachary |                                     | ', u'created': u'2013-10-06T17:12:23Z'}
22:33 dachary that's probably a side effect of something
22:33 dachary not obvious
22:33 dachary I'll figure it out
22:34 bodepd that means openstack failed
22:34 dachary yes
22:34 dachary the scheduler fails to find the compute node
22:34 dachary not sure why now and not the run before
22:34 bodepd check the scheduler_logs -> compute_logs -> libvirt_logs
22:34 bodepd I just ran a job in jenkins
22:34 dachary enjoy the bike ride :-)
22:34 bodepd http://openstack-jenkins.cisco.com/job/coi_ci/251/
22:34 bodepd we'll see if that passes
22:35 bodepd I really want to aggreaget the logs
22:35 bodepd I wish that when a failure happened, it shoudl you all logs for all services
22:35 bodepd (smokestack does this, it is pretty awesome)
22:35 dachary I'm quite sure it will. I tried a few things to forward the ports and connect to horizon, probably messed things up in the process ;-)
22:35 dachary bodepd: +1
22:36 bodepd I've just been using ssh port forwarding to see horizon
22:37 dachary bodepd: I suggest you connect to irc.oftc.net#ceph-devel alfredodeza is there and develops ceph-deploy and he is in your tz ( more than I am ;-)
22:37 bodepd I probably should get more into ceph
22:38 bodepd dachary: let me know if you run into anything else. I'm off for the day
22:38 dachary bodepd: I'd be more than happy to answer all the questions you have about ceph
22:38 dachary cheers
22:45 ari joined #puppet-openstack
23:12 dmsimard joined #puppet-openstack
23:21 badiane_ka joined #puppet-openstack

| Channels | #puppet-openstack index | Today | | Search | Google Search | Plain-Text | summary