Perl 6 - the future is here, just unevenly distributed

IRC log for #puppet-openstack, 2013-10-28

| Channels | #puppet-openstack index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:01 morazi joined #puppet-openstack
00:09 rongze joined #puppet-openstack
00:42 michchap joined #puppet-openstack
00:59 ryanycoleman joined #puppet-openstack
01:10 rongze joined #puppet-openstack
01:22 rongze joined #puppet-openstack
01:55 michchap joined #puppet-openstack
02:07 rongze joined #puppet-openstack
02:09 rongze joined #puppet-openstack
02:09 rongze joined #puppet-openstack
03:02 bodepd_ dachary: I thought you were using vagrant/vbox for the integration tests?
03:40 lnxnut joined #puppet-openstack
04:06 lnxnut joined #puppet-openstack
05:20 openstackgerrit Russell Sim proposed a change to stackforge/puppet-tempest: Fix pip install of tox on debian like OS's  https://review.openstack.org/54057
05:35 rongze joined #puppet-openstack
05:53 openstackgerrit Russell Sim proposed a change to stackforge/puppet-tempest: Added missing packages for Debian like OS's  https://review.openstack.org/54059
06:02 openstackgerrit Russell Sim proposed a change to stackforge/puppet-tempest: Added configuration option for admin role  https://review.openstack.org/54060
06:02 mattymo joined #puppet-openstack
06:11 openstackgerrit Russell Sim proposed a change to stackforge/puppet-tempest: Added missing packages for Debian like OS's  https://review.openstack.org/54059
06:32 rongze joined #puppet-openstack
07:04 ryanycoleman joined #puppet-openstack
07:06 bodepd_ dachary: could you make any sense of that README?
07:19 dachary bodepd_: I am using vagrant/vbox for the integration tests
07:20 bodepd_ dachary: yeah, that is what I thought. but you want to switch it to an openstack API endpoint?
07:20 dachary but I'm trying to use vagrant/openstack instead because I don't want to install CI on a bare metal
07:20 dachary right :-)
07:20 dachary VM are easier to allocate than bare metal
07:20 bodepd_ I just have a few bare metal nodes that are jenkins slaves with a single executor
07:21 bodepd_ I'm lucky to be somewhere with tons of free hardware
07:21 dachary lucky you :-)
07:21 bodepd_ lucky me for one more week anyways ;)
07:21 bodepd_ my laptop is also smoking fast
07:22 dachary I'm a bit lost regarding how to convince rspec-system to tell vagrant to use the openstack plugin. It's probably not much but it's buried in code I'm not familiar with
07:22 bodepd_ one of these days, I want to poke it with a stick.
07:22 dachary bodepd_: what kind of laptop do you have ?
07:22 dachary ;-)
07:22 bodepd_ a mac. 16G RAM and 500G SSD
07:22 dachary :-)
07:23 dachary bbl, got to move to a less impressive laptop but still better than what I currently have
07:23 bodepd_ I know the author rspec-system, so most of what I know about it, I know from talking to him
07:24 bodepd_ and from what he said, it shoudl be trivial to swap backend
07:24 bodepd_ s
07:24 bodepd_ but it may well be buried in the code somewhere
07:25 bodepd_ http://rubydoc.info/gems/r​spec-system#Node_providers
07:27 dachary it's not about switching rspec-system provider though. The idea is to still use vagrant.
07:28 dachary but pass options to the vagrant backend so that it uses openstack plugin instead of its default vbox
07:29 dachary bodepd_: bbl
07:52 dachary back
07:55 xingchao joined #puppet-openstack
07:55 dachary bodepd_: should I ask dmsimard to approve https://review.openstack.org/#/c/53566/ ?
07:56 dachary Or mgagne ?
08:08 mattymo bodepd_, hi!
08:22 marun joined #puppet-openstack
08:28 bodepd_ dachary: yeah, consider me a casual observer on the ceph stuff
08:28 bodepd_ dachary: I'm happy to help, answer questions, look at Puppet releated stuff
08:29 dachary ok :-) mgagne wanted oversight from you or him
08:29 bodepd_ dachary: but in general, I don't want to merge things
08:29 dachary ok, cool
08:29 bodepd_ I'll ask him what he had in mind tomorrow
08:29 bodepd_ or today
08:29 dachary I think just that we get used to how things are done by following your lead for a little while
08:30 bodepd_ I'm happy for one of us to +2 things
08:30 bodepd_ we've learned tons so maybe we can help you make all of the mistakes that the openstack modules have made
08:30 bodepd_ mattymo: hi
08:30 dachary :-)
08:30 mattymo bodepd_, will you be at the OpenStack Summit?
08:30 bodepd_ I'll be there
08:30 mattymo ok great
08:30 mattymo me too
08:31 mattymo I'm with Mirantis and we work on Fuel. I tried to find you at PuppetConf in August, but I heard you were suspiciously absent
08:31 bodepd_ mattymo: funny. yeah, I was best man in a wedding exactly those two days :)
08:32 mattymo yeah it's a very popular time for weddings
08:32 bodepd_ mattymo: would have liked to have been there
08:32 mattymo anyway, we've got Fuel (except manifests) up on stackforge, but we really need to find a good way to mesh with what's already in stackforge
08:33 bodepd_ I'm happy to chat about it. I know the content converged quite a while ago
08:33 mattymo we realize it's going to be a long process, but maybe there's some points where we could start first where there is lots of room for improvement
08:34 bodepd_ yeah, I saw a patch from you guys the other day
08:34 mattymo oh which?
08:34 bodepd_ on one of the openstack modules
08:34 bodepd_ I don't remember what it was
08:34 bodepd_ is the intention to align with upstream?
08:34 mattymo eventually
08:35 bodepd_ I helped Cisco with that. It actually wasn't that much effort.
08:35 bodepd_ (at least for core things:  nova, glance, etc)
08:35 mattymo I heard about your work with cisco on grooming fuel manifests
08:36 bodepd_ let me know if there's anything I can do to help. It would be great to get your guys on board.
08:36 bodepd_ I'm sure you guys have a realy robust set of content
08:37 mattymo we just finished a release last week, have a lot to blog about, and are geared up for the summit
08:37 mattymo I'm hopefully going to get a couple of things ready this week on our site
08:38 mattymo our biggest ambition next is to remove our master node from being a bottleneck by removing puppetmaster and cobbler
08:38 bodepd_ moving to puppet apply?
08:38 mattymo and do self-deploy with our own ENC (via astute) and deployment via a torrent-based system
08:38 mattymo yes
08:39 mattymo we hope to use tinycore linux with a script to download preinstalled image and write it to a designated block device
08:39 mattymo I was really hoping to move to another direction that Foreman is going, and I was wondering how many foreman users there are in this group
08:39 bodepd_ ready about astute now.
08:40 bodepd_ I've been looking at it, and chatting with those guys a bit.
08:40 bodepd_ I know of a few folks that are using it
08:40 mattymo astute is a wrapper around mcollective to gather information about nodes, generate yaml for node facts, and send commands and monitor their output
08:41 mattymo lots and lots of RH people are into foreman, of course
08:41 mattymo I used to work at RH and everyone I knew can say "omg it's awesome" but they haven't done anything more hardcore than what cobbler did
08:42 mattymo the best group for applying Foreman in practice I've met is the CERN team
08:42 mattymo I was wondering if they hang aroudn here
08:42 mattymo around*
08:42 bodepd_ mattymo: I never really here a peep from them
08:42 mattymo shoot. one guy from CERN did a great presentation on Foreman + puppet for deploying OpenStack
08:43 bodepd_ mattymo: AFAIK, they are happy downstream consumers, I see Tim replies sometimes in the mailing list, but in general they are quiet
08:43 bodepd_ mattymo: I saw the slides
08:43 mattymo bodepd_, are you on California time?
08:43 bodepd_ yeah.
08:43 mattymo pretty late there. I'm in Moscow
08:44 bodepd_ yeah. I work strange hours :)
08:45 mattymo so I believe our biggest improvements in the last 4 months were related to HA for quantum/neutron + ovs
08:45 mattymo is any of that stuff HA in puppet-openstack?
08:46 bodepd_ this probably says everything I need to know about astute :) https://github.com/Mirantis/astute/blob/ma​ster/lib/astute/cli/provision_schema.yaml
08:46 hlnb joined #puppet-openstack
08:47 bodepd_ mattymo: I don't think anyone has been cherry-picking commits from the mirantis fuel manifests
08:48 mattymo bodepd_, this is 1/2 of the puzzle
08:48 bodepd_ mattymo: and I'm pretty sure everyone is moving away from puppet-openstack specificall
08:48 bodepd_ specifically
08:48 bodepd_ I haven't been too focused on HA
08:48 mattymo the other half is https://github.com/stackforge/fuel-web/blob/m​aster/nailgun/nailgun/fixtures/openstack.json
08:49 mattymo the first half is cobbler config, openstack.json turns into yaml for puppet apply
08:52 bodepd_ I've been working on something conceptionally similarl
08:52 bodepd_ https://github.com/bodepd/scenario_node_terminus
08:52 bodepd_ https://github.com/CiscoSystems/op​enstack-installer/tree/master/data
08:52 bodepd_ it's still a bit rough around the edges
08:55 hlnb Hello, I have a small question about the readme, what is it exactly in the project, which is its main purpose?
08:56 beddari mattymo: you there? I'm on a new project that aims to use Foreman, and have been using it for a few years, successfully
08:56 rongze joined #puppet-openstack
08:57 beddari mattymo: I'm from Norway, working on a national (very early) Openstack IaaS project. Two universities cooperating to fund it so far
08:57 bodepd_ hlnb: what readme?
08:57 xingchao joined #puppet-openstack
08:59 beddari mattymo: I saw you commenting on tiny core earlier, I think its kind of hard to use that instead of something like RHEL/Ubuntu, kernel wise. Think firmware, drivers etc. Better to do something that will work on both phys and cloud instances
08:59 mattymo beddari, it wasn't my idea at all :)
09:00 mattymo tinycore just happened to be an option my colleagues chose
09:00 bodepd_ Razor uses tiny core
09:00 mattymo ovirt actually has some cool tools to take a centos/rhel image and strip out everything
09:00 bodepd_ (or at least it used to :)
09:00 beddari yeah Razor (and Foreman) used that to begin with, Foreman is moving away from it
09:01 bodepd_ Razor is being rewritten,so I don't know what it uses anymore
09:01 bodepd_ beddari: what is foreman moving to?
09:01 bodepd_ (stripped rhel image?)
09:01 mattymo bodepd_, mini fedora or rhel image
09:01 beddari mattymo: interesting that you are moving away from masters, I'm really in on that one, I have to start with a master just for people to get educated though (kind of a bootsrap problem)
09:01 beddari yes
09:01 beddari fedora first
09:01 mattymo beddari, we want to deploy a 1000 openstack cloud in 1 day
09:02 mattymo I did a presentation that covered this topic and we found that cobbler on a single mechanical disk would take ~4 hrs to install 1000 hosts
09:02 bodepd_ beddari: I'm not a big master fan. but I do like centrally storing reports,facts, and a central data store
09:02 beddari mattymo: for me it is not so much about scale (yet) as it is about .. pipeline
09:02 bodepd_ beddari: since you guys use mcollective, it's probably fine
09:02 mattymo and about 4 days to run puppet one at a time
09:03 mattymo err... one at a time for controllers, then cinder/compute nodes 10 at a time
09:03 mattymo any more and it dies
09:03 bodepd_ Puppet dies?
09:03 mattymo master
09:03 bodepd_ really?
09:04 mattymo puppetdb crashes or our mcollective fails or rabbitmq fails
09:04 bodepd_ I would expect it to be able to handle a lot more than that
09:04 mattymo something will die
09:04 mattymo clients time out
09:04 bodepd_ at what scale?
09:04 beddari mattymo: yeah it could possibly be solved but its just too hard fuzz with, I agree
09:04 bodepd_ more than one controller at a time
09:04 mattymo we expect only 2 cores with 4gb memory for our master
09:04 beddari ;-)
09:04 mattymo a very very small master
09:04 mattymo so we can do 20 with no issues. 50+ usually shows problems
09:04 bodepd_ I would like to learn more about those failures :)
09:05 bodepd_ mattymo: have you looked into pre-compiling catalogs?
09:05 bodepd_ and serving static catalogs?
09:05 mattymo I'd like to get to know people from puppetlabs who can actually push bugs through
09:05 beddari with 4gb RAM it is just not going to work with modules from the Forge, e.g
09:05 mattymo I know of lots of bugs in puppet that just won't be fixed because we're not a paying customer or the person who reports the bug isn't
09:06 mattymo bodepd_, I would like to experiment with that for sure
09:06 beddari mattymo: its the way it works, I've just started to learn shepherding
09:06 mattymo also benchmarking/profiling pre-compile would be great too
09:06 bodepd_ yeah
09:06 mattymo to see what the #s look like without network delays
09:06 marun joined #puppet-openstack
09:06 bodepd_ are the catlalogs always the same?
09:06 hlnb bodepd_ : As part of our studies, Mr Dachary allocated a task to be done to contribute to the project. At the moment we are trying to understand what was done and the general context. We would like to know the main purpose of readme.rm and function site.pp throughout the project.
09:06 bodepd_ for a given role?
09:07 mattymo our manifests are the same, but our number of role combinations is about 20
09:07 hlnb I'm sorry I'm not very good in English :/
09:07 mattymo but in most deployments, there are usually 3 or 4 roles in play
09:07 bodepd_ hlnb: can you be specific about a project?
09:07 bodepd_ hlnb: I'm not sure if you are talking about openstack or ceph, or something else
09:09 beddari mattymo: you on twitter? so as I not lose you heh
09:09 bodepd_ you can always find freelance consultants who know Puppet's source and details *cough*
09:10 mattymo I can fix the source, but really I don't want to rely on custom puppet builds forever and ever
09:10 mattymo it's better to get upstream fixes
09:10 bodepd_ I agree. you have patches that are sitting?
09:10 hlnb bodepd_ : oh sorry, I'm talking about puppet-ceph project
09:10 bodepd_ I'd love to have a look
09:10 bodepd_ hlnb: ah. probably best to ask dachary.
09:11 bodepd_ hlnb: the intention of the project is to generate configuration code for ceph. to consolidate lots of separeate community efforts
09:11 mattymo bodepd_, I'd love to see http://projects.puppetlabs.com/issues/4577 or http://projects.puppetlabs.com/issues/5456 applied
09:11 mattymo beddari, https://github.com/mattymo
09:11 beddari mattymo: hmm, so you use puppet for updating?
09:11 mattymo beddari, I don't like twitter
09:12 beddari mattymo: I use it as news .. so ..
09:12 mattymo beddari, I'm always on IRC so you can find me here
09:12 beddari and I tend to forget people with no picture
09:12 beddari hehe ok but US timezone I presume
09:12 mattymo I'm in Moscow, actually
09:12 beddari yaeh right
09:12 beddari cool, know any russian yet
09:13 mattymo yeah I've studied for 4 years
09:13 * beddari does know russian infact hehe
09:13 beddari I grew up in Norway in the border to Soviet so I kind of got a special interest in it
09:13 mattymo then you probably know about "сауна"?
09:14 beddari err been 10 years so don't test me too hard, not sure, sayna?
09:14 mattymo sauna
09:14 beddari ah
09:14 beddari sure
09:15 beddari my folks has one ;)
09:15 mattymo bodepd_, if you fix this rpm obsoletes bug, I'll be in your debt
09:16 bodepd_ as a workaround, you can override core providers from a module
09:16 beddari .. and you know the most voted bug ever, the "multiple packages per provider run" hehe
09:16 bodepd_ it looks like a patch already exists
09:16 mattymo beddari, that would speed up things a lot
09:16 bodepd_ beddari: yeah, that used to make me laugh about the voting
09:16 bodepd_ beddari: just that we instituted it, and didn't ever address the #1 issue
09:16 bodepd_ s/we/they/
09:16 hlnb bodepd_ Can I ask you which part of the project you are working now? Are you in the process of achieving a feature or fix a bug?
09:17 beddari bodepd_: I get it :)
09:17 mattymo what would be really cool is if we could use puppet pre-compile to find out what package resources would get generated, then add it to kickstart/debian presseed so they get installed during provisioning
09:17 beddari mattymo: you don't need to, put your data in hiera and query it
09:17 bodepd_ hlnb: I've just been helping dachary get started. I'm just reviewing patches, and making recommendations
09:17 mattymo beddari, somehow I've avoided hiera
09:17 beddari ?
09:17 beddari why
09:17 bodepd_ mattymo: I can build that! That sound fun
09:18 mattymo bodepd_, cool
09:18 bodepd_ mattymo: I always wanted to build something that grabbed packages out of the catalog to generate an image and tooka snapshot
09:18 mattymo this data could then be used to check against CVE threats or whatever
09:18 beddari tbh I'd say doing that specifically (if it is just packages) would not be something I'd do
09:19 bodepd_ so that as a part of your build-pipeline, you just reinitialize packages once a day
09:19 mattymo I always imagined mcollective does this too, but I want to get this info before deployment, not after
09:19 beddari what you need is moving this data out of puppet (if it is there today?)
09:20 bodepd_ I'm out.
09:20 mattymo it's really more like finding a quick route to see what a package set looks like when we deploy on OS A, B, or C
09:20 mattymo and comparing
09:20 beddari hiera, I tell you
09:21 mattymo bye bodepd_
09:21 mattymo beddari, if you can send me a link to something useful, I'd appreciate it
09:21 beddari exactly why you need that data externalized, it can be reused for whatever
09:21 mattymo beddari, let me give you one example
09:22 mattymo Fuel deploys a modified centos 6.4 with roughly 200 extra or rebuilt packages
09:22 mattymo and we have a huge repo
09:22 mattymo and it's hard to keep track of what's needed for our puppet master, for controllers, or for compute hosts
09:22 mattymo more for analysis and maintenance
09:23 beddari so where does this data (the packages, explicitly) live now, in the roles and profiles manifests?
09:24 beddari mattymo: and for links, how far along are you with understanding hiera, what puppet version are you at?
09:24 mattymo in a yum repo either on our site or on an ISO
09:24 mattymo beddari, we're sadly sadly still stuck at 2.7.x
09:24 beddari sure I get that, but where in the code do you specify what packages are pulled
09:24 mattymo we have a couple of modules that have scope issues in 3.x
09:24 beddari uho h
09:24 beddari ;)
09:25 mattymo https://github.com/Mirantis/fuel/blob/master/d​eployment/puppet/cobbler/manifests/packages.pp
09:25 mattymo like this?
09:25 beddari let me have a look :)
09:26 beddari so this gets old really quick when you got variants etc
09:26 beddari how far along are you at understanding hiera and data separation? sorry must ask :P
09:26 mattymo not at all
09:27 beddari ok so the Hiera documentation as it stands today is really geared towards having a master
09:27 mattymo we still have one for now
09:27 beddari but don't let that get in the way of anything :)
09:28 beddari when moving to a CI/build/CD masterless env you need some way of precompiling what Hiera-data goes to what node
09:28 beddari have you been coming up with anything as to how you'd do this with only Puppet DSL code?
09:28 beddari there is no standard solution for this at the moment
09:29 beddari not very hard to do, several has built their own .. Spotify comes to mind, a lot of multi-tenant sites as masterless is the only way when you need security separation
09:31 beddari mattymo: this is not too bad of an intro -> http://puppetlabs.com/blog/hi​era-for-pouncers-and-stalkers
09:35 beddari mattymo: what specifically was your interest point in relation to Foreman?
09:36 beddari mattymo: I agree with you wrt to CERN, nice that they are public about it
09:41 beddari mattymo: well since you went silent I'm off too, very interested in what you said about your motivation and direction for Fuel though, won't let you off now hehe. our thoughts and plans are very similar
10:06 rongze joined #puppet-openstack
10:18 mattymo beddari, sorry I had a meeting
10:19 mattymo beddari, the one thing of Foreman that looks cool is smart-proxy
10:20 mattymo you could in theory set up a distributed provisioning system with smart-proxy. Allocate maybe 10-15% of un-deployed hosts to this smart-proxy role to help the majority of hosts get provisioned much faster by acting as proxies
10:20 mattymo and that way we can limit cpu/disk/memory of the master node from becoming a bottleneck
10:21 mattymo but it's very new and not perfect yet
10:23 beddari mattymo: I see, you want to proxy provisioning .. I saw there were some recent activity on that. Kind of neat to use un-deployed hosts but then they aren't undeployed any more really :P
10:24 beddari mattymo: I think I like your p2p idea (BT) better, maybe combine that with lightweight messaging like serfdom.io
10:24 mattymo we use rabbitmq which is not light :(
10:24 beddari mattymo: or was that something you'd envision putting on the proxies
10:25 beddari mattymo: we use mcollective for ops-related messaging, was hoping it'd move up the layers too but no such luck so far:P
10:27 beddari mattymo: 1000 hosts in a go is kind of a unique problem, I'd agree keeping it as decentralized as possible will be key
10:32 beddari mattymo: hmmmm ... -> http://www.serfdom.io/docs​/internals/simulator.html  .. plus some custom events ..
10:49 mattymo beddari, it's central to our development process though
10:49 beddari mattymo: what is?
10:49 beddari rabbit :)
10:49 mattymo come in with 5-1000 servers ready to be installed, set up 1 really weak one with an ISO on a single disk with 50gb space
10:49 beddari ah that
10:50 mattymo and control with rabbitmq + mcollective + cobbler to provision them
10:50 mattymo you shouldn't need to set up HA or load balance it
10:50 beddari to do the provisioning, no
10:50 mattymo it should be able to come up with something efficient to deploy as fast as possible...
10:51 mattymo we do have a bootstrap lightweight live img we distribute first to gather info about disks, networks, and cpu
10:51 mattymo and it communicates back its info and waits for commands
10:51 mattymo it's centos based and is about 130mb in size
10:53 beddari how does that go out? PXE?
10:53 mattymo pxe over tftp
10:53 beddari so how long does it take to do that for say <1000 ?
10:54 mattymo and of course with all that broadcast and udp traffic flying around at the same time, it makes 20% error and reboots and tries again
10:54 beddari mm
10:54 mattymo I did the math, and it should be about 40 minutes
10:54 beddari ouch
10:54 mattymo that's if you power them on all exactly at the same time
10:54 mattymo but I'm sure no power system can actually handle this kind of instant load
10:54 mattymo so it can drop to about 30 if you stagger 100 at a time
10:55 beddari no but the theory is telling of the concept
10:55 mattymo and wait 1 minute in between
10:56 beddari lunchtime. when I saw serfdom.io first thing I thought about was node discovery and 'orchestrated convergence' eh somesuch .. the notion that you can spread a deploy like a messaging fanout
10:56 beddari read that page I linked
10:57 mattymo cool
10:57 beddari mattymo: what are the network requirements you typically get on a deployment like that though? as for how you could possibly spread a pxe boot out
10:58 mattymo it's in another damn obscure language
10:59 beddari :-D
10:59 beddari im off the keyboard for an hour or so
10:59 mattymo ok
11:00 mattymo right now it's 1 network segment, usually gigabit
11:00 mattymo we have no dhcp agents or anything to provide deployments that could really reach 1000 nodes
11:06 dachary dalgaaf: good morning sir :-)
11:07 mattymo but it's important to note that OpenStack beyond 1000 servers can't really work because of messaging and DB load
11:07 mattymo you need to make modifications or get some really serious hardware
11:40 xingchao joined #puppet-openstack
11:48 xingchao_ joined #puppet-openstack
12:17 dprince joined #puppet-openstack
12:25 rongze joined #puppet-openstack
12:57 lnxnut joined #puppet-openstack
13:04 lnxnut joined #puppet-openstack
13:06 lnxnut joined #puppet-openstack
13:09 ryanycoleman joined #puppet-openstack
13:10 lnxnut joined #puppet-openstack
13:11 feandil joined #puppet-openstack
13:13 morazi joined #puppet-openstack
13:19 _ilbot joined #puppet-openstack
13:19 Topic for #puppet-openstack is now Place to collaborate on Puppet/OpenStack tools: logs at http://irclog.perlgeek.de/puppet-openstack/today
13:26 xingchao joined #puppet-openstack
13:31 beddari mattymo: yes I know .. still this isolated problem deserves a good solution I think
13:31 xingchao joined #puppet-openstack
13:32 mattymo beddari, I hope we release our first draft bittorrent solution
13:33 mattymo it's still a POC that only works with ubuntu
13:35 dvanders joined #puppet-openstack
13:36 beddari the tracker is still a SPOF then, right? and that is after having PXE-booted all
13:47 mattymo of course it's a failure
13:47 mattymo but it's really not a load problem, not availability problem
13:47 mattymo tiny tiny web requests like torrent tracker is not high load
13:48 mattymo wow my sentence doesn't make much sense. we want to reduce load on provisioning server and speed up overall deployment
13:48 beddari mattymo: :) yeah, so the swarm approach seems like a good idea, yes
13:49 beddari mattymo: and before serfdom.io I wouldn't know of any easy-to-get alternatives
13:49 xingchao joined #puppet-openstack
13:49 mattymo what I'm nervous about is you have to make this initial boot env really small to make the benefits worthwhile
13:49 mattymo but if it's too small, we can't reboot the node via orchestration (unless ipmi is available)
13:50 mattymo and openstack can be deployed on commodity hardware, so some customers won't have it
13:50 beddari mattymo: ah that would not be good to require, no
13:52 beddari mattymo: if you did say a 10 divide on the total number of hosts, then set up proxies or what you'd call them first, things would go faster. with serfdom.io you could even make a small protocol on top of it yourself, have all <1000 hosts message "provision_me $host" first, then introduce one that answers say three of them with "provisioning_you $local $host" and turn them into proxies too, etc
13:53 beddari (but that part would only be useful past PXE of course)
13:53 beddari you have gpxe or similar on your first image I guess
13:54 mattymo yeah I was expecting something just like this
13:54 beddari mattymo: I think say up to 10 dhcp+tftpservers serving on the same subnet (with IPs auto-divided somehow) would work for the first step
13:55 mattymo there are 3 ways to segment and distribute tftp traffic: 1 via pre-configuring "next-server" field per host, 2 by configuring different scopes with next-server field, or 3 with some sort of ip anycast type voodoo
13:55 mattymo but I don't beleive ip anycast works on an l2 segment
13:56 xingchao joined #puppet-openstack
13:57 beddari hmm ah ok so the dhcp reply itself wouldn't need to be duplicated I guess (although I use this for avail)
13:57 beddari eh duplicated, was saying, you don't really need more than a single dhcp
13:57 beddari +d
13:58 mattymo if you put 2 dhcp servers on a network, whoever replies first wins
13:58 mattymo so you can balance load to some extent, but really extra broadcast traffic is bad
13:58 beddari yes I use that plus static mac->ip assignments
14:06 dmsimard joined #puppet-openstack
14:10 beddari mattymo: so this is the new code for splitting up serving 'templates' (OSes) https://github.com/theforeman/foreman/pull/751
14:10 beddari mattymo: afaik it does not yet have anything to split up serving pxe ..
14:11 beddari mattymo: I said tftp? ;) splitting up serving tftp
14:11 mattymo it's proxying for all services
14:11 mattymo tftp dns dhcp
14:11 mattymo it includes pxe then
14:11 mattymo pxe is dhcp + tftp
14:11 mattymo it doesn't currently proxy http traffic though
14:12 mattymo which would actually proxy a full install
14:12 beddari yeah, but I think proxying in this regards means stand up a single proxy somewhere to provide this outside of the "foreman server"
14:13 beddari I'm not sure as I've never had a need to split that up (and I don't think many has) ;-)
14:19 mjblack joined #puppet-openstack
14:22 mattymo beddari, fyi I forgot we published this doc too http://docs.mirantis.com/fuel-dev/develop/env.html
14:22 mattymo a little bit more about how to start developing on Fuel
14:23 beddari nice :)
14:24 beddari I'll see if I can hack together a bootstrap protocol example/POC in Vagrant
14:34 mjblack_ joined #puppet-openstack
14:34 rongze joined #puppet-openstack
14:35 rongze joined #puppet-openstack
14:36 prad_ joined #puppet-openstack
14:40 mattymo a nice-to-have goal would be a light boot env that boots via tftp in under 50mb
14:40 mattymo and can be orchestrated
15:08 ryanycoleman joined #puppet-openstack
15:20 mgagne joined #puppet-openstack
15:23 dachary mgagne: good morning
15:24 mgagne dachary: hi
15:31 ryanycoleman joined #puppet-openstack
15:51 hogepodge joined #puppet-openstack
15:59 _ilbot joined #puppet-openstack
15:59 Topic for #puppet-openstack is now Place to collaborate on Puppet/OpenStack tools: logs at http://irclog.perlgeek.de/puppet-openstack/today
16:01 rongze joined #puppet-openstack
16:12 rongze joined #puppet-openstack
16:36 badiane_ka joined #puppet-openstack
16:36 blentz joined #puppet-openstack
16:38 tnoor1 joined #puppet-openstack
16:40 tnoor2 joined #puppet-openstack
16:51 lnxnut joined #puppet-openstack
16:56 lnxnut joined #puppet-openstack
16:59 comptona joined #puppet-openstack
17:37 mgagne joined #puppet-openstack
17:37 mgagne joined #puppet-openstack
17:44 rongze joined #puppet-openstack
17:54 rmoe joined #puppet-openstack
18:16 bodepd_ mgagne: good morning!
18:16 bodepd_ dachary: I didn't really understand how to help your student helpers :)
18:16 bodepd_ dachary: I pointed them back at you
18:17 dachary bodepd_: you did well, it can be challenging to figure out how to help out beginners
18:18 dachary we're re-iterating the openstack 101 session to open the next summit. It was great in that regard ;-)
18:18 dachary dalgaaf: ping ?
18:18 dachary dmsimard: good morning sir :-)
18:21 mgagne bodepd_: hi
18:38 dmsimard dachary: good morning? it's almost 3 pm :p
18:38 dachary dmsimard: it's 7:38pm here ;-)
18:38 dachary dmsimard: if you feel like approving https://review.openstack.org/#/c/53566/ it looks like it's ready ;-)
18:39 dachary And a dedicated hardware is setup to continuously integrate from there
18:40 openstackgerrit A change was merged to stackforge/puppet-ceph: integration tests environment  https://review.openstack.org/53566
18:42 dachary \o/
18:44 bodepd_ dachary: now, we need the same for all of the openstack components :)
18:44 bodepd_ dachary: let me know when you have that completed :)
18:44 dachary let see how it goes with this one first
18:44 dmsimard dachary: I was questioning myself if 4GB RAM and 10GB space were too much…
18:44 dmsimard could probably fit in 2GB RAM and 8GB space
18:45 dachary probably
18:49 rongze joined #puppet-openstack
18:55 openstackgerrit Loic Dachary proposed a change to stackforge/puppet-ceph: add dependencies  https://review.openstack.org/54034
18:57 dachary please ignore the spam, activating ci
18:57 openstackgerrit Loic Dachary proposed a change to stackforge/puppet-ceph: add dependencies  https://review.openstack.org/54034
19:05 dmsimard dachary: I'm not sure we want to add these to the modulefile, maybe bodepd_ and mgagne can weigh in
19:05 mgagne you don't need them in Puppetfile
19:06 mgagne only fixtures
19:06 openstackgerrit Loic Dachary proposed a change to stackforge/puppet-ceph: add dependencies  https://review.openstack.org/54034
19:06 dachary mgagne: ok. I will amend. The merit of this review is mostly as a guinea pig for integration ;-)
19:07 dmsimard Ah, puppetfile, yeah, I mixed up modulefile and puppetfile.
19:07 mgagne dachary: better create an empty change for test purpose
19:08 dachary mgagne: ok
19:08 mgagne dachary: see what people are doing to test CI and stuff: https://review.openstack.org/#/c/29837/
19:09 dachary https://review.openstack.org/#/c/54034/ has the first integration test result :-)
19:09 dachary \o/
19:09 mgagne dachary: otherwise your change could be mistaken for a genuine change and get merged
19:09 mgagne dachary: NICE!
19:10 dachary mgagne: thanks for the example
19:13 dachary it's running as daemon now, as follows GEM_HOME=~/.gems ; gerritexec --hostname review.openstack.org            --username puppetceph            --script ' ( bundle install ; bundle exec rake spec:system ) > /tmp/out 2>&1 ; r=$? ; pastebinit /tmp/out ; exit $r # '            --project stackforge/puppet-ceph
19:14 dachary the setup is explained at http://dachary.org/?p=2448
19:14 ryanycoleman joined #puppet-openstack
19:22 bauzas joined #puppet-openstack
19:23 dachary from where does bundle figure out what it needs to install ?
19:23 dachary ( as in bundle install )
19:23 mgagne dachary: Gemfile
19:23 dachary thanks
19:24 mgagne http://bundler.io/v1.3/gemfile.html
19:25 badiane_ka joined #puppet-openstack
19:28 openstackgerrit Loic Dachary proposed a change to stackforge/puppet-ceph: development installation instructions  https://review.openstack.org/54034
19:32 feandil joined #puppet-openstack
19:38 iwi joined #puppet-openstack
19:44 dalgaaf dachary:  pong
19:48 dalgaaf dachary: sorry ... I'm a littlebit busy these days ... have to solve some performance issues with ceph on our setup
19:51 rongze joined #puppet-openstack
20:02 ryanycoleman joined #puppet-openstack
20:11 dachary dalgaaf: no worries, good luck with that.
20:21 ryanycoleman joined #puppet-openstack
20:23 ryanycoleman joined #puppet-openstack
20:26 openstackgerrit Loic Dachary proposed a change to stackforge/puppet-ceph: ceph::repo integration tests  https://review.openstack.org/54176
20:39 openstackgerrit Loic Dachary proposed a change to stackforge/puppet-ceph: ceph::repo integration tests  https://review.openstack.org/54176
20:41 openstackgerrit joined #puppet-openstack
20:44 openstackgerrit Loic Dachary proposed a change to stackforge/puppet-ceph: ceph::repo integration tests  https://review.openstack.org/54176
20:45 mgagne dachary: mind if I suggest updating any references to bug to "Closes-bug: #123456" ? There is automagic associated with this syntax.
20:45 dachary mgagne: thanks for the tip
20:45 mgagne dachary: https://wiki.openstack.org/wiki/GitCommi​tMessages#Including_external_references
20:46 tnoor1 joined #puppet-openstack
20:47 openstackgerrit Loic Dachary proposed a change to stackforge/puppet-ceph: ceph::repo integration tests  https://review.openstack.org/54176
20:47 tnoor1 joined #puppet-openstack
20:49 tnoor2 joined #puppet-openstack
20:50 otherwiseguy joined #puppet-openstack
20:52 rongze joined #puppet-openstack
21:31 dachary dmsimard: thanks for your review. Do you have a suggestion for improvement so I can get a +2 from you ?
21:35 dmsimard dachary: not at this time, I browsed quickly but really busy recently - someone else can +2/approve :)
21:37 dachary dmsimard: fair enough :-)
21:38 dmsimard dachary: are you going to close https://review.openstack.org/#/c/54034/ ?
21:40 dachary dmsimard: no, I think it's a useful thing to have. I used it for tests which confused things but now it's good. The next time I want to test something I'll follow mgagne advice.
21:46 ryanycoleman joined #puppet-openstack
21:52 rongze joined #puppet-openstack
22:14 lnxnut_ joined #puppet-openstack
22:28 Kupo24z joined #puppet-openstack
22:28 Kupo24z Hey All, tried to configure xen libvirt type and now I'm getting 'Connection to libvirt failed: no connection driver available for No connection for URI xen:///', is there a step im missing?
22:29 Kupo24z this is with Cent6.4 after doing http://wiki.centos.org/HowTos/Xen/Xen4QuickStart
22:45 xarses joined #puppet-openstack
22:49 bodepd_ Kupo24z: It's probably been ages since anyone used the xen code
22:50 bodepd_ it looks like it is missing the xen connection code
22:50 bodepd_ https://github.com/stackforge/puppet-nova/​blob/master/manifests/compute/xenserver.pp
22:50 bodepd_ actually, I think smokestack is testing it
22:51 bodepd_ Kupo24z: you probably need to look at that class and see if it adds the config you are missing
22:53 rongze joined #puppet-openstack
22:55 lnxnut joined #puppet-openstack
23:03 ryanycoleman joined #puppet-openstack
23:14 tnoor1 joined #puppet-openstack
23:30 ryanycoleman joined #puppet-openstack
23:56 rongze joined #puppet-openstack

| Channels | #puppet-openstack index | Today | | Search | Google Search | Plain-Text | summary