Perl 6 - the future is here, just unevenly distributed

IRC log for #fuel, 2015-11-18

| Channels | #fuel index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:29 sergmelikyan joined #fuel
00:52 shimura_ken joined #fuel
01:00 sergmelikyan joined #fuel
01:06 thumpba joined #fuel
01:06 zhangjn joined #fuel
01:12 serg_melikyan joined #fuel
01:27 az__ joined #fuel
01:30 az joined #fuel
01:33 jaypipes joined #fuel
01:34 rmoe joined #fuel
01:45 xarses joined #fuel
01:58 Liuqing joined #fuel
02:29 jerrygb joined #fuel
02:52 shimura_ken joined #fuel
02:53 yantarou____ joined #fuel
02:55 Liuqing joined #fuel
02:58 rmoe joined #fuel
03:35 yantarou joined #fuel
04:22 fedexo joined #fuel
04:38 zhangjn joined #fuel
04:38 tzn joined #fuel
05:14 zerda joined #fuel
05:26 zhangjn_ joined #fuel
05:38 zhangjn joined #fuel
06:11 Liuqing joined #fuel
06:13 mmalchuk joined #fuel
06:30 angdraug joined #fuel
06:56 elemoine joined #fuel
07:06 Liuqing joined #fuel
07:12 xarses joined #fuel
07:12 xarses joined #fuel
07:14 gals joined #fuel
07:15 gals left #fuel
07:19 gals_ joined #fuel
07:24 subscope joined #fuel
07:25 jerrygb joined #fuel
07:32 ppetit joined #fuel
07:38 fzhadaev1 joined #fuel
07:46 angdraug joined #fuel
07:50 Miouge_ joined #fuel
07:59 mkwiek07 joined #fuel
08:04 tzn joined #fuel
08:07 javeriak joined #fuel
08:17 alex_didenko joined #fuel
08:23 hyperbaba joined #fuel
08:24 Liuqing joined #fuel
08:29 javeriak_ joined #fuel
08:35 _tzn joined #fuel
08:36 nihilifer joined #fuel
08:52 rmoe joined #fuel
08:54 Liuqing joined #fuel
08:59 LinusLinne joined #fuel
09:01 sand__ joined #fuel
09:05 Billias joined #fuel
09:09 akislitsky_ joined #fuel
09:30 Miouge_ joined #fuel
09:32 e0ne joined #fuel
09:36 LinusLinne left #fuel
09:36 LinusLinne joined #fuel
09:37 xek joined #fuel
09:37 LinusLinne Hi, having trooble deploying Mirantis 7.0 with VmWare, keep getting bellow error. Deploying with Fuel 6.1 works without error
09:38 LinusLinne Deployment has failed. Method granular_deploy. Failed to execute hook 'shell' command: cd / && /usr/bin/python /etc/puppet/modules/osnailyfacter/modular/astute/vcenter_hooks.py --create_zones
09:39 LinusLinne I saw a bug report about this but it was classified as not valid, however i cannot se what the error should be? Have tried this on 2 diffrent vmware installations.
09:46 tkhno joined #fuel
10:03 Liuqing joined #fuel
10:14 LinusLinne joined #fuel
10:23 LinusLin_ joined #fuel
10:26 jerrygb joined #fuel
10:28 idvoretskyi__ joined #fuel
10:30 LinusLinne joined #fuel
10:31 az joined #fuel
10:42 tzn joined #fuel
10:42 tzn joined #fuel
10:51 zhangjn joined #fuel
11:00 sergmelikyan joined #fuel
11:04 e0ne joined #fuel
11:05 jeblair joined #fuel
11:07 sergmelikyan joined #fuel
11:09 omolchanov joined #fuel
11:14 Chlorum joined #fuel
11:25 sergmelikyan joined #fuel
11:29 mkwiek07 joined #fuel
11:31 alex_didenko joined #fuel
11:31 sergmelikyan joined #fuel
11:42 subscope joined #fuel
12:07 tzn joined #fuel
12:09 zhangjn joined #fuel
12:10 zhangjn joined #fuel
12:10 zhangjn joined #fuel
12:16 Miouge_ joined #fuel
12:20 neilus joined #fuel
12:21 subscope joined #fuel
12:26 fzhadaev joined #fuel
12:27 aarefiev joined #fuel
12:57 ppetit joined #fuel
13:06 jerrygb joined #fuel
13:10 _gryf joined #fuel
13:10 _gryf hi.
13:10 e0ne joined #fuel
13:14 tab joined #fuel
13:15 nihilifer joined #fuel
13:16 Guest99452 Reading the documentation on upgrading the Openstack (https://www.mirantis.com/blog/yes-can-upgrade-openstack-heres/) i am interested is there any new updates to automating Openstasck deployments, possible through Fuel?
13:16 nihilifer joined #fuel
13:18 Guest99452 is there any possible downtime expected on updates?
13:21 angdraug joined #fuel
13:30 Wida joined #fuel
13:31 Liuqing_ joined #fuel
13:32 Wida Hello! How I can change self-signed certificate to wildcard cert in mirantis openstack 7.0 ?
13:33 Wida self-signet cert generated automatically in fuel when I install openstack
13:33 gerst Wida: in the installation settings these is a place to add your own certificate instead of the autogenerated
13:36 bpiotrowski Guest99452: that article is quite old
13:36 bpiotrowski Guest99452: Mirantis developed a tool called octane in the meantime, here's the demo: https://www.youtube.com/watch?v=mOZA4fRQOBI
13:38 bpiotrowski Guest99452: and here is some documentation on it: https://docs.mirantis.com/openstack/fuel/fuel-6.1/operations.html#upgrade-environment-experimental
13:38 Guest99452 bpiotrowski: thank you for info! very grateful
13:38 Guest99452 so this tool is an option to update openstack, regardless the number of controller nodes?
13:39 bpiotrowski from what I know, yes
13:41 bpiotrowski IIRC ogelbukh knows more about it, so you should probably ping him in case of any questions
13:43 bpiotrowski Guest99452: and I accidentally gave you link to 6.1, 7.0 is up to date version: https://docs.mirantis.com/openstack/fuel/fuel-7.0/operations.html#upgrade-environment-experimental
13:48 Wida gerst: I should redeploy openstack now to change cert?
13:48 gerst Wida: think there might be a manual way to upgrade if its already deployed, let me check
13:49 gerst I mean switch the cert :P
13:49 Guest99452 bpiotrowski: thx
13:52 gerst Wida: https://docs.mirantis.com/openstack/fuel/fuel-7.0/operations.html#horizon-dashboard-and-the-openstack-publicurl-endpoints
13:53 gerst Changing keypairs for a cluster -- There is currently no automated way to do this. You can manually change the keypairs in /var/lib/astute/haproxy/public_haproxy.pem on Controller and Compute nodes. Make sure you restart the HAProxy service after you edit the file.
13:55 bpiotrowski also remember to restart it via pcs – not sure if it's mentioned in the docs as my ISP doesn't like me today
13:55 Guest99452 Maybe also this question. Let's say i prepare Fuel USB for Openstack Deployment. Now I want to have some additional configuration applied after the installation. Is it best way to do it through the system of Fuel plugins or to have some additional script which would run at the end of the fuel installation and would copy config files.?
13:56 Guest99452 What about if I want to have some additional images within glance (besided TinyOS), let's say that i want to have already available centos image withing glance repository for VMs. What would be the best way to do this, make additional puppet scripts, that would install such an image within already set-up openstack deployment?
13:57 bpiotrowski both ways are fine for simple things
13:58 bpiotrowski I'd lean towards a plugin for better flexibility
13:58 Guest99452 ok
14:00 Guest99452 bpiotrowski: maybe also this question. when doing upgrade of OS, when having 3 controllers set-up, do i need additional 3 new HWs to do update without downtime?
14:01 bpiotrowski from I saw in the documentation, no
14:01 Guest99452 aha i see yes. i first stared with youtube :)
14:01 bpiotrowski you will lose true HA for some time though, as controllers are moved to updated environment
14:01 Guest99452 ok thx
14:02 Guest99452 so no downtime, but loosing HA
14:05 ppetit_ joined #fuel
14:08 bapalm joined #fuel
14:16 fzhadaev joined #fuel
14:21 ppetit joined #fuel
14:22 NelsonPR joined #fuel
14:23 jaypipes joined #fuel
14:27 angdraug joined #fuel
14:28 sergmelikyan joined #fuel
14:30 bebebeko joined #fuel
14:31 akscram joined #fuel
14:31 ppetit joined #fuel
14:43 angdraug joined #fuel
14:43 bapalm joined #fuel
14:49 ppetit joined #fuel
15:07 RageLtMan joined #fuel
15:08 subscope joined #fuel
15:17 angdraug joined #fuel
15:20 blahRus joined #fuel
15:21 ppetit joined #fuel
15:30 ppetit joined #fuel
15:31 Verilium Hmm, tried a deploy with 7.0 and after 6 or so hours still running the deploy, with no noticeable errors while looking real quickly, decided to stop it.  And it seems it's been stuck stopping for the last hour.
15:31 Verilium Anything I can do to "force" stoppage?
15:32 mwhahaha you could using the cli command
15:33 claflico joined #fuel
15:33 mwhahaha fuel task list
15:33 mwhahaha then you could try to force delete it
15:34 mwhahaha fuel task delete --force --task ##
15:34 Verilium deployment/provision/stop_deployment are all in running state.
15:35 mwhahaha similar to what you're asking, https://answers.launchpad.net/fuel/+question/258562
15:36 Wida joined #fuel
15:37 dsutyagin joined #fuel
15:38 Wida Hi! I have already deployed openstack. How I can change all Service Endpoint (for example: "https://openstack.mydomain.com:5000/v2.0") to new domain name without redeploy?
15:39 anddrew joined #fuel
15:42 Verilium mwhahaha:  Interesting.  In my case, wasn't even that far yet.  Ubuntu 80%.  Strange.
15:42 anddrew Hey guys! I deplyed an openstack ha environment and everything was ok until i had to reboot a controller, the primary one actually. My deplyment has 3 controllers. I can't start the mysql service. if i run start p_mysql in crm resource it doesn't start, and  if i run /etc/init.d/mysql start it shows permission denied.
15:42 Verilium Well, the force stoppage worked.  Resetting now.
15:42 mwhahaha Verilium: yea but same way to deal with tasks
15:42 mwhahaha anddrew: use crm command ot manage mysql not init script
15:42 anddrew The mysql file exists in /etc/init.d, but it isn't executable
15:43 mwhahaha yea cause you shouldn't use it
15:43 mwhahaha so if it's not starting, check the mysql error logs
15:43 xarses joined #fuel
15:43 mwhahaha also if your mysql cluster is still up on the other nodes, you can use resource cleanup to reset it and it should join the cluster
15:45 anddrew ok, i will try that. No logs in mysqld.log
15:46 anddrew But it's normal for the fuel installation not to make the mysql file executable ?
15:46 mwhahaha not sure if we set it -x but it should not be used
15:47 anddrew So the only way to restart mysql is throw crm?
15:47 mwhahaha correct
15:47 mwhahaha it is managed via crm
15:47 ericjwolf joined #fuel
15:47 ppetit joined #fuel
15:50 mwhahaha 'crm resource cleanup clone_p_mysql' on the node that is broken should reset it (i think)
15:51 bpiotrowski actually what's the difference between crm and pcs?
15:51 mwhahaha i'm not actually sure, i've read that pcs should be used instead but who knows
15:51 mwhahaha seems inconsistent
15:52 mwhahaha pcs is provided via pacmaker and i think crm is corosync command set. they have a bunch of overlap
15:52 bpiotrowski yeah, I also felt there is an overlap
15:52 dsutyagin bpiotrowski: these are different command line tools which do mostly the same, but they differ a bit in functionality, some stuff related to corosync settings cannot be done via pcs, probably vice versa but not sure
15:52 bpiotrowski but I saw in our docs pcs a couple of times few releases ago, so I got confused
15:54 Verilium Both outputs seem so similar, I kept wondering if one was the 'older version' of the other or something.
15:54 Verilium http://www.nongnu.org/crmsh/faq.html
15:54 bpiotrowski https://github.com/ClusterLabs/pacemaker/blob/master/doc/pcs-crmsh-quick-ref.md looks like you can pretty much do the same with both
15:54 Verilium Ah...  " common misconception is that crm has been replaced by pcs (available at github.com/feist/pcs). pcs is an alternative command line interface similar to crm. Both projects are being actively developed, with slightly different goals."
15:55 Verilium Go figure.
15:55 ppetit joined #fuel
15:55 bpiotrowski okay, it's clear now
15:55 bpiotrowski thx
15:57 jerrygb joined #fuel
16:11 Wida joined #fuel
16:12 anddrew So...managed to start mysql-wss process, but now in logs it shows that node address is blacklisted..hmm
16:12 ppetit joined #fuel
16:13 elemoine joined #fuel
16:14 pmcg joined #fuel
16:24 aglarendil anddrew: did you start it with pcs?
16:26 anddrew crm
16:30 ppetit joined #fuel
16:42 aglarendil sorry, I do not have opportunity to read previous messages - what is your problem right now?
16:42 aglarendil you do not have mysql cluster running properly?
16:49 ppetit joined #fuel
16:50 aglarendil anddrew: ^^
16:52 ppetit joined #fuel
16:53 jaypipes joined #fuel
16:58 fedexo joined #fuel
17:00 anddrew aglarendil: yes, i can't start mysql , in crm status or pcs resource it shows stopped on all controller nodes
17:01 anddrew if i do a cleanup and start clone_p_mysql it starts the /usr/lib/ocf/resource.d/fuel/mysql-wss process, but nothing else
17:02 anddrew in mysqld.log i can see the last lines :  node-12 mysqld: 2015-11-18 10:13:56 10328 [Note] WSREP: (6b517d6c-8ddc-11e5-8468-d3c709dc8639, 'tcp://0.0.0.0:4567') address 'tcp://192.168.0.13:4567' pointing to uuid 6b517d6c-8ddc-11e5-8468-d3c709dc8639 is blacklisted, skipping
17:06 Guest70_ joined #fuel
17:08 aglarendil what is your wsrep cluster address parameter? does it contain the address being blacklisted?
17:08 aglarendil and, btw, which fuel version are you using?
17:08 aglarendil @holser I guess we could use some of your help here
17:08 anddrew wsrep_cluster_address="gcomm://192.168.0.13:4567,192.168.0.15:4567,192.168.0.16:4567?pc.wait_prim=no"
17:09 anddrew yes, the network is ok, i can ping each node
17:10 anddrew it can't be something in config, cuz it was no problem until i rebooted the controller node, and i havn't modify any configs manually
17:14 anddrew fuel version 6.1
17:33 aglarendil can you post crm status output into pastebin?
17:36 anddrew http://paste.openstack.org/show/479308/
17:41 anddrew Failed actions:     p_mysql_start_0 on node-12.ducadu.com 'unknown error' (1): call=2205, status=Timed Out, last-rc-change='Wed Nov 18 17:35:12 2015', queued=1ms, exec=300000ms
17:45 xarses joined #fuel
17:45 RageLtMan anddrew, i've seen something very similar to this on a single-controller-node setup as well, which didnt even have replica partners. unfortunately i didnt fully remediate since it was a test build and i just redeployed making it go away
17:46 xarses joined #fuel
17:50 anddrew sure, redeployment would surely resolve any issues....but what if this was in production?...
17:52 RageLtMan thats my question as well
17:53 RageLtMan galera scares me a bit, thats why i went single node controller with ZFS under it taking snapshots all the time so i can roll back state of a single instance and not worry about peering, after seeing that wsrep problem my trust for the replication topology is minimal
17:53 rmoe joined #fuel
17:53 aglarendil anddrew: I do not see your logs for galera, but I may suggest that you get latest wsrep ocf script and try with it
17:54 aglarendil with backing up the database of course
17:54 aglarendil in order to back your database up you will need to spin up one instance with disabled galera
17:54 jerrygb_ joined #fuel
17:54 anddrew i'm thinking of moving the resource from the affected node to another and start the resource there
17:55 jerrygb_ joined #fuel
17:59 aglarendil you can try
17:59 aglarendil well, actually, holser might be of help here
18:00 aglarendil he is our subject matter expert on wsrep OCF script
18:01 javeriak joined #fuel
18:01 jerrygb_ joined #fuel
18:01 holser RageLtMan: i can help
18:01 jerrygb_ joined #fuel
18:04 RageLtMan holser: i unfortunately no longer have the instance which was seeing this issue, but i can confirm i've seen it as well, and without peers to replicate to - a simple reboot of the controller node caused this
18:06 aglarendil holser: it is anddrew who needs help
18:07 javeriak_ joined #fuel
18:09 xarses_ joined #fuel
18:10 javeria__ joined #fuel
18:15 e0ne joined #fuel
18:17 Guest70_ joined #fuel
18:44 jerrygb joined #fuel
18:47 LinusLinne joined #fuel
19:02 sergmelikyan joined #fuel
19:02 az joined #fuel
19:09 poseidon1157 joined #fuel
19:11 poseidon1157 Hi all.  If I'm trying to reprovision a node, is it best just to power it off, remove it from the environment, and reprovision it when it comes back up?
19:12 javeriak joined #fuel
19:12 mwhahaha if it's already been provisioned if you can it would be better to do through the removal process from the environment because that'll trigger a cleanup of the disks, etc
19:13 mwhahaha if the node is totally fried and you want to manually clean up the disks, you can power off, remove it from the env then clear the disks and re-bootstrap it
19:15 javeriak_ joined #fuel
19:24 elemoine joined #fuel
19:25 javeriak joined #fuel
19:26 bapalm joined #fuel
19:49 elemoine_ joined #fuel
19:52 anddrew holser: if you can give me some indications...it will be much appreciated... :(
19:53 tzn joined #fuel
20:06 javeriak_ joined #fuel
20:06 subscope joined #fuel
20:10 subscope joined #fuel
20:18 javeriak joined #fuel
20:19 aglarendil anddrew: so, it would be good to understand what pacemaker is doing and why
20:19 aglarendil let's try it the following way
20:19 aglarendil you can make pacemaker not manage galera cluster
20:19 aglarendil for this run
20:19 aglarendil `crm resource unmanage clone_p_mysql`
20:19 aglarendil then we can try to start galer on one of the nodes and monitor script run in the realtime
20:20 aglarendil to do this, you will need to run following commands
20:20 aglarendil `export OCF_ROOT=/usr/lib/ocf`
20:20 aglarendil for each parameter in p_mysql resource configuration you will need to export
20:20 aglarendil export OCF_RESKEY_<param_name>=<param_value>
20:21 aglarendil then firstly we can run OCF script on our own
20:21 aglarendil '/usr/lib/ocf/resources.d/mirantis/mysql-wss <cmd>'
20:21 aglarendil we can start with status command and see script output
20:35 anddrew so...this is the output, http://paste.openstack.org/show/479340/
20:37 rmoe joined #fuel
20:41 bildz Anyone around to help me with a ceph issue?  I'm trying to upload images into glance and keep getting an error 500:  http://pastebin.com/E3LYWTT8
20:46 anddrew You should enable debug in glance config to get more information in logs
20:50 javeriak_ joined #fuel
20:55 bildz anddrew: thanks will try that
21:03 bildz anddrew: glance isnt the issue.  It's connecting to the ceph storage cluster
21:09 e0ne joined #fuel
21:09 DevStok joined #fuel
21:09 DevStok hi again
21:09 DevStok still have the port of public haproxy firewalled
21:10 DevStok what can i edit in order to open the openstack services port
21:12 jerrygb_ joined #fuel
21:20 alex_didenko joined #fuel
21:23 jerrygb_ joined #fuel
21:26 jerrygb__ joined #fuel
21:28 jerrygb joined #fuel
21:31 poseidon1157 joined #fuel
21:32 DevStok solved
21:32 DevStok damn haproxy conf file
21:36 mugleeta joined #fuel
21:47 e0ne joined #fuel
21:48 LinusLinne joined #fuel
21:55 Wida joined #fuel
21:56 Wida How I can change all Service Endpoint (for example: "https://openstack.mydomain.com:5000/v2.0") to new domain name without redeploy?
21:58 mwhahaha Wida: I'm not sure that's something you can change after the fact with fuel
21:58 holser anddrew: interesting ...
21:58 holser anddrew: do > export OCF_RESOURCE_INSTANCE=p_mysql
21:58 holser > export OCF_ROOT=/usr/lib/ocf
21:58 Wida only redeploy can help me?
21:59 holser bash -x /usr/lib/ocf/resources.d/mirantis/mysql-wss start
22:00 Wida so if it possible to turn off TLS (ssl) after deployed openstack?
22:01 mwhahaha holser: pretty sure it's .../resources.d/fuel/...
22:01 holser thanks mwhahaha
22:01 holser we renamed in 6.1 but I have old note still
22:02 holser anddrew: I do not see any critical issues in your log
22:02 holser anddrew: I need some more info such as ‘crm status’ or ‘pcs status'
22:05 DevStok @wida we are doing the same thing
22:06 Wida DevStok: as your success in this?
22:06 DevStok with kilo is ongoing
22:07 DevStok but with previous version we made it
22:07 ddmitriev joined #fuel
22:07 DevStok u have to change all config files of the services
22:07 DevStok nova neutron glance cinder swift
22:07 DevStok in order to get authenticate from another keystone
22:07 Wida I have kilo release
22:22 poseidon1157 joined #fuel
22:38 anddrew joined #fuel
22:47 jaypipes joined #fuel
23:05 jerrygb joined #fuel
23:08 tzn joined #fuel
23:14 jerrygb joined #fuel
23:18 tzn joined #fuel
23:32 Wida joined #fuel
23:53 Verilium Yeah.  I kept copy-pasting network + settings from one env to another when I wanted to duplicate...  And now, I stumble on the fact I can download/upload all of those settings at the CLI.
23:53 Verilium Now that should save me a bit of time going forward.

| Channels | #fuel index | Today | | Search | Google Search | Plain-Text | summary