Perl 6 - the future is here, just unevenly distributed

IRC log for #fuel, 2014-01-09

| Channels | #fuel index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
23:04 e0ne joined #fuel
23:12 angdraug albionandrew: around?
23:13 angdraug remember that cciss issue you brought up yesterday?
23:13 angdraug can you run "lsblk -ln" and see if there's any bangs (!) in the output?
23:13 albionandrew We destroyed that deployment... sorry
23:14 angdraug you don't need openstack on it, just a centos6 installed on any of the nodes
23:14 angdraug did you wipe the master node, too?
23:15 albionandrew No I don't think so.
23:15 angdraug too bad. we have a theory of another place were a special case is needed for cciss, wanted to confirm
23:16 angdraug thanks anyway, let me know if you see that problem again and we could get back to it
23:20 albionandrew angdraug I have worked out we have a failing computer node. It passed all the smoke tests. Any suggestions on how to proceed.
23:20 e0ne joined #fuel
23:22 angdraug define failing
23:22 albionandrew Cant run an instance on it.
23:22 angdraug failing hardware? failing nova?
23:22 albionandrew Goes straight to error
23:22 albionandrew Thought the issue was the availability zone
23:22 angdraug is that just one node or all of them?
23:23 albionandrew One node.
23:23 albionandrew node-19
23:23 albionandrew took it out put in new az and instances failed.
23:23 rongze joined #fuel
23:24 albionandrew swapped nodes so that what was in nova is in new zone
23:24 albionandrew instances booted.
23:24 albionandrew problem followed node
23:27 albionandrew What if I delete it? then re add it ?
23:28 angdraug might help
23:29 angdraug anything useful or interesting in nova logs on that node?
23:29 angdraug did you try to enable debug in nova configs?
23:30 albionandrew There is nothing in there [root@node-19 ~]# cat /var/log/nova/nova.log | wc
23:30 albionandrew 0       0       0
23:30 angdraug is debug enabled? syslog disabled?
23:30 angdraug you also need to specify log file path
23:31 albionandrew in nova.conf?
23:32 angdraug in /etc/nova*.conf
23:32 rmoe do you have anything in /var/log/nova-all.log?
23:32 albionandrew On the bad compute? ...Noi
23:32 albionandrew no
23:33 albionandrew | nova-compute     | node-22.raulmadeupstore.com | MYAVZ  | enabled | up    | 2014-01-08T23:33:14.000000 | None            |
23:33 albionandrew | nova-compute     | node-19.raulmadeupstore.com | nova     | enabled | up    | 2014-01-08T23:33:14.000000 | None            |
23:34 albionandrew nova service-list | grep compute
23:34 albionandrew node-19 is the one that doesn't seem to be working
23:41 albionandrew rmoe angdraug  - I'm about to delete the node and then redeploy ... unless you have any other suggestions?
23:44 angdraug I'd first try enabling debug and restarting nova
23:45 angdraug might save you time in case it's not a one-off error
23:45 albionandrew just deleted.
23:45 angdraug nevermind than, do that if it comes back up still broken
23:45 albionandrew enable debug in nova.conf on the bad node?
23:45 albionandrew if it happens again?
23:45 angdraug yeah, but I'd do all nova config files, not just nova.conf
23:45 angdraug most importantly nova-compute.conf
23:46 angdraug most likely that's where the exception would be
23:46 albionandrew ok Thanks
23:46 angdraug remember to do 3 things:
23:46 angdraug 1) enable debug
23:46 angdraug 2) disable syslog
23:46 angdraug 3) specify log_file path
23:46 angdraug then restart the corresponding service, of course
23:46 albionandrew ok.I'll most likely check in with you tomorrow.
23:46 angdraug applies to any openstack component that gives you trouble without saying anything in the logs
23:46 angdraug sure thing
00:23 rongze joined #fuel
00:46 IlyaE joined #fuel
01:05 jouston_ joined #fuel
01:17 e0ne joined #fuel
01:23 e0ne joined #fuel
01:23 rongze joined #fuel
01:44 jouston_ joined #fuel
01:59 rongze joined #fuel
02:13 rmoe joined #fuel
02:15 meow-nofer__ joined #fuel
02:18 xarses joined #fuel
02:24 e0ne joined #fuel
02:25 kevein joined #fuel
02:39 rmoe joined #fuel
03:11 kevein joined #fuel
03:25 e0ne joined #fuel
03:30 ArminderS joined #fuel
03:51 dnovakovskiy joined #fuel
03:54 dnovakovskiy hey guys. looking at http://docs.mirantis.com/fuel/fuel-4.0/pdf/Mirantis-OpenStack-4.0-ReferenceArchitecture.pdf and feeling a bit confused with NICs example
03:55 dnovakovskiy <quote>
03:55 dnovakovskiy eth0:
03:55 dnovakovskiy The internal management network, used for communication with Puppet & Cobbler
03:55 dnovakovskiy eth1:
03:55 dnovakovskiy The public network, and floating IPs assigned to VMs
03:55 dnovakovskiy eth2:
03:55 dnovakovskiy The private network, for communication between OpenStack VMs, and the bridge interface (VLANs)
03:55 dnovakovskiy The figure below illustrates the relevant nodes and networks in Neutron VLAN mode
03:55 dnovakovskiy </quote>
03:55 dnovakovskiy eth0 obviously has PXE traffic
03:56 dnovakovskiy but since in example it's tagged as "management" - does it mean that after deployment it's "converted" to management net, running services management trafic?
04:06 sparc joined #fuel
04:19 dhblaz joined #fuel
04:24 dhblaz joined #fuel
04:25 e0ne joined #fuel
04:49 bas_ joined #fuel
04:57 jouston_ joined #fuel
05:02 kevein joined #fuel
05:18 rongze joined #fuel
05:23 rongze joined #fuel
06:23 rongze joined #fuel
06:24 rongze_ joined #fuel
06:27 e0ne joined #fuel
06:38 oguz joined #fuel
06:46 dnovakovskiy joined #fuel
07:18 SergeyLukjanov joined #fuel
07:25 e0ne joined #fuel
07:48 evgeniyl joined #fuel
07:48 SergeyLukjanov joined #fuel
07:52 kevein joined #fuel
08:06 Yuri__ joined #fuel
08:07 Yuri__ left #fuel
08:43 zasimov joined #fuel
08:54 e0ne joined #fuel
08:54 AndreyDanin joined #fuel
09:05 yuri__ joined #fuel
09:06 yuri__ Hello
09:07 evgeniyl yuri__: hi
09:08 yuri__ I have some questions regarding Fuel and I was directed here, could you help me?
09:11 evgeniyl yuri__: yes, you can ask questions here, but part of our developers still on the vacation, so it depends on your questions can we help you or not :)
09:13 yuri__ cool
09:13 yuri__ Can we update Fuel?
09:14 yuri__ we tested several installations lately but did not see how we could update fuel 3.2 to 4.0 for example
09:18 mrasskazov1 joined #fuel
09:20 evgeniyl yuri__: Sorry, but right now we don't support fuel/openstack upgrading, we planning to start implementation of this feature (fuel\openstack upgrading between minor releases) in the current release cycle.
09:20 evgeniyl
09:20 evgeniyl
09:21 AndreyDanin yuri__, Also we don't have a documentation like 'steps to success' about that.
09:21 yuri__ ok.
09:23 yuri__ We currently have a very simple cluster running to see what runs where etc. (one controller, one compute, 2 ceph nodes)
09:23 yuri__ it seems a lot of services are installed on the controller node.
09:23 yuri__ can we for example tell fuel to install the ceph-mon service on the ceph machine instead of the controller?
09:24 yuri__ or on a different host...
09:24 yuri__ in or setup, the busiest machine is the controller node.
09:24 yuri__ by far..
09:25 SergeyLukjanov joined #fuel
09:26 xdeller joined #fuel
09:28 rvyalov joined #fuel
09:30 xdeller yuri__: can we for example tell fuel to install the ceph-mon service on the ceph machine instead of the controller? - not right now
09:31 xdeller will be possible in future or you can just add monitor manually or even migrate to the new set of monitors placed anywhere you want outside controllers
09:31 yuri__ I guess it's the same with changing settings afterwards like the hypervisor used, network settings, etc etc
09:32 yuri__ ok, cool, and nothing special gads to be done in fuel? will it accept a foreign set of mon services and allow us to remove the old ones?
09:35 teran joined #fuel
09:36 xdeller I`m not talking about fuel deployment phase when mentioned such possibility
09:36 xdeller of course it will override it
09:36 xdeller I mean some postinstall actions
09:36 xdeller check ceph.com for Adding/Removing Monitors actions
09:37 yuri__ adding min's etc is not the issue, but it's good to know that changing the setup involves modifying puppet.
09:37 xdeller you can add, say, new three for minimal HA quorum and delete old three
09:38 yuri__ indeed.
09:38 yuri__ 2 more questions.
09:38 yuri__ do you have any guidelines / best practices for monitoring of the openstack cluster?
09:39 yuri__ also, there is a Puppet module for nagios on the fuel server, but it is commented out. Are there plans to add this again in the future?
09:39 xdeller ehm, not right now. we managed to use nagios in the past and will use zabbix in the future
09:39 xdeller I can draw only most vulnerable points - mysql and rabbitmq cluster status
09:39 xdeller and of course disk space everywhere
09:44 yuri__ So for us it would be wise to test with zabbix. Since you're talking about the future I assume there is no documentation etc yet?
09:46 bogdando is it possible to reset state of unallocated nodes from error to new?
09:47 bogdando I want to reuse some nodes in new env. But I cannot add them from UI
09:47 xdeller yes, it will come in the 'near' future
09:47 bogdando (cuz of Error state)
09:47 bogdando I can add them to new env form CLI tho, but cannot reconfigure its interfaces
09:47 bogdando all bindings looks locked
09:48 bogdando btw, I've opened an issue regarding different CLI vs UI behavior
09:49 bogdando https://bugs.launchpad.net/fuel/+bug/1265036
09:50 AndreyDanin bogdando, they are booted as a bootstrap nodes now?
09:53 bogdando hm, no, I can log in to OS was provisioned before
09:53 bogdando but UI and CLI shows them as unallocated
09:54 bogdando and status error (its ok, I've made it to test statuses)
09:57 AndreyDanin How did you do it? Steps to reproduce?
09:58 bogdando see an issue referenced above
09:59 bogdando provisioned via CLI, deployed via CLI with interrupting - to put in Error state, removed env as well as nodes
09:59 bogdando but how to bootstrap them now forcibly?
10:07 bogdando looks like removing nodes from env via CLI puts them in broken (unreusable) state
10:13 bogdando ok, I've managed to clean and bootstrap them, by manually adding them to env by CLI (cannot use CLI to add nodes in error state), and by removing env
10:14 bogdando and state now is discover
10:23 AndreyDanin How did you interrupt nodes  to force them to the error state?
10:26 bogdando make deployment to fail
10:29 e0ne_ joined #fuel
10:52 mrasskazov1 joined #fuel
11:34 teran joined #fuel
12:04 MiroslavAnashkin joined #fuel
12:15 jouston joined #fuel
12:16 MiroslavAnashkin joined #fuel
12:19 MiroslavAnashkin joined #fuel
12:21 rongze joined #fuel
12:22 rongze_ joined #fuel
12:43 evgeniyl joined #fuel
13:02 getup- joined #fuel
13:17 rongze joined #fuel
13:24 MiroslavAnashkin joined #fuel
14:09 _ilbot joined #fuel
14:09 Topic for #fuel is now Fuel for Openstack: http://fuel.mirantis.com/ | Paste here http://paste.openstack.org/ | IRC logs http://irclog.perlgeek.de/fuel/
14:19 aleksandr_null joined #fuel
14:23 SergeyLukjanov joined #fuel
14:37 evgeniyl joined #fuel
15:18 yuri2 joined #fuel
15:21 yuri2 joined #fuel
15:27 IlyaE joined #fuel
15:27 kpimenova joined #fuel
15:36 MiroslavAnashkin joined #fuel
15:40 MiroslavAnashkin joined #fuel
15:48 yuri2 joined #fuel
15:52 syt joined #fuel
16:02 albionandrew joined #fuel
16:23 vk joined #fuel
16:28 miguitas joined #fuel
16:42 evgeniyl` joined #fuel
16:57 syt joined #fuel
17:06 SebastianKreutzb joined #fuel
17:17 angdraug joined #fuel
17:22 syt joined #fuel
17:39 albionandrew angdraug figured out the problem with the compute node - Kvm was turned off. We have updated a lot of firmware on a lot of equipment and I wonder if the settings got switched on this one machine?
17:44 xarses joined #fuel
18:08 rmoe joined #fuel
18:10 teran joined #fuel
18:19 syt joined #fuel
18:26 dhblaz joined #fuel
18:28 miguitas joined #fuel
19:01 e0ne joined #fuel
19:14 syt joined #fuel
19:21 angdraug albionandrew: afaik the qemu manifest is checking the flags in /proc/cpuinfo and would disable kvm if the required flags are not there
19:22 albionandrew angdraug thanks
19:23 albionandrew angdraug on another note do you have a how to on removing nodes? The compute I destroyed still shows on the service list
19:24 albionandrew I found something about remove it with mysql but I still see the node in there.
19:31 syt you need to remove them from 3 tables..
19:40 IlyaE joined #fuel
19:51 pvbill joined #fuel
20:09 SergeyLukjanov joined #fuel
20:09 tsduncan joined #fuel
20:13 syt joined #fuel
20:58 Bomfunk joined #fuel
21:31 vk joined #fuel
22:04 rmoe joined #fuel
22:31 e0ne joined #fuel

| Channels | #fuel index | Today | | Search | Google Search | Plain-Text | summary