Perl 6 - the future is here, just unevenly distributed

IRC log for #fuel, 2014-05-13

| Channels | #fuel index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
03:19 rmoe joined #fuel
03:30 rmoe joined #fuel
03:40 rmoe joined #fuel
04:05 Ch00k joined #fuel
05:47 artem_panchenko joined #fuel
06:43 al_ex joined #fuel
06:50 Ch00k joined #fuel
07:02 izinovik joined #fuel
07:11 al_ex joined #fuel
07:39 ykotko joined #fuel
07:44 topochan joined #fuel
07:50 odyssey4me joined #fuel
07:56 e0ne joined #fuel
08:47 geogdog joined #fuel
08:48 rmoe joined #fuel
08:55 e0ne joined #fuel
08:59 e0ne joined #fuel
09:04 e0ne joined #fuel
09:17 xdeller joined #fuel
10:26 e0ne joined #fuel
10:33 rmoe joined #fuel
11:09 zinovik joined #fuel
11:15 anotchenko joined #fuel
11:30 izinovik joined #fuel
11:52 rmoe joined #fuel
12:06 rmoe joined #fuel
12:45 justif joined #fuel
12:52 e0ne joined #fuel
12:57 anotchenko joined #fuel
13:03 topochan joined #fuel
13:15 b-zone joined #fuel
13:28 e0ne joined #fuel
14:32 xarses joined #fuel
15:29 e0ne joined #fuel
15:36 blahRus joined #fuel
15:42 xarses joined #fuel
16:16 e0ne joined #fuel
16:48 xdeller joined #fuel
16:56 Ch00k joined #fuel
17:03 rmoe joined #fuel
17:22 tatyana joined #fuel
17:46 rmoe joined #fuel
18:15 albionandrew joined #fuel
18:16 albionandrew What is the best way to troubleshoot a failure of the “Check network connectivity from instance via floating IP” in the health checks?
18:25 odyssey4me joined #fuel
18:26 thehybridtech joined #fuel
18:47 MiroslavAnashkin albionandrew: First, try to log in to instance and ping any external server, say google.com
18:49 albionandrew I tried that - No IP -  The l3 agent and dhcp should only be running on one node correct?
18:49 albionandrew MiroslavAnashkin ^
18:50 MiroslavAnashkin More accurate - single DHCP and L3 agent instances should run for each network
18:52 albionandrew IN production neutron list reports all smiles. I just checked. IN our lab lots of xxx
18:52 albionandrew can I use crm to restart eveything?
18:53 MiroslavAnashkin crm may nor restart MySQL
18:54 albionandrew but otherwise, log on to each node with the xxx and do a crm resource restart p_something ?
18:54 MiroslavAnashkin Please check free space on controllers first
18:55 albionandrew thats good. I think running out of space is what started this mess
18:55 Kupo24z joined #fuel
18:55 MiroslavAnashkin Then, please restart rabbitmq on each controller
18:55 Kupo24z Hey All, is there any estimations or goals to support regions in fuel in the future?
18:56 MiroslavAnashkin Then check if MySQL/Galera cluster is operable
18:57 albionandrew MiroslavAnashkin: by using “service rabbitmq-server restart”
18:57 MiroslavAnashkin Then restart neutron agents with`crm resource restart clone_p_neutron-openvswitch agent` or how it named
18:58 albionandrew OK thanks I’ll give it a go.
18:58 MiroslavAnashkin Yes, service rabbitmq-server restart works well
18:58 MiroslavAnashkin And last - restart all remaines openstack services on every node
19:00 MiroslavAnashkin Kupo24z: Hmm, can't even say if there are any estimations.
19:00 odyssey4me joined #fuel
19:02 albionandrew MiroslavAnashkin should crm status have no failed actions before restarting the openstack services?
19:03 albionandrew MiroslavAnashkin - How would you test galera?
19:03 MiroslavAnashkin Failed actions in crm status mean something failed in the past but now it may work
19:04 albionandrew so a cleanup for those?
19:05 MiroslavAnashkin For galera - please run `mysql -e "show status like 'wsrep%';"` on any controller and check cluster size and other statuses
19:06 Kupo24z MiroslavAnashkin: how hard is it to modify an install to support connecting to a Reginal keystone server?
19:06 albionandrew MiroslavAnashkin galera is good thanks
19:10 MiroslavAnashkin Kupo24z: Most probably it would be enough to change settings in /etc/keystone/ and restart keystone or run a couple of keystone commands
19:19 albionandrew MiroslavAnashkin I did as you suggested above. Gallera comes back clean. I did a crm cleanup on some services in there. crm status shows nothing failed now. On the cirros vm I see “ checking http://169.254.169.254/2009-04-04/instance-id ………….failed to read iid from metadata. tried 20”
19:21 albionandrew MiroslavAnashkin neutron agent-list shows 2 nodes with xxx in the DHCP agent column
19:26 MiroslavAnashkin You may also try `crm resource migrate ...` for dhcp agent
19:28 albionandrew MIgrate to ?
19:30 MiroslavAnashkin If not specified which node to migrate to, crm migrates resource to the next node in its list
19:37 albionandrew crm resource migrate  p_neutron-dhcp-agent has caused the neuton agent-list to :-)
19:38 MiroslavAnashkin Yes, sometimes it gets dirty cache or something like.
19:39 albionandrew MiroslavAnashkin - I don’t think I’m going to get a dhcp ip “failed 9/20: up 197.42. request failed”
19:39 xarses joined #fuel
19:40 MiroslavAnashkin Then you have to restart all non-working nova/openstack services. At least Scheduler and nova-compute, these to has no graceful reconnect procedure
19:40 MiroslavAnashkin s/these two
19:40 albionandrew will do.
19:43 Kupo24z MiroslavAnashkin: Do you know if Horizon is able to be shared between regions like keystone is? http://kimizhang.files.wordpress.com/2013/08/openstack-zoning.jpg
19:46 MiroslavAnashkin Kupo24z: Hmm, don't know.
19:47 MiroslavAnashkin Horizon runs inside apache - so, may be yes.
19:48 albionandrew MiroslavAnashkin I restarted the service openstack-nova-scheduler restart and service openstack-nova-compute restart on the controllers and computes but still no dhcp
19:50 albionandrew crm was clean but now shows - http://pastebin.com/uveyKuVx
19:55 e0ne joined #fuel
20:07 albionandrew MiroslavAnashkin ^ and now I get [root@node-28 init.d]# nova service-list ERROR: [Errno 113] No route to host
20:08 MiroslavAnashkin May be it is good idea to restart openstack-neutron or neutron-server on controllers...
20:09 albionandrew using “service …. restart “?
20:12 MiroslavAnashkin yes
20:15 albionandrew MiroslavAnashkin http://pastebin.com/g5GTVDwR
20:19 MiroslavAnashkin Great. Are your network switches OK?
20:20 albionandrew I think so we are trying to setup a lab which replicates what we have in production. I think the configs cam from there.
20:21 MiroslavAnashkin I mean cables and ports
20:21 albionandrew I think so.
20:24 albionandrew MiroslavAnashkin now all of the functional tests are failing, it was just the Check network connectivity from instance via floating IP
20:25 MiroslavAnashkin Please run `tail -f /var/log/nova/api.log` and check nova service-list one more time
20:26 MiroslavAnashkin And I don't trust built-in functional tests. Some of them require additional setup steps before running.
20:28 albionandrew nova service-list on 2 of the 3 controllers shows ERROR: [Errno 113] No route to host
20:28 albionandrew the tail shows accepted. Is there something I should be looking for.
20:28 albionandrew ?
20:28 albionandrew anything other than error?
20:29 MiroslavAnashkin please run `crm resource cleanup --resource p_neutron-dhcp-agent` on each controller
20:29 MiroslavAnashkin Then `crm resource cleanup --resource p_neutron-l3-agent`
20:30 MiroslavAnashkin then `crm resource cleanup clone_p_neutron-metadata-agent` on one of controllers
20:31 MiroslavAnashkin then `crm resource cleanup clone_p_neutron-openvswitch-agent` on one of controllers
20:31 MiroslavAnashkin and then `crm resource restart clone_p_neutron-openvswitch-agent`
20:32 MiroslavAnashkin and `crm resource restart clone_p_neutron-metadata-agent`
20:32 MiroslavAnashkin After that restart all openstack* services with service <service> restart
20:36 albionandrew MiroslavAnashkin All openstacj services that are currently running on all nodes?
20:38 MiroslavAnashkin yep
20:38 e0ne joined #fuel
20:53 dhblaz joined #fuel
21:06 albionandrew MiroslavAnashkin - still same issue, I did  “ls /etc/rc3.d/S* | egrep '(openstack)' | sort | xargs -I% -- sh % restart” on the nodes. They all the services restarted but unfortunately no change
21:14 thehybridtech joined #fuel
21:16 dhblaz joined #fuel
21:24 e0ne joined #fuel
23:26 justif joined #fuel
23:50 xarses joined #fuel

| Channels | #fuel index | Today | | Search | Google Search | Plain-Text | summary