Perl 6 - the future is here, just unevenly distributed

IRC log for #fuel-dev, 2015-10-05

| Channels | #fuel-dev index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
02:21 dimsum__ joined #fuel-dev
04:21 dimsum__ joined #fuel-dev
06:16 sylwesterB joined #fuel-dev
06:20 alex_bh joined #fuel-dev
06:26 alex_bh2 joined #fuel-dev
06:30 aarefiev joined #fuel-dev
06:30 alex_bh joined #fuel-dev
06:44 rlu joined #fuel-dev
06:55 HeOS joined #fuel-dev
07:24 hyperbaba joined #fuel-dev
07:42 tzn joined #fuel-dev
07:43 az__ joined #fuel-dev
07:46 tzn joined #fuel-dev
08:10 e0ne joined #fuel-dev
08:17 macjack joined #fuel-dev
08:23 s_berg joined #fuel-dev
08:29 mattymo joined #fuel-dev
08:30 pbrooko joined #fuel-dev
08:30 ekosareva joined #fuel-dev
08:30 HeOS joined #fuel-dev
08:39 amnk joined #fuel-dev
08:42 alex_didenko left #fuel-dev
09:02 skumar joined #fuel-dev
09:02 skumar hi
09:02 skumar anybody have pointers regarding fuel problem..
09:02 skumar Error running RPC method verify_networks: Network verification not avaliable because nodes ["3", "4", "6", "7"] not avaliable via mcollective, trace:
09:03 skumar verfiy_Networks fails..
09:05 skumar can anybody pls help ?
09:06 bpiotrowski skumar: can these nodes ping some external address?
09:07 skumar bpiotrowski: thannks for reply..
09:07 skumar bpiotrowski: they got their ip from fuel master node... eth0 interface..
09:08 bpiotrowski ah, wait, I misread the message
09:08 bpiotrowski looks like 3,4,6,7 nodes are offline
09:08 bpiotrowski can you ssh to them?
09:08 skumar bpiotrowski: so have ips like - 192.168.205.3, 192.168.205.4, 192.168.205.6, and 192.168.205.7
09:08 skumar bpiotrowski: yes i can ssh them..
09:09 skumar bpiotrowski: no all are online
09:09 skumar id | status   | name             | cluster | ip            | mac               | roles | pending_roles | online | group_id ---|----------|------------------|---------|---------------|-------------------|-------|---------------|--------|--------- 6  | discover | Untitled (30:c8) | 4       | 192.168.205.6 | ec:f4:bb:d9:30:c8 |       | controller    | True   | 4 4  | discover | Untitled (24:90) | 4       | 192.168.205.4 | ec:f4:bb:d9:2
09:10 bpiotrowski and 'mco ping' works fine?
09:10 skumar bpiotrowski: its gives master node only..
09:10 tzn joined #fuel-dev
09:10 skumar [root@fuel ~]# mco ping master                                   time=20.87 ms   ---- ping statistics ---- 1 replies max: 20.87 min: 20.87 avg: 20.87
09:11 bpiotrowski holser: ↑ can you take a look? not sure where to go further with this
09:11 skumar holser: can you pls guide.
09:12 bpiotrowski try 'dockerctl check', rabbitmq might be down
09:12 tzn joined #fuel-dev
09:13 skumar bpiotrowski: rabbitmq is up
09:13 skumar rabbitmq is ready.
09:14 bpiotrowski hum
09:14 skumar checking with command "curl -f -L -u "mcollective:stDY07TP" -s http://10.10.99.252:15672/api/exchanges | grep -qw 'mcollective_directed'" checking with command "! shell_container rabbitmq pgrep puppet" rabbitmq is ready.
09:15 bpiotrowski anything interesting in /var/log/docker-logs/mcollective.log?
09:16 skumar bpiotrowski: yes that seems to be okay..
09:16 skumar D, [2015-10-05T09:15:49.807634 #731] DEBUG -- : rabbitmq.rb:66:in `on_hbfire' Publishing heartbeat to stomp://mcollective@192.168.205.2:61613: send_fire, {:curt=>1444036549.8074887, :last_sleep=>30.49973177909851}
09:17 skumar I, [2015-10-05T09:10:39.882207 #1121]  INFO -- : rabbitmq.rb:10:in `on_connecting' TCP Connection attempt 0 to stomp://mcollective@192.168.205.2:61613 I, [2015-10-05T09:10:39.885185 #1121]  INFO -- : rabbitmq.rb:15:in `on_connected' Connected to stomp://mcollective@192.168.205.2:61613
09:17 bpiotrowski did you happen to redeploy master without rebooting slaves?
09:18 bpiotrowski also, check if mcollective is running on nodes
09:19 holser skumar: 'pgrep -f mcollectived'
09:19 holser on nodes
09:20 skumar holser: yes its running on all nodes..
09:20 holser check the logs
09:20 skumar [root@bootstrap ~]# ps -eaf | grep 1982 root       1982      1  0 Oct01 ?        00:00:11 /usr/bin/ruby /usr/sbin/mcollectived --pid=/var/run/mcollectived.pid --config=/etc/mcollective/server.cfg root      34373  29555  0 09:19 pts/0    00:00:00 grep 1982
09:20 holser if they all have connection to rabbitmq
09:20 luqas joined #fuel-dev
09:21 skumar holser: hey, they have failure in tcp connection
09:21 skumar I, [2015-10-05T09:20:55.590264 #1962]  INFO -- : rabbitmq.rb:10:in `on_connecting' TCP Connection attempt 11304 to stomp://mcollective@192.168.205.2:61613 I, [2015-10-05T09:20:55.591029 #1962]  INFO -- : rabbitmq.rb:25:in `on_connectfail' TCP Connection to stomp://mcollective@192.168.205.2:61613 failed on attempt 11304
09:22 holser here we go ...
09:22 skumar holser: how we can fix it ?
09:22 skumar holser: even all nodes have same failure..
09:22 holser check connection, restart mcollective … standard procedure
09:23 holser maybe your firewall filtered something … just do a standard procedure, check port, try to connect manually, restart service
09:24 skumar holser: if ping is working fine...
09:24 skumar holser: i kept tcpdump on fuel master node and found that tcp packets were coming from slave nodes.
09:26 skumar holser: could you pls tell me how to restart the mcollective
09:28 skumar holser: i have restarted the service
09:28 skumar still the same problem..
09:28 skumar I, [2015-10-05T09:27:40.236432 #6974]  INFO -- : mcollectived:35 The Marionette Collective 2.3.3 started logging at debug level D, [2015-10-05T09:27:40.236479 #6974] DEBUG -- : mcollectived:38 Starting in the background (true) D, [2015-10-05T09:27:40.239640 #6977] DEBUG -- : pluginmanager.rb:83:in `[]' Returning cached plugin global_stats with class MCollective::RunnerStats D, [2015-10-05T09:27:40.239867 #6977] DEBUG -- : pluginmana
09:28 holser test connection first … telnet IP PORT
09:29 skumar holser: yes, i can do ssh..
09:30 holser Did I ask SSH?
09:31 skumar holser: sorry, but telnet is not available on slave nodes..
09:31 skumar holser: so i did ssh..
09:31 bpiotrowski try telnet from master node to slave node
09:33 skumar holser: no its not able to do..
09:39 skumar skumar: one more thing mcollectived is not running on Fuel Master node
09:40 skumar holser:  bpiotrowski : one more thing mcollectived is not running on Fuel Master node
09:54 degorenko joined #fuel-dev
09:58 IBerezovskiy joined #fuel-dev
09:58 iberezovskiy joined #fuel-dev
10:02 tzn joined #fuel-dev
10:14 e0ne joined #fuel-dev
10:15 romcheg joined #fuel-dev
10:17 tzn joined #fuel-dev
10:24 dklenov joined #fuel-dev
10:27 tzn joined #fuel-dev
10:39 tzn joined #fuel-dev
10:55 tzn joined #fuel-dev
10:55 igajsin joined #fuel-dev
10:56 vstruk_ joined #fuel-dev
11:01 tzn joined #fuel-dev
11:14 vsakharov joined #fuel-dev
11:16 asaprykin joined #fuel-dev
11:23 yottatsa joined #fuel-dev
11:28 tzn joined #fuel-dev
11:28 sylwesterB joined #fuel-dev
11:34 amnk joined #fuel-dev
11:44 yottatsa left #fuel-dev
12:13 e0ne joined #fuel-dev
12:27 skumar holser: hi
12:28 skumar can anybody pls help in fuel deployment ..
12:28 skumar in `on_connectfail' TCP Connection to stomp://mcollective@192.168.205.2:61613 failed on attempt 320
12:29 skumar verfiy network is failing bcz of this..
12:35 romcheg joined #fuel-dev
12:35 jeh joined #fuel-dev
13:08 agrag joined #fuel-dev
13:08 agrag skumar: hi
13:09 jaypipes joined #fuel-dev
13:11 s_berg joined #fuel-dev
13:27 7GHABCFAO joined #fuel-dev
13:27 6A4AANKQ6 joined #fuel-dev
13:30 amnk joined #fuel-dev
13:40 6A4AANKQ6 left #fuel-dev
13:46 mattymo joined #fuel-dev
13:47 mattymo joined #fuel-dev
13:49 mattymo__ joined #fuel-dev
13:49 mattymo1 joined #fuel-dev
13:51 mattymo joined #fuel-dev
13:55 mattymo__ joined #fuel-dev
13:55 mattymo1 joined #fuel-dev
13:55 kaliya joined #fuel-dev
13:56 openstackstatus joined #fuel-dev
13:57 mattymo joined #fuel-dev
14:06 amnk joined #fuel-dev
14:27 tzn joined #fuel-dev
14:31 Ivan_ joined #fuel-dev
14:35 Ivan_ skumar can you share 'mco ping' output?
14:44 salmon_ joined #fuel-dev
14:48 sylwesterB joined #fuel-dev
15:47 dims joined #fuel-dev
15:47 Guest96374 joined #fuel-dev
15:48 angdraug joined #fuel-dev
15:48 dims_ joined #fuel-dev
15:54 ClaudeD joined #fuel-dev
16:14 ClaudeD Hi. I'm running Fuel 6.1 with Ceph. Since last week, I cant launch new instances (boot from image-creates new volumes). I get "Failure prepping block device" in Horizon. A "nova list" shows the instance with ERROR  | block_device_mapping | NOSTATE. The volume is created and attached to the instance however.
16:26 ClaudeD Seems like I hit this bug: https://bugs.launchpad.net/mos/+bug/1463802. Any advice on how to "restart the affected service"? What are affected services?
16:37 amnk joined #fuel-dev
16:41 rmoe joined #fuel-dev
16:42 salmon_ joined #fuel-dev
17:09 subscope joined #fuel-dev
17:17 e0ne joined #fuel-dev
17:38 jeh joined #fuel-dev
17:44 xarses joined #fuel-dev
17:57 HeOS joined #fuel-dev
18:14 yottatsa joined #fuel-dev
18:49 HeOS joined #fuel-dev
18:58 dims_ joined #fuel-dev
18:59 dims_ joined #fuel-dev
19:12 rgowrishankar joined #fuel-dev
19:32 sbog joined #fuel-dev
19:39 sbog joined #fuel-dev
19:39 sbog left #fuel-dev
19:41 sbog joined #fuel-dev
19:46 jeh joined #fuel-dev
19:47 sbog joined #fuel-dev
19:53 dims_ joined #fuel-dev
20:15 e0ne joined #fuel-dev
20:16 dims_ joined #fuel-dev
20:16 dims_ joined #fuel-dev
20:23 az__ joined #fuel-dev
20:55 angdraug joined #fuel-dev
21:10 zigo joined #fuel-dev
21:28 jeh joined #fuel-dev
21:52 xarses weekly meeting action reminder
21:53 xarses mihgen will task a core from each repo to start the first version of the MAINTAINERS file
21:53 xarses angdraug will task fuel-infra team to update core groups
21:54 e0ne joined #fuel-dev
22:11 angdraug joined #fuel-dev
22:31 jeh joined #fuel-dev
22:48 jobewan joined #fuel-dev
22:49 dims__ joined #fuel-dev
23:10 rmoe joined #fuel-dev
23:16 jeh joined #fuel-dev
23:17 dims_ joined #fuel-dev
23:34 youellet__ joined #fuel-dev
23:53 angdraug joined #fuel-dev

| Channels | #fuel-dev index | Today | | Search | Google Search | Plain-Text | summary