Perl 6 - the future is here, just unevenly distributed

IRC log for #fuel, 2016-05-04

| Channels | #fuel index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:02 linux00002 joined #fuel
00:03 linux00002 left #fuel
00:36 Julien-zte joined #fuel
00:51 Julien-zte joined #fuel
00:59 xarses joined #fuel
01:23 pmathews joined #fuel
01:48 ilbot3 joined #fuel
01:48 Topic for #fuel is now Fuel 8.0 (Liberty) https://www.fuel-infra.org/ | Paste here http://paste.openstack.org/ | IRC logs http://irclog.perlgeek.de/fuel/
01:50 raunak joined #fuel
02:55 akscram joined #fuel
03:01 rongze joined #fuel
03:14 rongze joined #fuel
03:54 fedexo joined #fuel
03:59 e0ne joined #fuel
04:02 e0ne joined #fuel
04:15 rongze joined #fuel
04:30 reddydodda joined #fuel
04:31 fedexo joined #fuel
04:46 Julien-zte joined #fuel
04:49 raunak joined #fuel
05:21 kszukielojc joined #fuel
05:28 akscram joined #fuel
05:35 natarej_ joined #fuel
05:40 pbrooko joined #fuel
05:59 rmoe joined #fuel
06:01 mnaser joined #fuel
06:03 Hulda joined #fuel
06:09 cartik joined #fuel
06:15 javeriak joined #fuel
06:21 javeriak joined #fuel
06:24 Liudvikas joined #fuel
06:37 HeOS joined #fuel
06:41 e0ne joined #fuel
06:42 pbelamge joined #fuel
06:46 fatdragon joined #fuel
06:51 rongze joined #fuel
07:04 magicboiz joined #fuel
07:06 iwi joined #fuel
07:08 Hulda joined #fuel
07:30 magicboiz joined #fuel
07:59 neilus joined #fuel
08:04 DevStok_ joined #fuel
08:04 DevStok_ http://paste.openstack.org/show/496033/
08:05 DevStok_ I get bad gateway
08:05 DevStok_ on fuel 7
08:05 DevStok_ I think keystone docker has problem
08:10 rongze joined #fuel
08:12 pbrooko joined #fuel
08:21 javeriak joined #fuel
08:37 yassine joined #fuel
08:42 iwi joined #fuel
08:42 DevStok_ joined #fuel
08:46 xarses joined #fuel
08:48 fatdragon joined #fuel
09:10 natarej__ joined #fuel
09:15 cartik joined #fuel
09:34 zefciu joined #fuel
09:45 javeriak joined #fuel
09:49 fatdragon joined #fuel
10:06 e0ne joined #fuel
10:33 artem_panchenko left #fuel
10:34 artem_panchenko joined #fuel
10:38 iwi left #fuel
10:50 fatdragon joined #fuel
10:52 hyperbaba joined #fuel
10:54 yottatsa joined #fuel
11:24 pbrooko joined #fuel
11:28 rtikunov joined #fuel
11:28 yottatsa_ joined #fuel
11:33 javeriak joined #fuel
11:35 rtikunov joined #fuel
11:56 Alcest joined #fuel
12:12 yassine joined #fuel
12:22 e0ne joined #fuel
12:33 kodokuu joined #fuel
12:33 kodokuu Hey, anyone works on vmware_dvs driver ?
12:51 fatdragon joined #fuel
12:59 reddydodda joined #fuel
13:02 javeriak joined #fuel
13:04 reddydodda Hi ... How i can install zabbix plugin after deployment. ?? i am using fuel8.0
13:06 reddydodda i was able to install zabbix plugin 2.5.0 on fuel node and it is reflected on fuel dashboard ( others section ) but i cant enable the feature.
13:30 krobzaur joined #fuel
13:34 reddydodda joined #fuel
14:00 javeriak joined #fuel
14:07 raunak joined #fuel
14:27 fatdragon joined #fuel
14:33 pmathews1 joined #fuel
14:36 v1k0d3n joined #fuel
14:39 rongze joined #fuel
14:55 Julien-zte joined #fuel
15:06 Miouge joined #fuel
15:07 raunak joined #fuel
15:15 neilus1 joined #fuel
15:20 ddmitriev joined #fuel
15:24 xarses joined #fuel
15:30 pmathews joined #fuel
15:37 pmathews1 joined #fuel
15:46 neilus joined #fuel
15:47 Miouge joined #fuel
15:49 e0ne joined #fuel
15:53 pmathews joined #fuel
16:03 brig-man joined #fuel
16:26 javeriak joined #fuel
16:31 pmathews joined #fuel
16:32 raunak joined #fuel
16:37 pmathews1 joined #fuel
16:43 kur1j_ joined #fuel
16:44 raunak joined #fuel
16:45 kur1j_ where can I get the release of fuel 9.0?
16:47 e0ne joined #fuel
16:48 rongze joined #fuel
16:50 e0ne left #fuel
16:51 mwhahaha kur1j_: https://www.fuel-infra.org/
16:53 kur1j_ ah hell, thanks I was going to the development site and it wasn't up yet
16:54 javeriak_ joined #fuel
16:55 gomarivera joined #fuel
17:00 shewless joined #fuel
17:01 shewless Hi. I'm using Fuel 9.0 to deploy an openstack and one of my nodes is failing to be provisioned.  The status is simply "Error". I can logon to the node and it has ubuntu on it, but I'm unsure which logs I need to check to see why it's failing. Any hints?
17:04 shewless I've search /var/log/ for "error" on both the fuel node and the failing node and nothing is standing out to me
17:06 raunak_ joined #fuel
17:06 mnaser left #fuel
17:08 xarses shewless: lets start with the astute log on the fuel master node, you can reach it from the Logs tab on the UI
17:09 xarses shewless: we want to switch to debug mode, find the error referring to the id of the node that failed (node-1 -> id:1)
17:10 xarses shewless: if the error post isn't very verbose look at the updates in the debug and see if the message was more descriptive there.
17:10 shewless xarses: Excellent I do see some more meaningful stuff in there
17:13 xarses shewless: its probably going to point you to the provision task failing, which is probably going to be written on the node's fuel-agent log, you can also see this in the fuel ui (but the node name is the UI display name not the node id so you might need for find the last 4 of the nodes MAC both ID and node "name" are shown in `fuel nodes` cli)
17:13 shewless I see two lines that are suspect
17:14 shewless [398] No nodes were sent to reboot for task: reboot_provisioned_nodes
17:14 shewless [398] Nodes failed to reboot: ["6"]
17:14 xarses yep, so lets look at node-6's fuel-agent log
17:15 shewless Okay. The UI doesn't show any logs for node-6. So I guess I'll go ssh into the box.
17:16 xarses you have to select "other servers" before it lets you select nodes
17:16 shewless Yes I did
17:16 xarses kk
17:17 shewless j
17:17 shewless var/log somewhere?
17:17 shewless I don't see fuel.log
17:19 xarses what's the hostname?
17:19 shewless node-6
17:20 xarses hmm, I'm not sure it's persisted. The log was on the bootstrap tmpfs prior to it rebooting the os it wrote
17:22 shewless hmm. I see a few different tmpfs mounted. /run, /run/lock, /run/shm, /run/user
17:22 xarses no, that's all gone, the node rebooted
17:23 shewless Do you think it's a timing issue? like my node is taking just a little too long to reboot or something
17:24 xarses the way it works is PXE -> discovery image in ram -> fuel-agent writes image to disk from fuel master -> node reboots gracefully -> node reports its online
17:24 xarses so we still have some things to check
17:25 xarses in the UI is the node currently 'online' ?
17:25 shewless In the UI it says "Error"
17:25 xarses it should also show online | offline like the other nodes
17:26 xarses from the fuel node cli run `mco ping` is '6' in the response list?
17:27 xarses you can also see the online|offline status `fuel node` output (its true|false here)
17:27 shewless In the node tab they others say "Ubuntu is installed"
17:28 shewless Okay I'll go onto the fuel node
17:28 shewless yes 6 is in the list (I've sshed to node-6 in the past as well)
17:29 shewless ah.. fuel node says "online = True"
17:29 shewless for all nodes.. including 6
17:30 xarses mco (mcollective) uses rabbitmq to send commands to the nodes, so it's not ssh. It also checks if rabbit looks sane
17:32 shewless oh i see.
17:32 xarses ok, so finally, we can look at /var/log/remote/node-6*/ and the fuel-agent log should be in there, otherwise i guess it was a fluke
17:32 shewless 6 is in the response list
17:34 shewless no fuel-agent log there. CLOUDINIT.log, CRON.log, kernel.log, login.log, nailgun-agent.log, ntpdate.log, ntpd.log, rsyslogd.log, sshd.log, systemd-logind.log
17:34 xarses weird
17:34 xarses explains why the UI didn't have it either
17:34 shewless so in my other nodes there is a "bootstrap" directory which contains fuel-agent.log
17:34 shewless but for 6 there is just no bootstrap directory at all
17:35 xarses yep
17:35 shewless I can try to provision again.  It's failed a couple times before
17:35 xarses you can just delete and re-do that node specifically if you want
17:39 shewless with the cli right?
17:40 shewless like "fuel --env 1 node --provision --node 6
17:41 xarses your using 9 right? you can do it in the UI on 9 too
17:41 xarses but yes, that is the CLI command
17:42 shewless yes using 9. I did try it from the UI the "provision only" option. That's cool
17:46 javeriak joined #fuel
17:46 shewless FYI my node is "ceph-osd,base-os"
17:46 shewless I don't need to make it a controller do I?
17:49 xarses no, and you don't need base-os unless you need it for a plugin, or you just want fuel to install the OS for you and you will do something later with it
17:49 rongze joined #fuel
17:49 xarses if you pick any role, the node will get the environments OS installed on it
17:49 gardlt joined #fuel
17:50 shewless good to know.
17:51 shewless Okay so it failed again. I checked the node when it failed and it was in the process of booting up. So my guess is that it's a timeout problem
17:51 xarses how long does it take to boot?
17:52 shewless maybe like 3-4 minutes. it's got a RAID controller and a bunch of disks so it takes a bit
17:52 xarses ya, but thats not nearly long enough for it get that way
17:52 xarses I think the boot timeout is much higher than that
17:53 shewless hmm. I can see it going into error state pretty quickly
17:53 shewless after the reboot is the device essentially provisioned anyways?
17:53 xarses there is a timeout on the other side, in that If we didn't think the fuel-agent closed properly, we issue a manual reboot to the node via SSH
17:54 xarses its 100% provisioned after fuel-agent issues the reboot
17:54 shewless ah.. and since there is no fuel-agent log it's likely not provisioned correct?
17:54 shewless just wondering if I can force it "online" and more to deployment
17:55 xarses well the lack of the log indicates that it's failing to get that status correctly
17:55 xarses since, it gets the log nearly the same way
17:56 xarses which leads it to try the fallback reboot code
17:56 xarses it may indicate a udp or other network issue with the node, but seems off that it got the image from the fuel node, and wrote it to disk
17:57 shewless I see. Anything else I can try? I do notice that the "imaging" part seems to go very fast. Does it cache the OS somewhere? Can I force it to do a clean install or is that already happening?
17:57 xarses so dono
17:58 shewless Is there anyway to disable the thing that forces the reboot so we can look at the system before reboot?
17:59 xarses hrm, we probably need to hack the ruby code that does the second reboot and probably hack fuel-agent its self so it doesn't try to either
18:00 shewless if you give me a hint as to where those files live I can try
18:00 shewless are they on the fuel node?
18:00 xarses alternatly, we can try to reboot the node back to discovery and manually start the provision task with the cmd that it ran automaticly
18:01 shewless sure
18:01 xarses which would probably be easier, there should be the full command in the astute log
18:01 shewless delete the node?
18:01 xarses ya
18:01 xarses we need to either way
18:02 shewless okay I'll do that
18:03 shewless so now the node is "discovered"
18:04 shewless do I need to remove it further?
18:04 shewless how do I manually start the provision task?
18:05 shewless crap. I had to reset the environment to remove the node and now there aren't any astute logs
18:06 xarses uhh, the astute.log should allways be there
18:15 shewless oops. yeah it's there
18:17 javeriak joined #fuel
18:17 shewless I have to step out for an hour or so. Am I just looking to run the provision commands that were logged in astute for node 6 one by one?
18:18 xarses basically
18:42 gomarivera does anyone know how can a fuel task be written such that it applies to the fuel node itself? or is this not possible - I tried with a deployment_task that had the role: master - but this was not successfull - any suggestions?
18:51 rongze joined #fuel
18:58 pmathews joined #fuel
19:07 javeriak_ joined #fuel
19:20 HeOS joined #fuel
19:52 rongze joined #fuel
20:16 shewless joined #fuel
20:17 shewless xarses: I found that "reboot_reprovisioned_nodes" gives a timeout of 240. Which I think is the problem
20:17 shewless 4 minutes is not enough time for the reboot on my system.
20:18 shewless any idea how i A) increase the timeout or B) somehow skip the "reprovisioned" part
20:21 raunak joined #fuel
20:24 pmathews1 joined #fuel
20:28 shewless does anyone have an idea of how I would force my fuel instance to accept the change I made to config.rb?
20:36 xarses what's the path?
20:46 mwhahaha he probably means the astute config
20:46 mwhahaha shewless: restart astute container
20:46 mwhahaha or astute service
20:46 mwhahaha depending on your version of fuel
20:53 rongze joined #fuel
20:59 xarses 9, so service
21:54 rongze joined #fuel
22:50 raunak joined #fuel
22:53 pmathews joined #fuel
23:21 rongze joined #fuel
23:40 natarej__ joined #fuel
23:42 raunak joined #fuel

| Channels | #fuel index | Today | | Search | Google Search | Plain-Text | summary