Perl 6 - the future is here, just unevenly distributed

IRC log for #fuel, 2016-08-11

| Channels | #fuel index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:06 Damjanek joined #fuel
00:08 Julien-zte joined #fuel
00:09 iberezovskiy joined #fuel
00:10 code-R joined #fuel
00:12 code-R_ joined #fuel
00:33 raunak joined #fuel
00:55 eric_lopez joined #fuel
00:57 code-R joined #fuel
01:21 elo joined #fuel
01:22 xarses joined #fuel
01:27 elo joined #fuel
01:34 elo joined #fuel
01:41 eric_lopez joined #fuel
01:48 ilbot3 joined #fuel
01:48 Topic for #fuel is now Fuel 8.0 (Liberty) https://www.fuel-infra.org/ | Paste here http://paste.openstack.org/ | IRC logs http://irclog.perlgeek.de/fuel/
02:05 elo joined #fuel
02:31 eric_lopez joined #fuel
02:36 elo joined #fuel
02:47 code-R joined #fuel
03:01 raunak joined #fuel
03:12 krotscheck_ joined #fuel
03:15 elo joined #fuel
03:25 elo joined #fuel
03:46 Bomfunk joined #fuel
03:50 aglarendil joined #fuel
03:51 asilenkov joined #fuel
03:53 code-R joined #fuel
03:54 raunak joined #fuel
04:14 elo joined #fuel
05:01 aglarendil_ joined #fuel
05:03 Julien-zte joined #fuel
05:41 fxpester|2 joined #fuel
05:42 omolchanov__ joined #fuel
05:43 agordeev_ joined #fuel
05:43 justif_ joined #fuel
05:43 ikalnitsky_ joined #fuel
05:43 serverascode_ joined #fuel
05:44 evgenyl_ joined #fuel
05:48 yantarou_ joined #fuel
05:48 bapalm_ joined #fuel
05:49 degorenko joined #fuel
06:14 Miouge joined #fuel
06:17 vkulanov joined #fuel
06:20 code-R joined #fuel
06:52 Damjanek joined #fuel
06:55 vkulanov joined #fuel
07:00 f13o joined #fuel
07:23 vkulanov joined #fuel
07:30 Kundan joined #fuel
07:34 DavidRama joined #fuel
07:38 Kundan Hello Experts
07:38 Kundan My Openstack Deployment has failed with the error - Deployment has failed. Method granular_deploy. Failed to execute hook 'copy_files' File does not exist or is not readable /var/lib/fuel/keys/1/haproxy/public_haproxy.crt
07:38 Kundan I checked this directory is not there is Fuel master node
07:39 Kundan I am using Fuel 7.0
07:40 Kundan http://paste.openstack.org/show/554022/ - Here is the complete Astute log.
07:59 Kundan Is there anyone available to help with this please ?
08:18 pbelamge joined #fuel
08:23 noshankus joined #fuel
08:25 dguryanov2 joined #fuel
08:29 Julien-zte joined #fuel
08:33 yassine joined #fuel
08:36 f13o joined #fuel
08:37 saibarspei joined #fuel
08:38 saibarspei Hello guys. There is an issue with Fuel7 when deploying Kilo, on specific hardware it seems that the centos bootstrap image enumerates interfaces differently than the ubuntu deployment image causing the deployment to fail
08:38 saibarspei Is there a way to specify the cards one by one to fuel ?
08:58 vkulanov romcheg, akasatkin, ikalnitsky, please review https://review.openstack.org/#/c/346454/ (python-fuelclient)
08:59 * romcheg is looking
09:04 BernhardW joined #fuel
09:04 romcheg vkulanov: could you please approve here, if it's ok for you https://review.openstack.org/#/c/341118/ ?
09:04 romcheg vkulanov: it was already approved but I had to rebase it
09:37 ekosareva_ joined #fuel
09:48 DavidRama left #fuel
09:59 ekosareva_ folks, please review patch with very small changes in openstack.yaml https://review.openstack.org/#/c/351255/
10:01 vkulanov joined #fuel
11:06 nokes joined #fuel
11:14 noshankus joined #fuel
11:19 f13o joined #fuel
11:20 thiagolib joined #fuel
11:20 agordeev dear fuelers, i'm delighted to tell that master fuel-agent's Fuel CI gets broken once again. so, please don't ask it for rechecks, it won't be green anyway.
11:52 noshankus joined #fuel
12:03 venkat joined #fuel
12:42 flerfb0rt joined #fuel
12:44 vkulanov joined #fuel
13:01 pbelamge joined #fuel
13:08 Sketch what's the recommended way to add persistent iptables rules (for monitoring agent purposes) to nodes provisioned by fuel?  using ubuntu's iptables-persistent does not seem to work well...
13:11 azemlyanov joined #fuel
13:14 Julien-zte joined #fuel
13:16 f13o joined #fuel
13:23 z- joined #fuel
13:33 johnavp1989 joined #fuel
13:45 noshankus joined #fuel
13:55 code-R joined #fuel
13:57 code-R_ joined #fuel
14:24 nokes joined #fuel
14:31 Miouge joined #fuel
14:35 romcheg Fuelclient cores
14:35 romcheg We need to backport a lot
14:35 romcheg https://review.openstack.org/#/q/status:open+project:openstack/python-fuelclient+branch:stable/mitaka+topic:deprecate_old_cli
14:35 Vijayendra joined #fuel
14:35 romcheg These patches allow to remove old fuel cli
14:36 romcheg They were merged to master and now we need them in in stable/mitaka
14:38 xarses joined #fuel
14:41 johnavp1989 Hello can anyone help me with a repo issue. apt is trying to pull from and ubuntu repo located at http://172.16.1.8 and failing
14:43 johnavp1989 First I'm not really sure what this IP is... some of the other repos are pointed to 172.16.1.13 which is the fuel node and those and those are working but the ubuntu repos don't work if I point them to that ip
14:47 saibarspeis joined #fuel
14:47 sneha joined #fuel
14:55 jhova joined #fuel
14:57 jhova joined #fuel
15:03 code-R joined #fuel
15:08 Topic for #fuel is now Fuel 9.0 (Mitaka) https://www.fuel-infra.org/ | Paste here http://paste.openstack.org/ | IRC logs http://irclog.perlgeek.de/fuel/
15:11 xarses the repos should be coming from the list of repos on the settings page
15:34 mwhahaha holser_, aglarendil, mattymo, alex_didenko, mmalchuk: please review/merge https://review.openstack.org/#/c/349721/ as it's been sitting over a week and is a swarm-blocker
15:47 code-R joined #fuel
15:57 aglarendil mwhahaha: I have already put my +2 into it :-)
15:58 mwhahaha yea i know, but you can also pummel the appropriate office people as necessary :D
15:58 aglarendil holser bogdando an alex_didenko are on PTO :-)
15:58 aglarendil I merged it
15:59 mwhahaha ha so we basically all the fuel cores were on PTO last week
16:03 yottatsa joined #fuel
16:04 raunak joined #fuel
16:28 Reepicheep_ joined #fuel
16:43 apopovych joined #fuel
16:59 ajafo joined #fuel
17:13 sfp joined #fuel
17:15 sfp I have fuel slaves that need a custom bootstrap for loading unsupported sfp mnetwork modules. So I built a custom bootstrap image. and it works. the nodes load to discover mode with all hardware operational. then I deploy. during deploy the nodes reboot (after ubuntu install) and then load without the changes I made, so the additional kernel module parameters are gone, the hardware is missing and deployment fails.
17:15 sfp is there a way to change the ubuntu installation (similar to the way you change the bootstrap image)
17:23 aglarendil sfp: which Fuel version are you using ?
17:23 sfp v 9.0
17:23 aglarendil you can adjust a list of packages that are installed during provisioning phase in environment settings
17:24 aglarendil sfp: ^^
17:25 sfp thanks aglarendil , but it's not a package. its parameters for a kernel module. it works great for the bootstrap, but once you deploy and ubuntu is installed the change isn't carried over to the installation.
17:25 sfp it's extra-dir added for bootstrap build
17:27 aglarendil well, you can set kernel module parameters in the kernel cmdline AFAIK and Fuel allows you to configure them as well
17:36 kur1j joined #fuel
17:37 kur1j So I deleted a few nodes from an environment (for some reason they are failing to deploy).
17:37 kur1j I deleted them and now they don't show up in my "equipment list
17:38 kur1j https://docs.mirantis.com/openstack/express/latest/operations.html#delete-nodes-from-the-environment this document states it might take 1-2 hours
17:38 kur1j is that still true?
17:42 aglarendil kur1j: which version of Fuel are you using?
17:42 sfp kur1j: I think you need to reboot them.
17:43 sfp aglarendil: can you point me to an example? I couldn't find any, or documentation. weird that it doesn't carry over to the installation
17:44 kur1j aglarendil: 9
17:44 kur1j sfp: I rebooted one of them and it still doesn't show up.
17:44 aglarendil sfp: there is 'Kernel parameters' field on the settings tab
17:44 kur1j I ran a fuel node and they aren't there either
17:45 aglarendil kur1j: so, node deletion should take faster, but not longer than controller deployment time
17:45 aglarendil as it may require partial controller redeployment depending on which node you delete
17:45 kur1j aglarendil: it deleted the node, it just doesn't show back up
17:45 kur1j its just a compute node
17:45 aglarendil okay, that would take several minutes only, I think no more than 10 for not so slow environments
17:45 kur1j i went and added like 10 compute nodes about 6
17:46 kur1j deleted the "failed" deployed nodes and they never came back to the unallocated pool
17:46 aglarendil did they actually reboot into bootstrap image ?
17:46 aglarendil can you do 'mco ping'?
17:46 kur1j no
17:46 kur1j they didn't they are like they were deployed
17:46 aglarendil can you see their console through IPMI or vnc ?
17:47 kur1j yes I can
17:47 aglarendil I am not sure I did get that. so what is the current state of the nodes that you deleted?
17:47 aglarendil you mean they were not deleted at all?
17:47 aglarendil which particular actions did you do?
17:47 aglarendil was it through UI or through CLI or API?
17:47 kur1j so they don't show up in the Fuel UI, at all. They are sitting at a prompt that says "node-25".
17:48 aglarendil can we restore the full list of actions you did?
17:48 aglarendil like 'clicked button A', 'ran command B' ?
17:49 kur1j Added 10 nodes to my environment, network tested successfully, changed the interfaces to all be the same config as the currently running compute nodes, (these are new compute nodes). I clicked deploy. I watched the fuel progress bar of each node. 4 of them
17:49 kur1j 4 of them got to 100%, while 6 of them went to a failed stated
17:49 kur1j then when fuel finalized entire deployment failed
17:50 kur1j well it was marked as failed
17:50 aglarendil okay. is that all?
17:50 kur1j Deployment has failed. All nodes are finished. Failed tasks: Task[tools/25], Task[tools/27], Task[tools/28], Task[tools/31], Task[tools/30] Stopping the deployment process!
17:50 kur1j at that point I deleted the ones that "failed" or didn't finish from the environment
17:50 aglarendil okay
17:50 kur1j redeployed, and it successfully completed
17:51 aglarendil so deletion was the latest action you applied, right?
17:52 kur1j well delete, then redeploy of the nodes that didn't fail
17:52 kur1j I would expect them to be in unallocated (the ones I deleted) but they aren't
17:52 aglarendil so you marked nodes for the deletion and clicked 'deploy changes' button?
17:52 kur1j yup
17:53 kur1j and then when that finished the ones I deleted were gone, and the rest of the system was "READY"
17:53 aglarendil okay. so, first of all, I would check if that nodes actually are accessible from the master node by their PXE addresses
17:53 kur1j i tried SSHing into this node and it isn't
17:53 kur1j fuel#: ssh node-25
17:53 kur1j and it doesn't work
17:53 kur1j but it looks to be up
17:54 kur1j (on the console that is)
17:54 aglarendil so, here how it works
17:54 aglarendil the node reboots
17:54 aglarendil nailgun-agent starts and puts an HTTP request into fuel master node
17:54 aglarendil fuel master node figures out whether this node existed previously or not
17:54 aglarendil and responds with its id
17:55 aglarendil if admin network is inaccessible for them, this cannot happen
17:55 aglarendil and I guess that's why 'tools' tasks failed as they install packages from the master node
17:55 aglarendil so, I think you lost the connectivity in admin network for these 6 nodes
17:56 kur1j which is silly, because these are blades...they are all exactly the same
17:56 kur1j running through the network, and network configuration
17:56 aglarendil it might be, but if you cannot ping them ... well, I suggest you try to reset them
17:57 aglarendil they could have fallen into some sort of kernel panic, perhaps
17:57 kur1j so can I format them and start all the way over?
17:57 aglarendil well, yeah, they should get their old IDs, unless you replace their network cards
17:57 kur1j aglarendil: I physical restarted the node and  it booted up fine
17:58 kur1j ok, ill try that
17:58 aglarendil is it in 'bootstrap' or in real Ubuntu ?
17:58 kur1j what really gets me is why it failed to deploy in the first place
17:58 kur1j real Ubuntu
17:58 kur1j shows (node-25) on the console
17:58 aglarendil then, you really have connectivity issue
17:58 aglarendil it should have booted through PXE
17:58 kur1j hmm, okay ill try it again
17:58 aglarendil but due to PXE unavailability it booted into the base system
17:59 kur1j ill format, reinstall and see whats going on
17:59 kur1j whats odd is in this blade there are 16 nodes total...
17:59 kur1j 1-8, 10, 13, 16 deployed with no problems, the others failed
18:00 kur1j so 9,11,12,14,15
18:00 aglarendil you might want to check puppet logs for these nodes
18:00 aglarendil why tools task failed
18:00 aglarendil in /var/log/remote/<node-id>/puppet-apply.log
18:13 kur1j I formatted that machine and restarted it and its PXE booting right now into the bootstrap image
18:20 kur1j so when a deployment happens all of the nodes "deploy" (they show a percentage of their deployment progress). Does this make them inaccessible?
18:20 kur1j all of them
18:30 goldenfri joined #fuel
18:41 vkulanov joined #fuel
18:47 aglarendil it is ok to have nodes in 'deployment' stage
18:48 aglarendil so, you say that when you click 'deploy' button, your nodes become inaccessible. is that right?
18:48 aglarendil do they become accessible in a while or not?
18:50 kur1j no, I'm just asking if I am doing work on an environment, say adding 5 more nodes. Should all of them look like they are deploying?
18:51 mwhahaha yes
18:51 aglarendil in 9.0 they should and will. but which particular tasks are executed depends on the type of change you do. for some of the nodes they will complete deployment really quickly, some of them will deploy longer
18:54 aglarendil but! they should not become inaccessible
19:02 kur1j ok, thats good to know. What about adding another controller node?
19:03 kur1j same thing (I was specifically talking about compute nodes)
19:08 mwhahaha same deal, when you add more nodes we have to execute some tasks on the other nodes to ensure the host information/configs are updated.
19:37 f13o joined #fuel
20:21 krobzaur joined #fuel
21:48 Reepicheep_ joined #fuel
21:49 Reepicheep_ joined #fuel
22:08 Reepicheep joined #fuel
22:26 xarses joined #fuel
22:50 DavidRama joined #fuel
23:48 skath_ joined #fuel

| Channels | #fuel index | Today | | Search | Google Search | Plain-Text | summary