Perl 6 - the future is here, just unevenly distributed

IRC log for #fuel, 2015-04-01

| Channels | #fuel index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:02 tzn joined #fuel
01:01 tobiash joined #fuel
01:30 tzn joined #fuel
01:46 LiJiansheng joined #fuel
02:25 xarses joined #fuel
03:03 Longgeek_ joined #fuel
03:09 mattgriffin joined #fuel
03:13 gongysh_ joined #fuel
04:07 emagana joined #fuel
05:05 emagana joined #fuel
05:15 monester_laptop joined #fuel
05:39 Longgeek joined #fuel
05:40 monester_laptop joined #fuel
05:53 thumpba joined #fuel
06:31 teran joined #fuel
06:45 willemgf joined #fuel
06:45 willemgf Hi, I have a question regarding the Fuel Network config. I was wondering what the "Internal network" stands for. Is this for the Tenant-part? In the documentation I could not found anything information about it.
06:53 dklepikov joined #fuel
07:09 hyperbaba joined #fuel
07:32 monester_laptop joined #fuel
07:33 stamak joined #fuel
07:58 dkusidlo joined #fuel
08:13 Longgeek joined #fuel
08:15 gongysh_ joined #fuel
08:16 e0ne joined #fuel
08:25 samuelBartel joined #fuel
08:28 adanin joined #fuel
08:40 alecv_ joined #fuel
09:05 teran joined #fuel
09:27 Longgeek joined #fuel
09:41 Miouge joined #fuel
09:53 evg willemgf: Hi, in "Network config" it's a default private network greated for default admin tenant. There is some mess in docs about it.
09:59 gongysh_ joined #fuel
10:19 Miouge joined #fuel
10:36 aarefiev joined #fuel
10:49 adanin joined #fuel
11:04 dkusidlo joined #fuel
11:23 ricolin joined #fuel
11:44 monester_laptop joined #fuel
11:57 corepb joined #fuel
12:02 gongysh_ joined #fuel
12:14 saibarspeis joined #fuel
12:15 admin0 joined #fuel
12:16 admin0 hello all .. i setup fuel .. setup the IPS .. when I try to deploy, the OS installation takes place, but after reboot, it says cannot get IP address from DHCP ..  how do I troubleshoot ?
12:21 gongysh_ joined #fuel
12:21 monester_laptop joined #fuel
12:42 Miouge joined #fuel
12:42 gongysh_ joined #fuel
12:46 kaliya hi admin0 check on the master that dhcp is running `dhcpcheck discover --ifaces eth0`
12:46 kaliya admin0: also you can tcpdump some traffic and see if at least you get the requests: `tcpdump  -n -i eth0 port 68`
12:50 monester_laptop joined #fuel
12:53 admin0 kaliya: i got past that issue .. now every server is booted and ubuntu is installed ( status shows ubuntu install ) .. but nothign else happening .. i logged into one, checked /etc/puppet and got the puppetserver as  fuel.domain.tld …  which I can ping from the fuel server itself, but not from other servers
12:54 kaliya admin0: so you're stuck before the OpenStack provisioning even begins?
12:54 kaliya I think we can look inside the nailgun logs, can you paste them? (on a paste service)
12:55 admin0 one moment please .. i am trying to restart the servers and see if it kicks up something
12:55 kaliya admin0: are they virtual ones, or hardwdare?
12:56 mattgriffin joined #fuel
12:56 admin0 virtual for now .. to see if all works .. and then later copy it to hardware
12:56 kaliya admin0: are you on Fedora + Vbox?
12:56 admin0 ubuntu base server + KVM …  ..
12:56 kaliya admin0: we have some strange issue with network-manager when you try to deploy a HA cluster
12:56 kaliya ah ok
12:57 kaliya how big is your deployment?
12:57 admin0 all nodes (   controller 1 2 3 ,  ceph 1 2 3 and compute  1 2 3 ) are via virt-install
12:57 kaliya 6.0?
12:57 admin0 yep
12:57 kaliya admin0: sometimes to me happens that some server doesn't reboot properly after the OS install on KVM, and I have to force a restart
12:58 kaliya if all are not aligned and rebooted, the installer will wait forever or so, that all are installed+rebooted
12:58 admin0 after a reboot, dns seems to work .. means they can now ping the puppet-master server ( fuel )
12:58 admin0 so hopefully deployment will begin soon
13:00 admin0 yay
13:00 admin0 now I finally see installing openstack :)
13:03 kaliya nice
13:17 monester_laptop joined #fuel
13:33 dkusidlo joined #fuel
13:39 ddmitriev1 joined #fuel
13:39 ddmitriev joined #fuel
13:45 gongysh_ joined #fuel
13:47 admin0 you have a nice nickname: kaliya :D
13:57 gongysh_ joined #fuel
13:58 kaliya thank you admin0
13:58 kaliya did you finish provisoning openstack?
14:00 admin0 controllers are done .. storage almost done … compute note = 5% each
14:00 admin0 i see no neutron nodes .. does this install the DVR ?
14:00 kaliya it installs the neutron server on the controllers
14:01 admin0 what is the impact if say 1 out of 2 controllers fail ? is the network disturbed ?
14:01 admin0 or part goes down
14:01 kaliya admin0: in HA deployments, HA is guaranteed
14:02 kaliya when the quorum is reached, services will go over anything (earthquakes, ufo invasions etc)
14:02 admin0 :)
14:02 admin0 ok
14:03 KenV joined #fuel
14:04 admin0 if there are 3 controllers, is it safe to reboot 2 of them and still have it functional .. and another question is .. if all 3 go out ( power failure ) and come back at the same time, is there a split brain or similar situation or they will be in a quorum themself ?
14:19 KenV joined #fuel
14:24 admin0 my server is almost crashing due to puppet provisioning 6 machines at the same time :)
14:25 youellet joined #fuel
14:28 youellet I have add lbaas plugins to my 6.0.1 fuel. Install successfully but i don't see the plugins check box of lbaas in setting of the new deployment.
14:30 kaliya admin0: in that case they will try to sync after some time to avoid splitbrains
14:31 admin0 ok
14:36 claflico joined #fuel
14:37 youellet vpnaas-plugins does't work too.
14:39 admin0 if for any reason a server goes into error, is it safe to reboot and hope it fixes itself ?
14:39 gongysh_ joined #fuel
14:45 kaliya admin0: depends which error, if it's an upstream openstack or fuel bug, a reboot doesn't help
14:46 admin0 since my test environment is all inside a single physical machine, its cpu is used 100% .. which is why lockups and errors are happening i think
14:48 bazilza joined #fuel
14:48 admin0 another question.. on a environment deplpyed by fuel, what can be chanaged safely, and what will be oeverwritten by puppet ?
14:48 daniel3_ joined #fuel
14:48 daniel3_ joined #fuel
14:49 kaliya admin0: nothing will be overwritten by puppet if you don't manually `puppet apply` some pp
14:49 kaliya admin0: just be sure to manage HA services via pcs instead than via upstart/init.d
14:49 admin0 oh .. so this deploy is just a 1 time thing ? if i change ips etc .. or add say a new controller, doesn’t all other machines know about it ?( puppet apply )
14:49 kaliya that's all, you're free to configure your services
14:50 kaliya admin0: it would require some manual trick, but you can always script this with some rsync and `ssh node-X puppet apply`
14:50 admin0 ok .. so if i do a puppet apply manually, it will re-run the deployment scripts .. if I don’t do that, my changes will be saved
14:51 kaliya that is
14:52 admin0 so if say a server got rebooted or turned-off during deployment, i need to restart it, and then run the agent again manually right ?
14:52 bazilza https://bugs.launchpad.net/mos/+bug/1371723 is there gonna be a release for this fix? fuel 6.0 HA can't be in production without that.
14:53 bazilza meta-data takes sometime more than 2 minutes to fetch and cloud-init just times out
14:55 bazilza i'm running while true; do curl http://169.254.169.254/; sleep 1; done  as a poor mans heartbeat
14:58 emagana joined #fuel
14:59 admin0 kaliya: my controller and compute are READY ..   storage/ceph got into ERROR .. i need to restart, login as root and do puppet apply right ?
14:59 admin0 or how do I start the puppet-deplopyment process again ?
14:59 mihgen dmitryme: please take a look at bazilza issues ^^^
14:59 mihgen actually bug is fix committed, but I see that patches are still being landed
15:00 kaliya bazilza: the fix is commited but not released yet
15:00 mihgen looks like we need to target to 6.1 and set "in progress" ?
15:00 bazilza kaliya, i figured that out. what i can do to pull it?
15:01 bazilza kaliya, installing oslo.messaging 1.8.0 seems like too much trouble.. it has many new deps
15:01 bazilza is there a patch for 1.4.1?
15:05 mattgriffin joined #fuel
15:05 dmitryme bazilza, mihgen: folks, actually the fix is both committed and released
15:05 dmitryme in 6.0
15:06 dmitryme the ‘Fix Released’ status is reserved for QA to mark bugs as verified
15:06 kaliya dmitryme: can you please answer to bazilza 'how to pull the fix'?
15:06 bazilza dmitryme, could you point me to the package?
15:06 dmitryme baziliza: the fix should be already in 6.0
15:07 bazilza dmitryme, i have 6.0 installed
15:07 dmitryme than the fix should be already there
15:07 bazilza dmitryme, it's not
15:07 kaliya bazilza: do you run the official 6.0 or a 6.0 community build?
15:08 bazilza kaliya, official
15:08 bazilza kaliya, signed up and downloaded from mirantis
15:08 admin0 how do I re-start the puppet process on a system which has ERROR ?
15:09 adanin joined #fuel
15:09 kaliya admin0: you can ssh to the node and `puppet apply -v -d /etc/puppet/manifests/site.pp` so you'll even troubleshoot what happened
15:09 admin0 great
15:09 admin0 thanks
15:10 bazilza dmitryme, rabbitmq disconnects clients sicne they don't answer to heartbeat messages. then clients for some reason have trouble connecting. answering with RST to SYN/ACKs from rabbitmq. it takes time but they do reconnect
15:10 bazilza dmitryme, but it's too late and cloud-init gives up already
15:11 bazilza dmitryme, my while true; loop assures constant communication with rabbitmq so there is no need to send heartbeats and meta-data works just fine
15:11 bazilza but running this loop is not a proppa solution
15:12 dmitryme bazilza: could you file a bug, describe the symptoms and attach a snapshot with logs? I think we need further examination to understand if it is the same issue or a new one
15:14 kaliya dmitryme: since you're here :) the fix for 5.x branches. Released? I mean, if need to patch a running cluster
15:15 dmitryme kaliya: let me check
15:16 bazilza dmitryme, what's a snapshot?
15:18 bazilza kaliya, what's the difference between official and community release? i can't any meaningful explanation in the internets
15:19 bazilza kaliya, besides "mirantis logo"
15:19 admin0 kaliya: puppet runs OK .. no issues .. btu in the GUI, the servers are shown as offline
15:19 kaliya bazilza: official is a stable release / nightly are made adding the fixes to the stable, sort of master branch
15:20 daniel3_ joined #fuel
15:20 kaliya admin0: not that easy, since turning the nodes from ERROR to provisioned requires steps like writing in Postgres, you'll have to redeploy
15:20 kaliya admin0: but you can check in Nailgun logs what happened, some timeout?
15:21 kaliya admin0: do you run on qcow2 or raw?
15:21 admin0 raw
15:21 admin0 they are LVM partitions
15:21 admin0 i did the redploy .. will see how it goes
15:21 kaliya on SSD maybe?
15:21 admin0 err .. 4 sata’s on raid10
15:21 admin0 this is a dev machine to test mirantis
15:22 bauzas joined #fuel
15:22 admin0 while doing the reploy, which file I need to tail to what what it is doing ? is it nailgun log or puppet log ?
15:23 kaliya admin0: this page shed some light on the Fuel architecture http://docs.mirantis.com/fuel-dev/develop/architecture.html
15:23 dmitryme kaliya: the fix is present in 5.1.1 packages
15:23 kaliya thanks for the info dmitryme
15:24 admin0 thank you .. this is very helpful
15:24 dmitryme kaliya, also could you please describe bazilza how to make a snapshot? I don’t do that often :-)
15:24 bauzas left #fuel
15:27 admin0 i did redeploy and status is at 0% .. what log file in the fuel server do I need to check whats going on with the process ? i see one docket-nailgun.log .. but the timestap is like 10 minutes back
15:27 kaliya bazilza: go to Support on the top, click "Generate diagnostic snapshot". It could require 10 or 15 mins or so. Then can you please share the archive with us? Maybe by uploading to some public place (google drive or dropbox) and sharing the public URL
15:28 kaliya admin0: in the UI there are some nicelooking logs, just click on the paper icon near the node name in the nodes panel
15:28 kaliya bazilza: I will file the bug for you, if you share the snapshot URL
15:29 admin0 all i see in all is finished catalog run
15:29 admin0 maybe i reboot fuel server itself
15:31 blahRus joined #fuel
15:42 mattgriffin joined #fuel
15:48 mrasskazov left #fuel
15:52 admin0 fuel task list .. i have deploy, deployment and stop_deployment .. is there a way to stop it, so that I can ( for my storage node only in ERROR) use something like fuel --env 1 --deploy --node 6.. where 6 is my node id for the storage server in error
15:59 teran_ joined #fuel
16:00 admin0 what to do if deploy or cancel deploy or redeploy is stuck at 0%
16:00 mattymo admin0, you can delete the task
16:00 admin0 how/where ?
16:00 mattymo fuel task -l    and then fuel task delete --task-id $TASK
16:00 mattymo and if it still doesn't delete, there's -f flag
16:07 admin0 i issued the command .. and clicked redeploy .. when i do task list again, i see  deploy, node_deletion and deployment in the list
16:07 admin0 i guess those are normal
16:07 admin0 however, the status is at 0%
16:07 admin0 and the only servers in ERROR are 2 storage servers, where I can run the puppet agent just fine without errors
16:07 admin0 so hwo do I know where its stuck at 0% ?
16:08 admin0 hmm.. it moved to 54% just like that :)
16:08 rongze joined #fuel
16:08 admin0 i just need some patience it seems
16:08 admin0 where do I read the diference between deploy vs deployment
16:09 admin0 the % is going down :) ..  0 - 54% .. then 51% .. now at 47% :)
16:10 alecv joined #fuel
16:14 tzn joined #fuel
16:25 admin0 joined #fuel
16:31 gongysh joined #fuel
16:42 admin0 if it says deployment failed, check these nodes .. what exactly do I check in there ?
16:43 admin0 i run puppet manually, it works without errors
16:55 xarses joined #fuel
17:04 xarses_ joined #fuel
17:07 e0ne joined #fuel
17:20 thumpba joined #fuel
17:24 stamak joined #fuel
17:31 emagana joined #fuel
17:33 teran joined #fuel
17:34 e0ne joined #fuel
17:37 tzn joined #fuel
17:41 MiroslavAnashkin Depending on roles, assigned to a node Astute may run Puppet several times. Please check Astute logs first
17:56 corepb joined #fuel
17:58 teran joined #fuel
17:59 e0ne joined #fuel
18:09 claflico joined #fuel
18:19 mattgriffin joined #fuel
18:20 claflico joined #fuel
18:27 bazilza joined #fuel
18:28 e0ne joined #fuel
18:32 zerda joined #fuel
18:33 KenV joined #fuel
18:35 KenV I am running Fuel 6.0 with Openstack running on CentOS.  I noticed that every time I delete a cluster, all the floating IPs assigned to the cluster's instances don't get released. I need to manually delete them. Is this expected a expected behavior?  I have not noticed this in the past.
18:36 KenV This is in regards to Sahara clusters.
18:41 monester_laptop joined #fuel
18:54 claflico joined #fuel
18:58 eren joined #fuel
19:04 teran joined #fuel
19:09 emagana joined #fuel
19:19 gongysh joined #fuel
19:46 gongysh joined #fuel
20:16 gongysh joined #fuel
20:18 mattgriffin joined #fuel
20:30 emagana joined #fuel
20:38 corepb joined #fuel
20:52 e0ne joined #fuel
20:57 gongysh joined #fuel
20:58 e0ne joined #fuel
21:00 emagana joined #fuel
21:04 emagana_ joined #fuel
21:06 admin0 joined #fuel
21:22 mattgrif_ joined #fuel
21:33 adanin joined #fuel
21:39 admin0 hello all .. i get:  $primary_controller_nodes[0] is :undef, not a hash or array at /etc/puppet/modules/osnailyfacter/manifests/cluster_ha.pp:148 on node node-10.domain.net
21:39 admin0 what could be the issue?
21:39 admin0 controllers an compute is UP ..   storage is always at ERROR
21:43 daniel3_ joined #fuel
22:11 mattgriffin joined #fuel
22:15 angdraug joined #fuel
22:21 julien_ZTE joined #fuel
22:21 julien_ZTE joined #fuel
22:27 julien_ZTE joined #fuel
22:27 julien_ZTE joined #fuel
22:34 daniel3_ joined #fuel
22:35 emagana joined #fuel
23:11 thumpba joined #fuel
23:28 thumpba_ joined #fuel
23:59 admin0 joined #fuel
23:59 julien_ZTE joined #fuel

| Channels | #fuel index | Today | | Search | Google Search | Plain-Text | summary