Perl 6 - the future is here, just unevenly distributed

IRC log for #fuel, 2015-02-03

| Channels | #fuel index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
01:18 justif joined #fuel
01:29 LiJiansheng joined #fuel
01:36 justif joined #fuel
02:09 rmoe joined #fuel
02:12 teran joined #fuel
02:12 Longgeek joined #fuel
02:15 Longgeek joined #fuel
02:16 xarses joined #fuel
03:13 teran joined #fuel
03:21 dhblaz joined #fuel
03:33 moizarif joined #fuel
03:36 mattgriffin joined #fuel
03:38 vks joined #fuel
03:42 moizarif_ joined #fuel
03:58 vks ping
04:04 Longgeek joined #fuel
04:09 Longgeek joined #fuel
04:13 jobewan joined #fuel
04:14 teran joined #fuel
04:17 Longgeek joined #fuel
04:24 vks ping
04:24 vks hi anyone here???
04:50 Longgeek joined #fuel
04:53 sressot joined #fuel
05:15 teran joined #fuel
05:36 warpc__ joined #fuel
06:16 Longgeek joined #fuel
06:16 xarses vks: its either early or late, wait about another two hours and some people will be around
06:17 vks xarses, thnx will wait
06:17 vks xarses, i think its late for u
06:17 vks ?
06:17 xarses yep, about to head to bed
06:17 teran joined #fuel
06:19 vks xarses, ok will talk some other day :)
06:27 evgeniyl___ joined #fuel
06:36 mattgriffin joined #fuel
06:55 sambork joined #fuel
07:13 dklepikov joined #fuel
07:14 sambork joined #fuel
07:18 teran joined #fuel
07:28 mkwiek joined #fuel
07:30 corepb joined #fuel
07:38 teran joined #fuel
07:44 Miouge joined #fuel
07:55 monester_laptop joined #fuel
08:12 sambork joined #fuel
08:21 Miouge joined #fuel
08:45 okosse joined #fuel
08:45 hyperbaba Hi, i have to move the deployed cloud to another ip address space. Is it possible to change public ip addresses on already running cloud . I have 5.1 deloyed
08:56 alecv joined #fuel
09:00 azemlyanov joined #fuel
09:04 teran joined #fuel
09:24 vks hyperbaba, if it only mean to change the ip of public access, yes u can do that but apart from that I don't think so
09:34 bogdando joined #fuel
09:39 Miouge joined #fuel
09:40 sambork joined #fuel
09:42 stamak joined #fuel
09:46 teran joined #fuel
09:47 e0ne joined #fuel
09:48 teran_ joined #fuel
09:57 hyperbaba vks: thank you.
10:08 omolchanov joined #fuel
10:09 kaliya joined #fuel
10:19 mescanef joined #fuel
10:27 gongysh joined #fuel
10:29 gongysh hi, I am trying to PXE a centos node after installing fuel master node, but  the centos installation UI pops up and asks for language and installation method,
10:29 adanin joined #fuel
10:30 gongysh isn't the node should be installed automatically after booting from pxe?
10:30 gongysh shouldn't the node be installed automatically after booting from pxe?
10:35 moizarif joined #fuel
10:36 aliemieshko hi, http://docs.mirantis.com/openstack/fuel/fuel-6.0/user-guide.html?highlight=pxe#boot-the-fuel-master-node
10:37 aliemieshko be sure to enable PXE-boot on the interface that is on the same network you configured for PXE booting on the Fuel Master Node.
10:55 IvanKliuk joined #fuel
11:00 aleksandr_null joined #fuel
11:10 e0ne joined #fuel
11:19 sambork joined #fuel
11:32 aleksandr_null joined #fuel
11:39 ddmitriev joined #fuel
12:09 IvanKliuk left #fuel
12:23 mkwiek joined #fuel
12:28 saibarspeis joined #fuel
12:28 Longgeek joined #fuel
12:51 evg gongysh: It should be installed automaticaly, mb really wrong interface?
12:57 xarses joined #fuel
12:59 subscope joined #fuel
13:03 agordeev hi. does anybody know what password will be set for root user for slave nodes?
13:13 moizarif agordeev: try r00tme
13:14 gongysh evg, what do u mean by "mb really wrong interface"?
13:15 gongysh evg, I have just one nic on master and a trial slave node.
13:24 evg gongysh: aliemieshko wrote you above "sure to enable pxe boot on the right interface"
13:27 evg gongysh: reboot the node and don't touch it's console. It should boot in the right way.
13:28 evg gongysh: you seem to have touched pxe menu
13:29 championofcyrodi joined #fuel
13:35 Longgeek joined #fuel
13:38 gongysh evg: Tue Feb  3 21:34:21 2015 - INFO | Exception occured: <class 'cobbler.cexceptions.CX'>
13:38 gongysh Tue Feb  3 21:34:21 2015 - INFO | Exception value: 'Error templating file, check cobbler.log for more details'
13:38 gongysh Tue Feb  3 21:34:21 2015 - INFO | Exception Info:
13:38 gongysh File "/usr/lib/python2.6/site-packages/cobbler/remote.py", line 1025, in generate_kickstart
13:38 gongysh return self.api.generate_kickstart(profile,system)
13:38 gongysh File "/usr/lib/python2.6/site-packages/cobbler/api.py", line 684, in generate_kickstart
13:38 gongysh return self.kickgen.generate_kickstart_for_profile(profile)
13:38 gongysh File "/usr/lib/python2.6/site-packages/cobbler/kickgen.py", line 311, in generate_kickstart_for_profile
13:38 gongysh return self.generate_kickstart(profile=g)
13:38 gongysh File "/usr/lib/python2.6/site-packages/cobbler/kickgen.py", line 289, in generate_kickstart
13:38 gongysh data = self.templar.render(raw_data, meta, None, obj)
13:38 gongysh File "/usr/lib/python2.6/site-packages/cobbler/templar.py", line 116, in render
13:38 gongysh data_out = self.render_cheetah(raw_data, search_table, subject)
13:38 gongysh File "/usr/lib/python2.6/site-packages/cobbler/templar.py", line 215, in render_cheetah
13:38 gongysh raise CX("Error templating file, check cobbler.log for more details")
13:39 gongysh evg, # This kickstart had errors that prevented it from being rendered correctly.
13:39 gongysh # The cobbler.log should have information relating to this failure.
13:39 gongysh <gongysh> mattymo: https://10.20.0.2/cblr/svc/op/ks/system reads above.
13:41 lnguyen left #fuel
13:44 mattymo gongysh, diagnostic snapshot. Look in support tab in Fuel UI
13:45 mattymo that error is not enough
13:45 evg gongysh: interesting. could you share your cobbler.log or find what's happening
13:45 mattymo I'm not sure how else I can explain this to you. There is more info to grab
13:45 gongysh evg: where is the cobbler.log?
13:45 mattymo Please generate a diagnostic snapshot
13:46 evg gongysh: diagnostic snapshot will do
13:47 gongysh evg, mattymo, the diagnostic is a tgz file,  where should I put?
13:47 evg gongysh: /var/log/docker-logs/cobbler/cobbler.log    as you've asked
13:47 mattymo gongysh, google drive, dropbox, or in a bug report
14:00 gongysh evg, mattymo: https://bugs.launchpad.net/fuel/+bug/1417588
14:15 evg gongysh: I've read you bug. Thank you for launching it. But could you try just reboot your node and wait. Please _don't_ choose "centos" from PXE menu.
14:20 evg gongysh: let default (local) choice to be loaded
14:25 gongysh evg, ok let me have a try
14:25 SysadminTurncoat joined #fuel
14:28 mattgriffin joined #fuel
14:33 pal_bth joined #fuel
14:51 Longgeek joined #fuel
14:58 denis_makogon joined #fuel
15:04 julien_ZTE joined #fuel
15:06 blahRus joined #fuel
15:34 rbowen joined #fuel
15:47 rongzez joined #fuel
15:50 kozhukalov joined #fuel
15:55 jobewan joined #fuel
16:15 rbowen joined #fuel
16:25 gongysh joined #fuel
16:34 championofcyrodi what does this error mean when deploying an environment?  Controller - storage - loglevel: ERR, "edd: unable to match edd entry 0x80"
16:35 championofcyrodi seem to be getting failures deploying compute/ceph nodes w/ fuel 6.0 on our Dell PowerEdge C6220
16:45 mpetason joined #fuel
16:48 MiroslavAnashkin championofcyrodi: Are you deploying Ceph OSD to the partitions on the disk, which has another partitions-members of soft RAID array?
16:57 Longgeek joined #fuel
17:02 rmoe joined #fuel
17:15 kupo24z joined #fuel
17:16 championofcyrodi MiroslavAnashkin: I didn't think we were... but I just found a /dev/md0 on one of the Dell nodes.
17:18 moizarif joined #fuel
17:18 julien_ZTE joined #fuel
17:18 MiroslavAnashkin 6.0 has new waiting algorithm for busy devices, including /mdxxx ones. And looks like it is buggy algorithm. We expect fix somewhere on this week.
17:22 MiroslavAnashkin Please track this https://bugs.launchpad.net/fuel/+bug/1410471 for progress
17:31 sambork joined #fuel
17:52 daniel3_ joined #fuel
17:54 codybum joined #fuel
17:54 daniel3_ joined #fuel
17:54 codybum Hi All: Using 6.0.1 is anyone having luck with HA?
17:55 codybum I am experiencing stability problems related to RabbitMQ timeouts.
17:55 codybum The system is basically unusable, and requires RabbitMQ resets every 30min
17:56 kupo24z left #fuel
18:15 mattgriffin joined #fuel
18:17 jobewan joined #fuel
18:20 championofcyrodi codybum: what OS are you using?
18:20 championofcyrodi centos or ubuntu?
18:25 daniel3_ joined #fuel
18:27 xarses joined #fuel
18:35 angdraug joined #fuel
18:41 dhblaz joined #fuel
18:48 adanin joined #fuel
18:48 Longgeek joined #fuel
18:50 daniel3_ joined #fuel
19:07 codybum joined #fuel
19:08 daniel3_ joined #fuel
19:08 codybum Does anyone know when the openstack code for rabbitmq keep alive will make it into fuel
19:09 xarses codybum: it's been in fuel in a form or another since 5.0
19:09 xarses the better form is in 5.1 and above
19:09 codybum Mmm.. What about 6.0.1?
19:10 xarses codybum: https://review.openstack.org/#/c/126330/ this one?
19:10 codybum Having specific problem with 3 controller HA in 6.0.1
19:10 codybum @xarses: best I can tell that is what is happening
19:11 codybum resetting rabbitmq or the failing service (neturon, etc.) takes care of the problem temporary.
19:12 codybum I have had the same problem with the 6 release along with nightly builds.
19:12 xarses 6.0.1? you mean its from a nightly from 6.0 stable?
19:12 championofcyrodi if using centos, check the swappiness.
19:13 MiroslavAnashkin Ubuntu also has default swappiness=60
19:13 codybum right.  I have tried experenced the same problem with 6.0 release and nightly builds, which are now listed as 6.0.1
19:13 xarses codybum: hearbeat only takes care of the clients being left hung. If rabbitmq-server is falling down
19:14 MiroslavAnashkin But - it should be already fixed in 6.0.1
19:14 xarses then i would expect all the clients to reconnect automaticly when it's back
19:14 xarses but you would need to look into why rabbit might be falling down
19:14 codybum Here is what is shown in the logs: ERR oslo.messaging._drivers.impl_rabbit [-] Failed to publish message to topic
19:15 codybum then: oslo.messaging._drivers.impl_rabbit [-] Delaying reconnect for 5.0 seconds
19:15 xarses and does reconnecting the client fix it, or restarting rabbit?
19:15 codybum The controllers do reconnect, but this causes provisioning failures.
19:16 MiroslavAnashkin codybum: please check  `cat /proc/sys/vm/swappiness`
19:16 codybum Sometimes just trying to provision something (launch new VM) will work after failure and other times a restart of the controller service (neutron) and/or rabbitmq is required.
19:16 codybum MiroslavAnashkin: ok I will
19:16 MiroslavAnashkin On any controller
19:17 codybum cat /proc/sys/vm/swappiness = 60
19:17 codybum what should the swappiness be?
19:18 MiroslavAnashkin Please set it to recommended 10 on every controller with `sysctl vm.swappiness=10`
19:19 MiroslavAnashkin You may also perform the following check `numactl --hardware`
19:19 xarses codybum: I've heard of similar happening. the root cause for the it was that there was network partitioning issues between the controllers on the management network
19:20 codybum xarses: I am using a VLAN for management
19:20 xarses on centos?
19:21 codybum Ubuntu
19:21 codybum Network validates
19:22 MiroslavAnashkin That network partitioning happened due to all services used the only available NIC on the each node and Ceph generated a lot of traffic, prevented other services from normal syncing.
19:23 codybum Ceph is actually on another interface, but management, internal, and public are on the same interface
19:25 MiroslavAnashkin And our Oslo team is hunting similar bug in Oslo messaging for the third month. While, they claim Oslo needs about month to hang up RabbitMQ...
19:25 codybum cross fingers.  going to try and launch a few nodes
19:25 codybum my config is failing every 30min to hr
19:26 mkwiek joined #fuel
19:26 jobewan joined #fuel
19:27 MiroslavAnashkin Yes. 30 mins is too fast. Can't imagine how Whatsapp works over the RabbitMQ.
19:28 codybum amazing
19:28 codybum Just launched a bunch of host and I have not seen any errors
19:28 codybum Whu would changing swappiness help?
19:29 codybum Errors was almost constant before
19:34 codybum MiroslavAnashkin: Thanks so much.  This issue has been holding up a campus research deployment for several months.
19:34 stamak joined #fuel
19:34 codybum I might still implement a dedicated network for management
19:36 MiroslavAnashkin Well, usually it appears on powerful systems with several CPU sockets. These usually have parts of memory plugged into each socket. Kernel calculated free memory on per-socket basis. And starts using swap for memory defragmentation on new memory block reservations if there is 60 or less percent of free memory available. Such operations usually break RabbitMQ, which has to process any message in just 3 seconds.
19:36 xarses codybum: how much ram do you have on the controller?
19:37 MiroslavAnashkin BTW, it was championofcyrodi who first found that we are stupid and do not follow swappiness recommended settings for nodes with database-like services.
19:38 codybum Ah.. There are three controllers.  Each controller is a Dell 720 with 20 cores and 64G ram.  About half the ram is ued.
19:40 codybum Do you think I should increase the ram on the controllers to 96G?
19:41 xarses and you have two or four sockets?
19:42 teran joined #fuel
19:44 codybum There are 2 sockets : CPU0: Intel(R) Xeon(R) CPU E5-2660 v2 @ 2.20GHz
19:45 mkwiek joined #fuel
19:45 MiroslavAnashkin codybum: Probably, you may start from these steps http://blog.jcole.us/2012/04/16/a-brief-update-on-numa-and-mysql/
19:46 codybum fantastic
19:49 e0ne joined #fuel
20:04 rbowen joined #fuel
20:05 dhblaz joined #fuel
20:22 julien_ZTE joined #fuel
20:24 daniel3_ joined #fuel
20:25 FakeNetEng joined #fuel
20:26 championofcyrodi MiroslavAnashkin: Thanks for the shout out, but the fuel team is far from stupid.  the work you guys are doing is very challenging and tedious.  I'm just glad to be able to contribute something back.
20:42 daniel3_ joined #fuel
20:51 codybum joined #fuel
20:56 Longgeek joined #fuel
21:01 thumpba joined #fuel
21:09 rubick_ joined #fuel
21:10 daniel3_ joined #fuel
21:16 xarses joined #fuel
21:20 rbowen joined #fuel
21:41 dkaigarodsev joined #fuel
21:42 denis_makogon joined #fuel
21:42 aliemieshko joined #fuel
21:42 aarefiev joined #fuel
21:42 avlasov joined #fuel
21:51 adanin joined #fuel
21:53 thumpba joined #fuel
22:06 daniel3_ joined #fuel
22:06 rbowen joined #fuel
22:10 e0ne joined #fuel
22:26 jpf joined #fuel
22:30 rbowen joined #fuel
22:35 julien_ZTE joined #fuel
22:37 FakeNetEng joined #fuel
22:44 jobewan joined #fuel
22:52 maudgib joined #fuel
22:58 mkwiek joined #fuel
23:01 daniel3_ joined #fuel
23:02 Maud`Gib left #fuel
23:06 daniel3_ joined #fuel
23:07 jaypipes joined #fuel
23:26 daniel3_ joined #fuel
23:27 justif joined #fuel
23:29 julien_ZTE joined #fuel
23:35 julien_ZTE joined #fuel

| Channels | #fuel index | Today | | Search | Google Search | Plain-Text | summary