Perl 6 - the future is here, just unevenly distributed

IRC log for #fuel, 2015-03-06

| Channels | #fuel index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:37 MarkDude joined #fuel
01:16 fandi joined #fuel
01:19 rmoe joined #fuel
01:33 fandi joined #fuel
02:25 mattgriffin joined #fuel
03:51 xarses joined #fuel
04:08 Longgeek joined #fuel
04:35 zerda joined #fuel
04:52 claflico joined #fuel
05:00 adanin joined #fuel
05:28 gongysh_ joined #fuel
06:48 sambork joined #fuel
07:19 Longgeek joined #fuel
07:23 aliemieshko_ joined #fuel
07:26 monester_laptop joined #fuel
07:40 dklepikov joined #fuel
07:44 teran_ joined #fuel
07:46 brain461 joined #fuel
07:50 HeOS joined #fuel
08:07 teran joined #fuel
08:17 gongysh_ joined #fuel
08:22 mutex joined #fuel
08:32 saibarspeis joined #fuel
08:41 pbeskow joined #fuel
08:44 JohnGiorno joined #fuel
08:46 ChrisNBlum joined #fuel
08:57 stamak joined #fuel
09:08 alecv joined #fuel
09:32 Longgeek joined #fuel
09:36 adanin joined #fuel
09:37 neophy joined #fuel
09:45 aarefiev joined #fuel
09:49 teran joined #fuel
10:09 e0ne joined #fuel
10:09 sambork1 joined #fuel
10:10 sambork2 joined #fuel
10:14 Longgeek joined #fuel
10:33 teran joined #fuel
10:36 ddmitriev1 joined #fuel
10:36 teran_ joined #fuel
10:39 e0ne joined #fuel
10:43 sambork joined #fuel
10:48 sambork joined #fuel
11:01 sambork1 joined #fuel
11:03 DaveJ__ joined #fuel
11:03 DaveJ__ Hi
11:03 DaveJ__ I'm hitting this issue: https://bugs.launchpad.net/fuel/+bug/1397279
11:04 DaveJ__ Can anyone tell me is it possible to manually update the ubuntu repos on the fuel master manually ?
11:07 hyperbaba joined #fuel
11:09 kaliya DaveJ__: maybe better to pull packages manually
11:13 DaveJ__ kaliya: Where to? Onto the machines where it needs installed, or into the repo on fuel ?
11:13 pal_bth joined #fuel
11:24 neophy joined #fuel
11:33 dklepikov DaveJ__ : what fuel version are you use?
11:35 sambork joined #fuel
11:41 DaveJ__ dklepikov
11:41 DaveJ__ It's 6.0
11:42 DaveJ__ dklepikov: it started as a 5.1 then upgraded to 5.1.1 and finally 6.0
11:47 sambork joined #fuel
11:47 e0ne joined #fuel
11:58 ofkoz joined #fuel
12:03 neophy Hi All, Fuel collects all the openstack node log under /var/log/docker-logs/remote/ directory. there is a directory for each node and they have all the service logs in their own log file. I want to maintain one more copy of logs in another server in the same way which is stored in Fuel /var/log/docker-logs/remote/ path. I am able to configure openstack nodes to forward one more copy to another server by adding : *.* @@(o)<another_server_ip>:514;RemoteLog
12:03 neophy in /etc/rsyslog.d/00-remote.conf file. But I am not sure how to store the in individual service log file. can some one give some thought about how to this?
12:12 sambork1 joined #fuel
12:24 neophy_ joined #fuel
12:44 sovsianikov DaveJ__, Hi! You can take the script from https://github.com/stackforge/fuel-main/blob/master/regenerate_ubuntu_repo.sh but you must pre-test it before using in the work environment ...
12:47 sovsianikov DaveJ__, sorry it's for CentOS only
12:55 sovsianikov _
13:04 sambork joined #fuel
13:11 pal_bth joined #fuel
13:37 maximov joined #fuel
13:46 DaveJ__ sovsianikov_ Thanks anyway
13:56 HeOS__ joined #fuel
14:05 pal_bth joined #fuel
14:09 kat_pimenova joined #fuel
14:09 jkirnosova joined #fuel
14:20 championofcyrod2 joined #fuel
14:20 Longgeek joined #fuel
14:23 championofcyrod2 I can't seem to win with fuel openstack... ever.  I think i've stopped the timed out rabbitmq handshakes that apparently don't start happening until the system actually has been up for a few days and hosting VMs... but now openstack can't find a suitable hosts for any VMs.  I have 4 hosts, and 1 has 0 while another has 6...
14:23 championofcyrod2 i suspect it's disk_allocation_ratio related... but I'm using simplescheduler.
14:24 championofcyrod2 nevermind, just got another handshake_timeout
14:26 championofcyrod2 iowait is extreamly low, there is no swap, and i have 16GB of RAM on each controller with dedicated disks...
14:31 championofcyrod2 ping latency between hosts timing out is between .089-.165 ms
14:31 championofcyrod2 so network latency can't be causing the timeout...
14:31 championofcyrod2 ntp is installed, clocks are synced... so that that can't be it.
14:36 championofcyrod2 rabbitmqctl list_connections shows all of them RUNNING, and none blocked.
14:40 MiroslavAnashkin About suitable hosts - are there " Filter DiskFilter returned 0 hosts" errors?
14:43 championofcyrod2 searching...
14:44 championofcyrod2 the only DiskFilter returns are 4 and then later 3...
14:45 championofcyrod2 in /var/log/nova/scheduler.log
14:45 MiroslavAnashkin These errors usually appear in nova*.log on controllers
14:46 championofcyrod2 it's a DEBUG log
14:46 championofcyrod2 entry
14:47 MiroslavAnashkin If there are no such messages then it is not the disk space reservation issue.
14:47 championofcyrod2 looks to be rabbitmq issues here: http://pastebin.com/9mDsFGAc
14:48 championofcyrod2 again from the /var/log/nova/scheduler.log
14:49 championofcyrod2 yesterday one of my rabbitmq hosts did not show the other two controllers as being part of the cluster.  so i restarted the controller.  it had a few entries in the rabbitmq log about being shutdown, but once it came back up, rabbitmqctl cluster_status showed all controllers were back in the cluster.
14:50 clauded joined #fuel
14:51 championofcyrod2 ran cluster_status on all 3 controllers, and they all are reporting each other as running_nodes...
14:51 championofcyrod2 wondering if openstack AMPQ clients need to be restarted.
14:53 clauded Hi. I'm testing Fuel 6.1. In prior version, I could add custom puppet code for the deployment of nodes. What I did was just adding "include custom_packages" in the file /etc/puppet/manifests/site.pp. Is it still to right way to do it in 6.1?
15:00 MiroslavAnashkin championofcyrod2: Our Oslo team has suspision it is either Oslo slowly kills Rabbit or Rabbit has bug.
15:00 MiroslavAnashkin clauded: There is no more site.pp in 6.1
15:02 MiroslavAnashkin clauded: Deployment starting from  6.1 is broken into small granular tasks.  You may add your package into one of *.pp files, mentioned here:
15:02 MiroslavAnashkin https://github.com/stackforge/fuel-library/blob/master/deployment/puppet/osnailyfacter/modular/roles/tasks.yaml
15:03 MiroslavAnashkin Or add a whole new task.
15:05 MiroslavAnashkin If deployment failed - you may continue from failed task and skip the tasks finished successfully. I hope it already works.
15:10 clauded @MiroslavAnashkin: to create a new task, I add it to tasks.yaml and create a corresponding custom_task.pp right?
15:13 clauded MiroslavAnashkin: and this would create a new role, correct?
15:24 MiroslavAnashkin clauded: You may add new role or add just a package to already existing role.
15:32 e0ne joined #fuel
15:41 claflico joined #fuel
15:42 MiroslavAnashkin championofcyrod2: Forwarded the issue to Oslo team. Let them dig deeper.
15:42 CheKoLyN joined #fuel
15:58 stamak joined #fuel
16:02 Miouge joined #fuel
16:10 mattgriffin joined #fuel
16:14 jaypipes joined #fuel
16:16 clauded MiroslavAnashkin: when a new task is created, what services need to be restarted so it's available in the webui?
16:20 daniel3_ joined #fuel
16:27 championofcyrod2 MiroslavAnashkin:  thanks.  I guess I'll switch to CLI and see if I can get things working again.
16:27 championofcyrod2 since Horizon is hosed.
16:31 daniel3_ joined #fuel
16:32 aleksandr_null joined #fuel
16:43 thumpba joined #fuel
16:51 thumpba joined #fuel
16:55 thumpba_ joined #fuel
16:57 championofcyrod2 well.  i just shut down ALL instances, and restarted all compute+ceph nodes as well as all controllers...
17:01 DaveJ__ Has anyone tried out multi domain authentication with Fuel 6.0 ?  Not sure if Keystone supports it our not
17:01 DaveJ__ I 'd like to authenticate the Openstack services against sql, but let users auth against an ldap
17:01 DaveJ__ cant seem to find any clear instructions
17:07 clauded joined #fuel
17:13 championofcyrod2 Am I running the right version if I wanted to use this software for production?  Are the older versions more stable for targeting system uptime?
17:15 MiroslavAnashkin 6.0 is the most stable version currently. However, both 5.1.1 and 6.0 (and 6.1) use the same RabbitMQ+Oslo messaging version.
17:16 clauded joined #fuel
17:20 rmoe joined #fuel
17:36 championofcyrod2 well, i can't get anything to power on, all vms are dead. rabbitmq is just erroring out and restarting itself with this error: http://paste.openstack.org/show/190470/
17:36 championofcyrod2 it accepts a few connections, then times out, then halts on merge_schema_failed, incompatible schema cookies.  and wants me to restart rabbitmq from an old backup.
17:37 championofcyrod2 i see mentions of cstruct, so maybe some kind of serialization issue?
17:43 adanin joined #fuel
17:46 MiroslavAnashkin championofcyrod2: Please stop rabbitMQ with `pcs resource disable master_p_rabbitmq-server`, wait 2 minutes, and start it with `pcs resource enable master_p_rabbitmq-server`
17:50 championofcyrod2 do this on all three controllers?
18:12 thumpba joined #fuel
18:12 thumpba_ joined #fuel
18:21 xarses joined #fuel
18:22 e0ne joined #fuel
18:33 championofcyrod2 okay... well rabbitmq seems better... however i noticed ceph osds were all down except 1... so i just restarted those and it's healthy again... let's see if i can get these vms up!
18:38 championofcyrod2 okay.  things are back up...
18:39 championofcyrod2 it was a combination of rabbitmq failing... my reboot i guess not starting ceph osds?  and a volume not being able to report attachment status correctly because it was attached to 'None' since the instance was deleted while it was attached.
18:39 championofcyrod2 once i manually detached it in mysql, the API started working again
18:39 championofcyrod2 and all my VMs fired up blazing fast
18:41 championofcyrod2 awesome...
18:41 championofcyrod2 so all this started last night when i tried to spawn an instance with a 1TB volume that was imported.
18:42 championofcyrod2 just now spawned in a few seconds and is up and running fine in the console.
18:43 championofcyrod2 good grief... its so freakin responsive, like after a fresh install... i think need to run the pcs resource disable/enable nightly in a crontab.
18:43 championofcyrod2 ;) not really.  but it seems to have helped.
18:47 adanin joined #fuel
18:59 championofcyrod1 joined #fuel
19:05 daniel3__ joined #fuel
19:05 championofcyrod1 left #fuel
19:34 okosse1 joined #fuel
19:36 daniel3_ joined #fuel
19:45 Akshik joined #fuel
19:46 angdraug joined #fuel
19:48 tatyana joined #fuel
19:49 adanin joined #fuel
19:50 Guest58 joined #fuel
19:50 Guest58 Trying to find docs on how to set up Provider networks using FUEL.
19:54 clauded left #fuel
20:17 adanin joined #fuel
20:45 adanin joined #fuel
21:17 jobewan joined #fuel
21:27 teran joined #fuel
21:52 mattymo joined #fuel
21:56 stamak joined #fuel
22:10 thumpba joined #fuel
22:10 adanin joined #fuel
22:13 thumpba_ joined #fuel
22:14 thumpba__ joined #fuel
22:22 e0ne joined #fuel
22:37 Akshik_ joined #fuel
23:15 Arminder joined #fuel

| Channels | #fuel index | Today | | Search | Google Search | Plain-Text | summary