Perl 6 - the future is here, just unevenly distributed

IRC log for #fuel, 2014-01-06

| Channels | #fuel index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
23:44 rongze joined #fuel
00:44 rongze joined #fuel
01:02 sparc joined #fuel
02:00 rongze joined #fuel
03:36 ArminderS joined #fuel
03:50 e0ne joined #fuel
04:28 dhblaz joined #fuel
04:36 dhblaz joined #fuel
04:39 ArminderS- xarses: did you happen to test NIC bonding?
04:51 e0ne joined #fuel
04:56 Arminder joined #fuel
05:03 ArminderS joined #fuel
06:04 e0ne joined #fuel
06:18 rongze joined #fuel
06:21 rongze_ joined #fuel
06:42 fandikurnia01 joined #fuel
09:43 rongze joined #fuel
10:06 alex_didenko joined #fuel
11:05 rongze joined #fuel
11:35 rongze joined #fuel
11:43 e0ne joined #fuel
12:35 rongze joined #fuel
12:46 e0ne joined #fuel
13:25 rongze joined #fuel
14:41 sparc joined #fuel
14:46 miguitas joined #fuel
14:59 rongze joined #fuel
15:08 sparc joined #fuel
15:10 _ruhe joined #fuel
15:20 rongze joined #fuel
15:26 richardkiene_ joined #fuel
15:51 dhblaz joined #fuel
15:52 dhblaz http://paste.openstack.org/show/60445/
15:54 dhblaz The 2nd and 3rd errors look like https://bugs.launchpad.net/fuel/+bug/1259066
15:54 dhblaz This ticket attributes the problem to an old version of puppet.
15:54 dhblaz I installed from the 4.0 img downloaded from the web site.
15:55 dhblaz The nodes are CentOS4
15:57 angdraug joined #fuel
15:58 albionandrew joined #fuel
16:16 albionandrew_ joined #fuel
16:17 rongze joined #fuel
16:19 rongze joined #fuel
16:19 rongze joined #fuel
16:20 rongze joined #fuel
16:21 rongze joined #fuel
16:22 rongze joined #fuel
16:23 rongze joined #fuel
16:25 rongze joined #fuel
16:26 rongze joined #fuel
16:27 rongze joined #fuel
16:27 rongze joined #fuel
16:28 rongze joined #fuel
16:29 rongze joined #fuel
16:30 rongze joined #fuel
16:30 rongze joined #fuel
16:31 rongze joined #fuel
17:02 sparc joined #fuel
17:07 e0ne_ joined #fuel
17:21 xarses joined #fuel
17:22 e0ne joined #fuel
17:24 e0ne joined #fuel
17:42 rmoe joined #fuel
17:46 angdraug joined #fuel
17:50 rongze joined #fuel
17:51 rongze joined #fuel
17:52 rongze joined #fuel
17:53 rongze joined #fuel
17:54 rongze joined #fuel
17:55 rongze joined #fuel
17:55 rongze joined #fuel
17:56 rongze joined #fuel
17:57 e0ne joined #fuel
17:57 rongze joined #fuel
17:58 rongze joined #fuel
17:59 rongze joined #fuel
18:01 rongze joined #fuel
18:02 rongze joined #fuel
18:04 e0ne joined #fuel
18:05 rongze_ joined #fuel
18:05 rongze joined #fuel
18:06 rongze joined #fuel
18:09 dhblaz http://paste.openstack.org/show/60445/
18:09 dhblaz The 2nd and 3rd errors look like https://bugs.launchpad.net/fuel/+bug/1259066
18:09 rongze joined #fuel
18:09 dhblaz I installed from the 4.0 img downloaded from the web site.
18:09 dhblaz The nodes are CentOS4
18:10 dhblaz We are also seeing the bug described here:
18:10 dhblaz https://bugs.launchpad.net/fuel/+bug/1259935
18:10 rongze_ joined #fuel
18:10 dhblaz But I can't find the commit that fixed it.
18:11 rongze_ joined #fuel
18:14 rongze joined #fuel
18:37 angdraug dhblaz: #1259935 was fixed by a change in mcollective deb package, are you seeing it on CentOS, too?
18:40 angdraug re lines 2 and 3, I've seen these show up in successful deployments
18:41 angdraug it's definitely not #1259066
18:44 dhblaz angdraug: 1259935: we see this error message when trying to "Verify Networks"
18:44 dhblaz So it is in the boot strap
18:45 rongze joined #fuel
18:45 dhblaz bootstrap rather
18:46 rongze joined #fuel
18:46 rongze joined #fuel
18:47 rongze joined #fuel
18:48 rongze joined #fuel
18:49 rongze joined #fuel
18:50 angdraug ah. looks like we should update the init script in the mcollective rpm with a similar fix, too
18:50 rongze joined #fuel
18:50 angdraug can you post the logs on the bug in LP?
18:51 rongze_ joined #fuel
18:51 dhblaz Which logs?
18:52 dhblaz albionandrew: Will you coordinate with angdraug to get him what he needs please?
18:52 rongze joined #fuel
18:52 albionandrew dhblaz sure
18:53 rongze joined #fuel
18:53 angdraug I would expect something like what's in #1259935's description to show up in /var/log/naily/naily.log
18:53 angdraug on the master node
18:54 rongze joined #fuel
18:54 angdraug then I'd be able to reopen that bug and forward it to the packaging team
18:55 rongze joined #fuel
18:56 rongze joined #fuel
18:57 rongze joined #fuel
18:58 rongze joined #fuel
18:59 rongze joined #fuel
18:59 rongze joined #fuel
19:00 rongze joined #fuel
19:01 rongze joined #fuel
19:02 rongze joined #fuel
19:03 rongze joined #fuel
19:04 albionandrew angdraug What specifically would you like from me with regard to logs etc? I assume I can't submit the snapshot because it will have private info in.
19:04 albionandrew angdraug but the message in the nail log is the same
19:04 rongze joined #fuel
19:06 rongze joined #fuel
19:06 angdraug if it's exactly the same, just add the note to the bug that you're seeing the same from 4.0 GA version when verifying networks with just the bootstrapped nodes
19:07 albionandrew angdraug will do that now thanks
19:07 rongze joined #fuel
19:08 rongze joined #fuel
19:09 rongze joined #fuel
19:10 rongze joined #fuel
19:11 rongze joined #fuel
19:11 rongze joined #fuel
19:12 rongze joined #fuel
19:13 rongze joined #fuel
19:17 rongze joined #fuel
19:18 rongze joined #fuel
19:18 rongze joined #fuel
19:20 rongze joined #fuel
19:23 rongze_ joined #fuel
19:25 rongze_ joined #fuel
19:26 rongze joined #fuel
19:27 rongze joined #fuel
19:27 rongze joined #fuel
19:28 rongze joined #fuel
19:29 rongze joined #fuel
19:31 rongze_ joined #fuel
19:33 rongze joined #fuel
19:34 rongze_ joined #fuel
19:35 rongze_ joined #fuel
19:39 rongze joined #fuel
19:39 rongze joined #fuel
19:40 rongze joined #fuel
19:40 rongze joined #fuel
19:41 rongze joined #fuel
19:42 rongze joined #fuel
19:42 rongze joined #fuel
19:45 rongze joined #fuel
19:46 rongze joined #fuel
19:48 albionandrew joined #fuel
19:48 rongze joined #fuel
19:49 dhblaz I'm not sure if anyone saw this on Saturday:
19:49 dhblaz On Fuel 4.0 using the web gui my storage nodes are detected with the storage arrays as sda and the drives indended for os as sdb.  This sets the minimum os size to (in my case) 48G but the OS drives are only 32G.  Is there an easy way to work around this?
19:49 rongze joined #fuel
19:51 dhblaz I ended up modifying  def _calc_root_size(self) in /opt/nailgun/lib/python2.6/site-packages/nailgun/volumes/manager.py so it always returned gb_to_mb(15)
19:51 dhblaz Not pretty but it did get me to the next step.
19:52 rongze_ joined #fuel
19:53 rongze joined #fuel
19:53 rongze joined #fuel
19:54 rongze joined #fuel
19:54 rongze joined #fuel
19:56 rongze_ joined #fuel
19:58 rongze_ joined #fuel
19:59 rongze_ joined #fuel
20:00 rongze joined #fuel
20:02 rongze_ joined #fuel
20:04 rongze_ joined #fuel
20:05 rongze_ joined #fuel
20:06 rongze__ joined #fuel
20:07 rongze joined #fuel
20:08 rongze joined #fuel
20:18 angdraug albionandrew: have you seen AndreyK's comment on #1259935? can you check how many mco processes you get on your bootstrap nodes when that error occurs?
20:19 rongze joined #fuel
20:19 angdraug dhblaz: _minimum_ size at 48G? that can't be right
20:19 rongze joined #fuel
20:20 dhblaz angdraug: if you look at _calc_root_size
20:20 rongze joined #fuel
20:20 dhblaz it is the way it is written
20:20 dhblaz I think the feature is unintended
20:21 rongze joined #fuel
20:21 dhblaz https://github.com/stackforge/fuel-web/blob/master/nailgun/nailgun/volumes/manager.py#L648
20:22 dhblaz sorry
20:22 dhblaz https://github.com/stackforge/fuel-web/blob/master/nailgun/nailgun/volumes/manager.py#L689
20:22 dhblaz The problem is in the first line
20:22 dhblaz size = int(self.disks[0].size * 0.2)
20:22 angdraug yep, I see it
20:23 dhblaz So the minimum size could be even higher than my case
20:23 dhblaz 50GB+swap
20:23 rongze_ joined #fuel
20:23 angdraug it assumes that OS always goes on the first drive
20:24 dhblaz Yes, You can move the OS to another drive but it doesn't recalculate the size
20:25 angdraug I'll create a bug
20:25 dhblaz I am not sure if it makes sense but whoever writes the fix should know that the gui will let you put the OS on more than one of the volumes.  I haven't looked into how this actually works.  Perhaps it uses LVM or it just doesn't work.
20:26 angdraug it uses LVM
20:26 dhblaz My point being that if you try to find the volume for the OS you may find that it is on more than one block device according to the config.
20:26 angdraug yeah
20:26 rongze_ joined #fuel
20:26 dhblaz So getting the size that way may not be the best way to go either.
20:27 rongze_ joined #fuel
20:28 rongze joined #fuel
20:30 angdraug seems that whole commit was made with only Ubuntu in mind
20:31 angdraug on Ubuntu, we always use the first drive and there's no LVM, that's the only case where that assumption makes sense
20:32 angdraug one more problem with that code is that it's supposed to set the default root size, not minimum root size
20:32 dhblaz Ubuntu at this stage doesn't make any sense
20:32 dhblaz it is in bootstrap
20:32 angdraug drives are not partitioned in bootstrap
20:32 angdraug they're partitioned during OS install by either anaconda or preseed
20:33 dhblaz The paramenters for the gui are setup in boot strap
20:33 dhblaz and then nailgund uses these parameters to check the disk setup before deployment
20:34 dhblaz So even when I made the gui accept my parameters deploy would fail before it started because of nailgund's response
20:34 rongze_ joined #fuel
20:35 angdraug I'm not trying to justify that code, just figuring out the line of reasoning that has led to it
20:35 dhblaz I'm pretty sure this code isn't run after the OS is on the drive
20:35 rongze joined #fuel
20:35 angdraug no, you didn't get my point about Ubuntu
20:36 angdraug because Ubuntu preseed doesn't let you put OS on anything but the first drive, it's reasonable to assume that root size calculations can be based on whatever you get as the first drive in bootstrap
20:36 angdraug well, reasonable to a point
20:36 rongze_ joined #fuel
20:37 dhblaz https://github.com/stackforge/fuel-web/blob/master/nailgun/nailgun/task/task.py#L438
20:37 angdraug since CentOS allows you to put your root anywhere, even across multiple drives with LVM, that assumption doesn't work there
20:37 rongze_ joined #fuel
20:38 dhblaz https://github.com/stackforge/fuel-web/blob/master/nailgun/nailgun/task/task.py#L406
20:38 dhblaz and L417
20:39 rongze joined #fuel
20:39 dhblaz will someone +b rongze?
20:40 rongze joined #fuel
20:42 rongze_ joined #fuel
20:43 rongze joined #fuel
20:45 rongze_ joined #fuel
20:47 rongze joined #fuel
20:51 rongze joined #fuel
20:55 angdraug https://bugs.launchpad.net/fuel/+bug/1266569
20:58 angdraug dhblaz: thanks for digging this up!
20:59 dhblaz angdraug: Nice summary
21:05 dhblaz When bootstrap_admin_node.sh starts we see this error message:
21:05 dhblaz ls: cannot access /dev/cciss!c0d0?: No such file or directory
21:05 dhblaz I'm trying to hunt down where in the puppet manifests this comes from
21:06 angdraug rmoe: didn't you fix this recently?
21:07 dhblaz While I'm digging in this log file I see this too:
21:07 dhblaz Could not autoload package: Could not autoload /usr/lib/ruby/site_ruby/1.8/puppet/provider/package/yum.rb: No child processes at /etc/puppet/modules/postgresql/manifests/server.pp:26 on node xxx
21:09 angdraug the most recent cciss fix we made was this: https://review.openstack.org/#/c/61289/4/bin/agent
21:09 angdraug looks like bootstrap_admin_node.sh needs something similar
21:13 dhblaz The last line of that script is puppet apply  /etc/puppet/modules/nailgun/examples/site.pp
21:13 dhblaz the problem comes from running that command
21:22 rongze joined #fuel
21:23 rongze joined #fuel
21:24 rongze joined #fuel
21:25 rongze joined #fuel
21:26 rongze joined #fuel
21:27 rongze joined #fuel
21:28 rongze joined #fuel
21:29 rongze joined #fuel
21:30 rongze joined #fuel
21:30 dhblaz The code path is too complex for me to follow without running it
21:30 rongze joined #fuel
21:31 dhblaz I don't think it is safe to run puppet on this node again (as we are trying to make a cluster with it)
21:31 rongze joined #fuel
21:32 rongze joined #fuel
21:32 rmoe_ joined #fuel
21:33 rongze_ joined #fuel
21:34 rongze_ joined #fuel
21:35 rongze__ joined #fuel
21:36 rongze_ joined #fuel
21:37 rongze_ joined #fuel
21:38 rongze joined #fuel
21:39 rongze joined #fuel
21:40 rongze joined #fuel
21:41 rongze joined #fuel
21:42 rongze joined #fuel
21:42 rongze_ joined #fuel
21:43 rongze joined #fuel
21:44 rongze joined #fuel
21:45 rongze joined #fuel
21:46 rongze joined #fuel
21:47 rongze_ joined #fuel
21:48 rongze joined #fuel
21:49 mrasskazov1 joined #fuel
21:49 rongze joined #fuel
21:50 rongze joined #fuel
21:51 angdraug can you compare contents of /etc/puppet/ between that node and master?
21:51 rongze joined #fuel
21:51 angdraug puppet must be seriously broken on that node, first thing to check if it failed to rsync some manifests
21:52 rongze joined #fuel
21:54 rongze_ joined #fuel
21:55 rongze_ joined #fuel
21:56 rongze joined #fuel
21:57 rongze joined #fuel
21:57 rongze joined #fuel
21:58 rongze_ joined #fuel
22:01 rongze joined #fuel
22:01 rongze joined #fuel
22:02 rongze joined #fuel
22:04 rongze_ joined #fuel
22:08 rongze joined #fuel
22:09 rongze_ joined #fuel
22:10 rongze_ joined #fuel
22:11 rongze joined #fuel
22:12 rongze joined #fuel
22:12 rongze joined #fuel
22:14 rongze joined #fuel
22:14 rongze joined #fuel
22:16 rongze__ joined #fuel
22:50 rongze joined #fuel
22:51 rongze joined #fuel
22:52 rongze joined #fuel
22:53 rongze joined #fuel
22:54 rongze joined #fuel
22:55 rongze joined #fuel
22:56 rongze joined #fuel
22:57 rongze joined #fuel
22:58 rongze joined #fuel
22:59 rongze joined #fuel
22:59 rongze joined #fuel

| Channels | #fuel index | Today | | Search | Google Search | Plain-Text | summary