Perl 6 - the future is here, just unevenly distributed

IRC log for #openstack-rally, 2015-01-18

| Channels | #openstack-rally index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:55 rook__ joined #openstack-rally
01:29 baker joined #openstack-rally
02:19 openstackgerrit Mikhail Dubov proposed stackforge/rally: Restructure Rally docs  https://review.openstack.org/120018
02:19 openstackgerrit Mikhail Dubov proposed stackforge/rally: Several improvements in Rally info  https://review.openstack.org/148078
02:22 openstackgerrit Mikhail Dubov proposed stackforge/rally: Several improvements in Rally info  https://review.openstack.org/148078
02:24 openstackgerrit Mikhail Dubov proposed stackforge/rally: [WIP] Add SLA to check for outliers  https://review.openstack.org/148079
02:28 openstackgerrit Mikhail Dubov proposed stackforge/rally: Several improvements in Rally info  https://review.openstack.org/148078
02:29 openstackgerrit Mikhail Dubov proposed stackforge/rally: Several improvements in Rally info  https://review.openstack.org/148078
02:44 rook__ joined #openstack-rally
03:57 baker joined #openstack-rally
04:33 rook__ joined #openstack-rally
05:10 andreykurilin_ joined #openstack-rally
05:23 tfreger joined #openstack-rally
05:24 tfreger joined #openstack-rally
06:13 openstackgerrit Mikhail Dubov proposed stackforge/rally: Restructure Rally docs  https://review.openstack.org/120018
06:17 openstackgerrit Mikhail Dubov proposed stackforge/rally: Several improvements in Rally info  https://review.openstack.org/148078
06:20 yfried|afk joined #openstack-rally
06:22 rook__ joined #openstack-rally
06:24 yfried joined #openstack-rally
07:56 nmagnezi joined #openstack-rally
08:10 rook__ joined #openstack-rally
08:15 boris-42 yfried: hey
08:16 boris-42 yfried: you know I hate to say but Nova is again broken
08:16 yfried boris-42: good morning
08:16 boris-42 yfried: morning
08:16 yfried boris-42: I swear it was like this when I got here :)
08:16 yfried boris-42: so?
08:16 boris-42 yfried: no I fixed it
08:16 boris-42 yfried: https://review.openstack.org/#/c/147751/
08:17 boris-42 yfried: one time
08:17 boris-42 yfried: but they broke it yesterday one more time
08:17 yfried boris-42: what are we talking about?
08:17 yfried boris-42: I noticed tempest patches fail on icehouse
08:17 yfried boris-42: is that related?
08:17 boris-42 yfried: not sure ( I think not)
08:18 boris-42 yfried: btw now we have rally-gate-dsvm-unstable job
08:19 boris-42 yfried: where I will put everything unstable (so we won't have blocked gates)
08:19 yfried boris-42: seems nice
08:19 yfried boris-42: so I have a question
08:20 boris-42 yfried: this http://logs.openstack.org/18/120018/39/check/gate-rally-dsvm-rally/cceeb5b/rally-plot/results.html.gz#/NovaServers.boot_and_delete_server-2 is failing
08:20 boris-42 yfried: now very often*
08:20 boris-42 yfried: what is the question?
08:20 yfried boris-42: your thing first
08:21 yfried boris-42: are we sure it's a nova issue, and not a gate limited resources thing?
08:21 boris-42 yfried: previous time a got the same question
08:22 boris-42 yfried: but it's just boot-and-delete method
08:22 boris-42 yfried: if it was a resource limited issue -> we will get more errors in other benchmarks
08:22 boris-42 yfried: and they will be randomly
08:22 boris-42 yfried: but we are getting always the same error
08:22 boris-42 (like we have during boot from volume bug)
08:24 yfried boris-42: I'm still not familiar with the rally gate or the whole list of rally benchmarks. sorry
08:25 yfried so it's a consistent failure for all runs?
08:25 boris-42 yfried: most of them
08:25 boris-42 yfried: hm rally gates are just dsvm job
08:25 boris-42 + rally task start command
08:25 boris-42 that is all
08:25 yfried boris-42: do you have a way (logstash maybe?) to see all recent failed runs?
08:26 boris-42 yfried: you can use logstach
08:26 boris-42 yfried: in the same way as in tempest
08:27 boris-42 yfried: as well you can click here http://logs.openstack.org/18/120018/39/check/gate-rally-dsvm-rally/cceeb5b/
08:28 boris-42 yfried: logs of all services
08:28 yfried boris-42: seems like a single patch log.
08:29 yfried boris-42: I was wondering about a pie chart of all patches from today
08:29 boris-42 yfried: ?
08:30 boris-42 yfried: take a look here
08:30 boris-42 http://logstash.openstack.org/#eyJzZWFyY2giOiJwcm9qZWN0Olwic3RhY2tmb3JnZS9yYWxseVwiIEFORCBcIk5vIHZhbGlkIGhvc3Qgd2FzIGZvdW5kXCIiLCJmaWVsZHMiOltdLCJvZmZzZXQiOjAsInRpbWVmcmFtZSI6ImN1c3RvbSIsImdyYXBobW9kZSI6ImNvdW50IiwidGltZSI6eyJmcm9tIjoiMjAxNS0wMS0xNVQwODoyOTozMiswMDowMCIsInRvIjoiMjAxNS0wMS0xOFQwODoyOTozMiswMDowMCIsInVzZXJfaW50ZXJ2YWwiOiIwIn0sInN0YW1wIjoxNDIxN
08:30 boris-42 TY5ODIxNzA0fQ==
08:30 boris-42 fuu
08:30 Miouge joined #openstack-rally
08:31 yfried boris-42: please repost the link
08:31 boris-42 yfried: http://goo.gl/tghqOD
08:31 boris-42 yfried:  it's just logstach with this erorror for 2 days
08:33 yfried boris-42: yeah.
08:33 yfried boris-42: I'm a logstash novice
08:34 boris-42 yfried: so you see on top graph
08:34 yfried boris-42: can we see the percentage of jobs that failed with this error out of all jobs
08:34 boris-42 yfried: nope
08:34 yfried boris-42: too bad
08:35 boris-42 yfried: these numbers are enough
08:35 yfried boris-42: i guess
08:35 boris-42 yfried: to say that we are fucked one more time by nova team
08:35 yfried boris-42: we can definitely see a spike
08:35 boris-42 yfried: so first group of errors was when they broke boot from volume functionallity
08:35 boris-42 yfried: then it stopped when my patch in nova got merged
08:36 boris-42 yfried: and now it retruned back
08:36 yfried http://logstash.openstack.org/#eyJzZWFyY2giOiJcIk5vIHZhbGlkIGhvc3Qgd2FzIGZvdW5kXCIiLCJmaWVsZHMiOltdLCJvZmZzZXQiOjAsInRpbWVmcmFtZSI6ImN1c3RvbSIsImdyYXBobW9kZSI6ImNvdW50IiwidGltZSI6eyJmcm9tIjoiMjAxNS0wMS0xMVQwODoyOTozMiswMDowMCIsInRvIjoiMjAxNS0wMS0xOFQwODoyOTozMiswMDowMCIsInVzZXJfaW50ZXJ2YWwiOiIwIn0sInN0YW1wIjoxNDIxNTcwMTMwNjExfQ==
08:38 boris-42 yfried: without filter for project*
08:38 yfried boris-42: yeah
08:38 boris-42 yfried: it doesn't make too much sense
08:38 yfried boris-42: hang on a sec
08:38 yfried boris-42: http://logstash.openstack.org/#eyJzZWFyY2giOiJcIk5vIHZhbGlkIGhvc3Qgd2FzIGZvdW5kXCIiLCJmaWVsZHMiOltdLCJvZmZzZXQiOjAsInRpbWVmcmFtZSI6Ijg2NDAwIiwiZ3JhcGhtb2RlIjoiY291bnQiLCJ0aW1lIjp7ImZyb20iOiIyMDE1LTAxLTE0VDE1OjIxOjA4KzAwOjAwIiwidG8iOiIyMDE1LTAxLTE4VDA3OjU0OjMzKzAwOjAwIiwidXNlcl9pbnRlcnZhbCI6MH0sInN0YW1wIjoxNDIxNTcwMjUzNTg2fQ==
08:38 yfried boris-42: ^ that's today
08:39 yfried http://logstash.openstack.org/#eyJzZWFyY2giOiJcIk5vIHZhbGlkIGhvc3Qgd2FzIGZvdW5kXCIiLCJmaWVsZHMiOltdLCJvZmZzZXQiOjAsInRpbWVmcmFtZSI6ImN1c3RvbSIsImdyYXBobW9kZSI6ImNvdW50IiwidGltZSI6eyJmcm9tIjoiMjAxNS0wMS0xMVQwODoyOTozMiswMDowMCIsInRvIjoiMjAxNS0wMS0xMlQwODoyOTozMiswMDowMCIsInVzZXJfaW50ZXJ2YWwiOiIwIn0sInN0YW1wIjoxNDIxNTcwMjc2MzI5fQ==
08:39 yfried boris-42: ^ that's last sunday
08:39 yfried boris-42: so it seems it's a rise across all tests
08:39 boris-42 yfried: first link is about whole week
08:39 yfried boris-42: I'm talking about last 2 links
08:43 yfried boris-42: just saying that the error is happening on other projects as well
08:43 yfried boris-42: http://logstash.openstack.org/#eyJzZWFyY2giOiJcIk5vIHZhbGlkIGhvc3Qgd2FzIGZvdW5kXCIiLCJmaWVsZHMiOlsicHJvamVjdCJdLCJvZmZzZXQiOjAsInRpbWVmcmFtZSI6Ijg2NDAwIiwiZ3JhcGhtb2RlIjoiY291bnQiLCJ0aW1lIjp7ImZyb20iOiIyMDE1LTAxLTE0VDE1OjIxOjA4KzAwOjAwIiwidG8iOiIyMDE1LTAxLTE4VDA3OjU0OjMzKzAwOjAwIiwidXNlcl9pbnRlcnZhbCI6MH0sInN0YW1wIjoxNDIxNTcwMjUzNTg2LCJtb2RlIjoic2NvcmUiLCJhbmFseXplX2ZpZWxkIjoicHJvamVjdCJ9
08:44 yfried boris-42: looks like rally isn't the projects that's hit the most
08:45 boris-42 yfried: one more time
08:45 boris-42 yfried: you are looking for whole week
08:45 boris-42 yfried: not for 2 days
08:46 boris-42 ah no this time it's ok
08:47 boris-42 yfried: in any case there is too much noise
08:47 yfried boris-42: so how do you want to solve this?
08:47 boris-42 yfried: cause tempest is running against everything everywhere (including bad patches)
08:47 yfried boris-42: yeah, rally might be a better indicator in that sense
08:48 boris-42 yfried: take a look at level of bugs on 14 ane 15
08:48 boris-42 yfried: I mean no valid host() messages
08:48 boris-42 yfried: it's the same as after 16()
08:48 yfried boris-42: but tempest project patches are also running against the "good" code
08:48 boris-42 yfried: and?
08:48 yfried boris-42: nothing
08:48 boris-42 yfried: I am saying just about too much noise to detect
08:48 boris-42 =)
08:49 yfried boris-42: so how can we get nova to wake up?
08:50 boris-42 yfried: I think with tempest
08:50 boris-42 yfried: with devstack***
08:50 boris-42 yfried: I think run locally rally and revert one by one patch
08:50 boris-42 okay I installed fresh devstack=)
08:53 psd boris-42, hey!
08:53 yfried boris-42: that seems a little tedious. good luck
08:53 boris-42 yfried: lol
08:54 yfried boris-42: can I ask you a question while you're waiting for installation?
08:54 boris-42 yfried: actually I won't to get rally at least on experiment check
08:54 boris-42 yfried: ?
08:54 boris-42 yfried: question?
08:54 yfried boris-42: I'm working on the tempest integration from last week
08:55 boris-42 yfried: that sounds great
08:55 boris-42 psd: hey hey
08:55 yfried boris-42: I wonder if we can introduce GitPython to rally and use it instead of all the "subprocess" calls
08:55 boris-42 yfried: not sure about that
08:55 boris-42 yfried: is it in global requriments?
08:56 yfried it's not a requirement currently
08:56 boris-42 yfried: so you'll need to get it before
08:56 boris-42 yfried: cause othewise our gate will fail
08:56 boris-42 yfried: you can test=)
08:56 yfried boris-42: we can add it to "requirements.txt" can't we?
08:56 boris-42 yfried: I mean is it in global requirments?
08:57 boris-42 yfried: not rally requiremnts?
08:57 boris-42 yfried: it should be here https://github.com/openstack/requirements/blob/master/global-requirements.txt
08:57 boris-42 yfried: otherwise if you try to add it to rally gates they will fail
08:58 yfried boris-42: why? won't the rally-gate install it if it's in rally/requirements.txt?
08:58 exploreshaifali joined #openstack-rally
08:58 psd boris-42, whatsup?
08:59 boris-42 yfried: if you try to put anything that is not in global-requriments
08:59 boris-42 yfried: rally gates will fail
08:59 boris-42 yfried: cause rally is aligned with other openstack projects
08:59 yfried boris-42: so we can't use other packages?
08:59 boris-42 yfried: and there is special test that checks that requiremnts.txt is sycnded
08:59 boris-42 yfried: you can use everything
08:59 boris-42 https://github.com/openstack/requirements/blob/master/global-requirements.txt
09:00 boris-42 ^ from this file
09:00 boris-42 or if you would like to use anything new you need to put it first there
09:02 boris-42 yfried: make a patch and try to add just requiremnt to rally
09:02 yfried boris-42: I see. it will be -1ed by the gate
09:02 yfried boris-42: so if we want to use stuff that's not in global requirements?
09:03 boris-42 yfried: no way
09:03 boris-42 yfried: except adding them to global requriments
09:04 yfried boris-42: have you tried to add stuff to g-r?
09:04 boris-42 yfried: yep
09:04 boris-42 yfried: I added osprofiler to it
09:04 yfried boris-42: and what are the odds they will accept GItPython ?
09:04 boris-42 yfried: nobody knows
09:05 boris-42 yfried: mostly they will say
09:05 boris-42 yfried: nobody is using it
09:05 yfried boris-42: or - can we bypass it with "install_rally.sh"
09:05 boris-42 yfried: and rally is not a core project
09:05 boris-42 yfried: nope
09:05 boris-42 yfried:  I won't do that
09:05 yfried boris-42: ok, so back to subprocess hacks? or do you have a better idea
09:06 boris-42 yfried: cause  GitPython may be not aligned with depencies of OpenStack
09:06 boris-42 yfried: I am not sure what was you going to hack?
09:07 boris-42 yfried: I think you talk with andreykurilin and for some reason I didn't read that conversation..
09:07 boris-42 yfried: could you say in few words what was the issue?
09:07 yfried boris-42: I need to check the source of the gitrepo installed in TEMPEST_BASE
09:08 yfried boris-42: and if it's not matching the desired source, clone to a new dir
09:09 yfried boris-42: so it's not longer a matter of "call"
09:09 yfried boris-42: I have to use Popen to catch and evaluate the command output
09:10 yfried boris-42: currently you allways clone from github
09:11 yfried boris-42: I've added a patch to allow cloning from different sources (ie - redhat's tempest repo or your own local repo)
09:12 yfried boris-42: problem is. the next run of "verify tempest install" will need to check if the desired source is already cloned locally
09:12 yfried boris-42: another way is to add a DB entry for each local clone, that would also hold the source
09:13 yfried boris-42: and check the db instead of the git api for each local clone
09:14 yfried boris-42: but that's a more complex change and I'm not familiar with how to work with the db
09:14 yfried boris-42: I don't think the tempest repos have a db entry
09:14 boris-42 yfried: one moment please
09:19 openstackgerrit Merged stackforge/rally: Replace ' with " in rally/*.py  https://review.openstack.org/148000
09:19 boris-42 ^ yfried hm strange
09:20 boris-42 yfried: but we have the almost similiar issue
09:20 boris-42 yfried: with current workflow
09:20 boris-42 yfried: like we are testing is tempest installed (and if it is not we are installing new one)
09:21 boris-42 yfried: so you can just call via subprocess git and check ouptut
09:21 boris-42 yfried: i don't see anything impossible here...
09:22 yfried I now need "git --get remote.origin.url"
09:22 yfried boris-42: catch the output and compare it to self.source_repo
09:22 yfried boris-42: it's not impossible
09:23 yfried boris-42: it's just much (much!!!) nicer with GitPyhon
09:23 boris-42 yfried:  I understand but as a PTL I will get in big troubles
09:23 boris-42 yfried: like Rally won't be able to be under openstack/*
09:23 yfried self.source_repo == dir.git.config("--get", "remote.origin.url")
09:24 yfried boris-42: I see
09:24 boris-42 yfried: and that is much more complicated=) then to write a bit ugly popen code=) or try to merge GitPython to global requiremnts
09:24 yfried boris-42: I'm unfamiliar with the whole project-integration process
09:24 boris-42 yfried: it's bullshit politic and haotic
09:24 boris-42 =)
09:25 yfried boris-42: that's why I asked you
09:25 boris-42 yfried:  we where rejected previous time
09:25 boris-42 yfried: because we are nice project but there is no collabaration between tempest and us
09:26 boris-42 that repeated ex PTL of QA Sean Dague billions times
09:26 boris-42 as well he said that Rally is duplicating effort of Tempest
09:26 yfried boris-42: ok, how do you solve this?
09:26 boris-42 yfried: reforms in what means being in OpenStack
09:27 boris-42 recently rules changed => so now duplicating effort is ok
09:27 yfried boris-42: btw, there's some sense in trying to collaborate with tempest so you don't duplicate eachother's work
09:27 boris-42 yfried: as you saw
09:27 boris-42 yfried: we integrated tempest ot rally
09:27 boris-42 yfried: as well making compare of runs in tempest
09:27 boris-42 yfried: automate installation and so on
09:28 boris-42 yfried: as well we have Tempest benchmark
09:28 boris-42 yfried: but it's called from QA side that we are not collabarting
09:28 boris-42 collabaration in their opinion, is get from rally load generators and nice graphs
09:28 boris-42 and put them to tempest
09:28 boris-42 and get rid of Rally
09:29 boris-42 I disliked such kind of collabartion
09:31 yfried boris-42: I hear you :)
09:31 boris-42 yfried: =)
09:31 boris-42 yfried: so we are continue to improve this tempest stuff (but without qa guys)
09:31 boris-42 yfried: as well we are trying to get spec in tempest
09:32 boris-42 yfried: https://review.openstack.org/#/c/94473/
09:32 boris-42 =)
09:32 boris-42 I think 1 more year and we will get it in=)
09:36 boris-42 yfried: btw
09:36 boris-42 yfried: could you ask dkranz to help us to port his script to rally
09:36 boris-42 yfried: at least until we get that spec merged we can keep it in rally
09:37 yfried boris-42: It's part of our todo
09:37 boris-42 yfried: nice
09:37 yfried boris-42: my team leader was supposed to contact you for a better collaboration plan
09:37 boris-42 yfried: he has some tech troubles=)
09:37 boris-42 yfried: I hope he will fix them soon
09:38 yfried boris-42: he's been overrun by work, vacation and now the flu
09:38 psd joined #openstack-rally
09:38 yfried boris-42: I hope he'll be able to introduce himself this week
09:40 boris-42 yfried: we can make a huge meeting (i can introduce my team )
09:40 boris-42 =)
09:41 boris-42 yfried: sooo
09:41 boris-42 yfried: let's try got get GitPython in global requriments
09:41 boris-42 yfried: could you make a patch to requirments?
09:42 e0ne joined #openstack-rally
09:44 boris-42 yfried: but I think it will be simpler to make just a bit more complicated code via subprocess
09:46 yfried I'm using subprocess ATM
09:47 yfried boris-42: I'll try to push gitpython later unrelated
09:55 psd boris-42, got some task for me?
09:59 rook__ joined #openstack-rally
10:06 boris-42 psd: actually I have
10:06 boris-42 psd: if you do next steps
10:06 boris-42 ./install_rally.sh
10:07 boris-42 rally deployment create ....
10:07 boris-42 ./install_rally.sh
10:07 boris-42 rally task start
10:07 boris-42 you'll get misleading error
10:07 boris-42 https://www.irccloud.com/pastebin/cOGNOi89
10:08 boris-42 ^ psd so could you work on it to say something about missing deployment
10:08 psd boris-42, Okay Thanks
10:13 boris-42 yfried: seems like we will need to move that NovaServers.boot_and_delete[1] scenario to unstable=(
10:13 yfried boris-42: I looked at it, you have 3 boot scenarios there
10:14 yfried boris-42: #2 is with network, right?
10:15 boris-42 yfried: yep yep
10:15 boris-42 yfried: that one
10:16 boris-42 yfried: sometimes it just produce races http://logs.openstack.org/78/148078/5/check/gate-rally-dsvm-rally/6e83b9d/rally-plot/results.html.gz#/NovaServers.boot_and_delete_server-2/overview
10:17 boris-42 yfried: btw amaretsky said that he finished his patch https://review.openstack.org/#/c/144337/
10:22 psd_ joined #openstack-rally
10:50 Miouge joined #openstack-rally
11:00 rook__ joined #openstack-rally
11:11 psd boris-42, Regarding https://review.openstack.org/#/c/147962
11:11 psd boris-42, so you want me to keep the function definition?
11:12 psd as in without body?
11:35 yfried boris-42: could be that our current network handling isn't too good?
11:38 yfried boris-42: re https://review.openstack.org/#/c/144337/ - needs a manual rebase. can't review it until it gets one
11:56 openstackgerrit Prabhjyot Singh Sodhi proposed stackforge/rally: Removing "failed" column entirely from rally cli as well as db  https://review.openstack.org/147962
11:59 e0ne joined #openstack-rally
12:28 Miouge joined #openstack-rally
12:49 rook__ joined #openstack-rally
13:50 rook__ joined #openstack-rally
14:09 e0ne joined #openstack-rally
14:53 openstackgerrit Mikhail Dubov proposed stackforge/rally: Restructure Rally docs  https://review.openstack.org/120018
15:27 yfried joined #openstack-rally
16:42 e0ne joined #openstack-rally
17:09 yfried joined #openstack-rally
17:24 Miouge joined #openstack-rally
17:41 exploreshaifali joined #openstack-rally
19:18 andreykurilin_ joined #openstack-rally
19:34 boris-42 psd: let me review it better + comment
19:35 boris-42 psd: sorry today is workout day=)
19:35 psd boris-42, no worries :)
20:39 e0ne joined #openstack-rally
20:40 nmagnezi joined #openstack-rally
21:51 e0ne joined #openstack-rally

| Channels | #openstack-rally index | Today | | Search | Google Search | Plain-Text | summary