Perl 6 - the future is here, just unevenly distributed

IRC log for #openstack-rally, 2015-03-05

| Channels | #openstack-rally index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:15 openstackgerrit Boris Pavlovic proposed stackforge/rally: Proposal for new task input file format  https://review.openstack.org/159065
00:22 openstackgerrit Boris Pavlovic proposed stackforge/rally: Future improvment of plugin base  https://review.openstack.org/160142
00:22 openstackgerrit Boris Pavlovic proposed stackforge/rally: Switch to plugin base: context, sla, runners  https://review.openstack.org/150647
00:26 dmorita joined #openstack-rally
00:26 bhunter71 joined #openstack-rally
00:29 boris-42 meteorfox: I update new input format ^
00:30 boris-42 meteorfox: you can take a look
00:31 meteorfox Sure will do
01:10 echoingumesh joined #openstack-rally
01:26 rook joined #openstack-rally
02:19 aarefiev_ joined #openstack-rally
03:35 yingjun joined #openstack-rally
03:50 yingjun joined #openstack-rally
04:44 nkhare joined #openstack-rally
05:20 zerda joined #openstack-rally
05:29 tfreger joined #openstack-rally
05:39 rdas joined #openstack-rally
05:40 yingjun joined #openstack-rally
05:40 k4n0 joined #openstack-rally
05:41 yingjun joined #openstack-rally
05:49 neeti joined #openstack-rally
06:15 nkhare joined #openstack-rally
06:36 e0ne joined #openstack-rally
06:42 e0ne joined #openstack-rally
07:07 exploreshaifali joined #openstack-rally
07:13 yingjun joined #openstack-rally
07:26 anshul joined #openstack-rally
07:26 yingjun joined #openstack-rally
07:27 nkhare joined #openstack-rally
07:31 yingjun joined #openstack-rally
07:36 openstackgerrit joined #openstack-rally
07:45 exploreshaifali joined #openstack-rally
07:46 nmagnezi joined #openstack-rally
07:47 amaretskiy joined #openstack-rally
07:48 krishna13 joined #openstack-rally
07:56 openstackgerrit Olga Kopylova proposed stackforge/rally: Implementation of 'rally task abort' command  https://review.openstack.org/161636
08:00 openstackgerrit Roman Vasilets proposed stackforge/rally: Add murano benchmarks  https://review.openstack.org/137661
08:00 nkhare joined #openstack-rally
08:04 k4n0 left #openstack-rally
08:06 nkhare joined #openstack-rally
08:09 arxcruz joined #openstack-rally
08:18 nkhare_ joined #openstack-rally
08:22 yingjun joined #openstack-rally
08:28 nmagnezi joined #openstack-rally
08:40 openstackgerrit Olga Kopylova proposed stackforge/rally: Mark 'rally use {deployment|task|verification}' deprecated  https://review.openstack.org/159192
08:58 krishna13 joined #openstack-rally
08:58 krishna13 Hi
08:59 amaretskiy hi
09:02 psd joined #openstack-rally
09:10 oanufriev joined #openstack-rally
09:15 openstackgerrit Nikita Konovalov proposed stackforge/rally: [Sahara] Fix Job Binaries download  https://review.openstack.org/161309
09:26 psd_ joined #openstack-rally
10:15 yingjun joined #openstack-rally
10:15 yingjun joined #openstack-rally
10:18 psd__ joined #openstack-rally
10:24 krishna13 joined #openstack-rally
10:26 nkhare_ joined #openstack-rally
10:26 e0ne joined #openstack-rally
10:27 e0ne joined #openstack-rally
10:35 yingjun joined #openstack-rally
10:42 openstackgerrit Kairat Kushaev proposed stackforge/rally: Add security group for neutron ports  https://review.openstack.org/161669
10:47 openstackgerrit Nikita Konovalov proposed stackforge/rally: [Sahara] cleanup clusters correctly  https://review.openstack.org/161293
10:47 openstackgerrit Nikita Konovalov proposed stackforge/rally: [Sahara] Fix Job Binaries download  https://review.openstack.org/161309
10:49 openstackgerrit Roman Vasilets proposed stackforge/rally: Adding Multiple Request Scenario  https://review.openstack.org/117705
10:52 zerda joined #openstack-rally
10:55 nkhare_ joined #openstack-rally
10:55 aix joined #openstack-rally
10:58 psd__ joined #openstack-rally
11:00 openstackgerrit Pavel Boldin proposed stackforge/rally: Add the generic script-based benchmark for the VMs  https://review.openstack.org/141671
11:11 krishna13 joined #openstack-rally
11:14 e0ne joined #openstack-rally
11:16 yingjun joined #openstack-rally
11:18 openstackgerrit Olga Kopylova proposed stackforge/rally: Mark 'rally use {deployment|task|verification}' deprecated  https://review.openstack.org/159192
11:53 openstackgerrit Roman Vasilets proposed stackforge/rally: Add murano benchmarks  https://review.openstack.org/137661
11:55 cdent joined #openstack-rally
11:55 kairat_kushaev amaretskiy: Hello!
11:56 kairat_kushaev amaretskiy: Could you help me with question: why string is more preferable than object here
11:56 kairat_kushaev amaretskiy: https://review.openstack.org/#/c/155346/16/tests/unit/benchmark/scenarios/heat/test_stacks.py
11:57 psd__ joined #openstack-rally
12:11 amaretskiy kairat_kushaev hi
12:11 kairat_kushaev kairat_kushaev: Hi
12:12 amaretskiy kairat_kushaev: 1) object() in thi scase does nothing - it does not have human readable name, it is not writable, etc...
12:13 amaretskiy generally we do not use object() in Python, we usually inherit classes from it
12:13 amaretskiy so, in this particular case
12:13 amaretskiy you just place some python instance
12:13 amaretskiy as a marker
12:14 amaretskiy to make further checks (return value, etc)
12:14 amaretskiy so in this case you do not need Mock, so basically use string
12:14 amaretskiy because string is something that is easy to read in code
12:15 amaretskiy for example, you can make self.assertEqual(some_result, "foo_stack")
12:16 amaretskiy in this case "foo_stack" is easy to understand, and this does not require extra variable
12:16 amaretskiy I hope that is clear now :)
12:17 amaretskiy just a small example
12:17 amaretskiy take a look at some tests
12:18 amaretskiy where there are a lot of passed values into some methods or functions
12:18 amaretskiy if use extra variable for each argument - this will be a huge code
12:20 amaretskiy https://github.com/stackforge/rally/blob/master/tests/unit/benchmark/scenarios/vm/test_vmtasks.py#L42-L47
12:20 amaretskiy look at this ^^^
12:20 amaretskiy :)
12:21 msdubov_ joined #openstack-rally
12:24 kairat_kushaev amaretskiy: Thanks for explanation!
12:24 amaretskiy np  :)
12:26 krishna13 left #openstack-rally
12:30 openstackgerrit Andrey Kurilin proposed stackforge/rally: Updated from global requirements  https://review.openstack.org/161596
12:38 openstackgerrit Andrey Kurilin proposed stackforge/rally: Updated from global requirements  https://review.openstack.org/161596
12:46 coolsvap_ joined #openstack-rally
12:48 pboros joined #openstack-rally
12:48 openstackgerrit Alexander Maretskiy proposed stackforge/rally: Restore fixed IP support in boot_runcommand_delete  https://review.openstack.org/161699
12:49 psd_ joined #openstack-rally
12:52 openstackgerrit Andrey Kurilin proposed stackforge/rally: Updated from global requirements  https://review.openstack.org/161596
13:01 mwagner_lap joined #openstack-rally
13:02 dmellado joined #openstack-rally
13:03 openstackgerrit Kairat Kushaev proposed stackforge/rally: Add security group for neutron ports  https://review.openstack.org/161669
13:11 openstackgerrit Kairat Kushaev proposed stackforge/rally: Add create-check-delete stack scenario  https://review.openstack.org/155685
13:11 openstackgerrit Kairat Kushaev proposed stackforge/rally: Add create-suspend-resume-delete stack scenario  https://review.openstack.org/155346
13:21 exploreshaifali joined #openstack-rally
13:24 boris-42 yfried: ping
13:24 boris-42 yfried: https://review.openstack.org/#/c/159065/ I updated input task fromat proposal
13:25 boris-42 yfried: could you take a look?
13:29 e0ne joined #openstack-rally
13:33 psd_ joined #openstack-rally
13:34 openstackgerrit Olga Kopylova proposed stackforge/rally: Mark 'rally use {deployment|task|verification}' deprecated  https://review.openstack.org/159192
13:40 dmellado_ joined #openstack-rally
13:47 openstackgerrit Alexander Maretskiy proposed stackforge/rally: Restore fixed IP support in boot_runcommand_delete  https://review.openstack.org/161699
13:49 openstackgerrit Andrey Kurilin proposed stackforge/rally: Add Python 3.4 as supported interpreters  https://review.openstack.org/158743
13:58 psd__ joined #openstack-rally
13:58 openstackgerrit Andrey Kurilin proposed stackforge/rally: Updated from global requirements  https://review.openstack.org/161596
14:01 aix joined #openstack-rally
14:11 openstackgerrit Mikhail Dubov proposed stackforge/rally: Unify RPS and constant runners and tune constant runner  https://review.openstack.org/155225
14:12 openstackgerrit Andrey Kurilin proposed stackforge/rally: Add Python 3.4 as supported interpreters  https://review.openstack.org/158743
14:16 openstackgerrit Nikita Konovalov proposed stackforge/rally: [Sahara] Suppurt batch job execution  https://review.openstack.org/161188
14:52 psd_ joined #openstack-rally
15:00 openstackgerrit Andrey Kurilin proposed stackforge/rally: Add Python 3.4 as supported interpreters  https://review.openstack.org/158743
15:12 exploreshaifali joined #openstack-rally
15:23 psd__ joined #openstack-rally
15:27 openstackgerrit Marian Krcmarik proposed stackforge/rally: Add scenarios for live migration of an instance with volumes  https://review.openstack.org/157844
15:49 jjmb joined #openstack-rally
15:53 openstackgerrit Andrey Kurilin proposed stackforge/rally: Add Python 3.4 as supported interpreters  https://review.openstack.org/158743
15:58 openstackgerrit Andrey Kurilin proposed stackforge/rally: Updated from global requirements  https://review.openstack.org/161596
16:00 peng_li_ joined #openstack-rally
16:00 morgan_orange joined #openstack-rally
16:23 psd_ joined #openstack-rally
16:23 openstackgerrit Merged stackforge/rally: Add security group for neutron ports  https://review.openstack.org/161669
16:35 aarefiev joined #openstack-rally
16:38 jjmb joined #openstack-rally
16:39 openstackgerrit Merged stackforge/rally: Mark 'rally use {deployment|task|verification}' deprecated  https://review.openstack.org/159192
17:02 strummer_ie joined #openstack-rally
17:09 jaypipes joined #openstack-rally
17:14 echoingumesh joined #openstack-rally
17:18 openstackgerrit Andrey Kurilin proposed stackforge/rally: Add Python 3.4 as supported interpreters  https://review.openstack.org/158743
17:23 yfried boris-42: I didn't work today (holiday) will take a look during the weekend
17:27 openstackgerrit Andrey Kurilin proposed stackforge/rally: Add Python 3.4 as supported interpreters  https://review.openstack.org/158743
17:28 openstackgerrit Andrey Kurilin proposed stackforge/rally: Add Python 3.4 as supported interpreters  https://review.openstack.org/158743
17:29 openstackgerrit Andrey Kurilin proposed stackforge/rally: Add Python 3.4 as supported interpreters  https://review.openstack.org/158743
17:35 psd__ joined #openstack-rally
17:39 meteorfox andreykurilin: it seems I reviewed to early, didn't know it was still WIP.
17:39 openstackgerrit OpenStack Proposal Bot proposed stackforge/rally: Updated from global requirements  https://review.openstack.org/161802
17:41 andreykurilin meteorfox: thanks for review. It is not WIP:(
17:42 openstackgerrit Andrey Kurilin proposed stackforge/rally: Add Python 3.4 as supported interpreters  https://review.openstack.org/158743
17:43 andreykurilin meteorfox: fixed your comments. now it should work
17:43 meteorfox andreykurilin: ah I see. Sure np, I have py3.4 setup with tox, and I'm on linux, so I can try locally your patches.
17:43 meteorfox andreykurilin: awesome
17:44 meteorfox andreykurilin: yup, it passed on my laptop at least
17:45 andreykurilin meteorfox: patch-set #9 was tested on py34, but some guys said to me that "sorted" sucks, since it creates new object and I tried ".sort" without testing process:)
17:50 jlk joined #openstack-rally
18:01 openstackgerrit OpenStack Proposal Bot proposed stackforge/rally: Updated from global requirements  https://review.openstack.org/161802
18:01 meteorfox andreykurilin: well besides the point that another way to fix that would've been to do list(atomic_actions).sort(), that would've also created another object. The builtin sorted, even though it will also create another list (or object), it does it in C, including the sort, which I believe is faster that the two step approach of list() and then sort(). Here's
18:01 meteorfox the code for sorted() https://github.com/python/cpython/blob/master/Python/bltinmodule.c#L2619
18:04 meteorfox andreykurilin: either way, atomic_actions will probably be at most a few dozen keys, I think it's irrelevant. I would've been worried if atomic_actions was in the order of millions
18:24 echoingumesh joined #openstack-rally
18:28 meteorfox boris-42: hey, regarding my patch for the median. Why do you 'recheck'? Is there a race condition or something that causes it to fail?
18:29 echoingumesh joined #openstack-rally
18:57 psd_ joined #openstack-rally
18:57 echoingumesh joined #openstack-rally
18:59 meteorfox boris-42: also, jenkins reports it as a failure, but it seems the failures are unrelated to the changeset.
19:10 Vishal_ joined #openstack-rally
19:15 echoingumesh joined #openstack-rally
19:17 Vishal_ Hi All...I have one query..In rally for neutron create_and_list_router.json is there a way to generate an overlapping cidr for every network
19:17 Vishal_ subnet ?
19:21 exploreshaifali joined #openstack-rally
19:24 e0ne joined #openstack-rally
19:45 psd_ joined #openstack-rally
19:54 openstackgerrit Andrey Kurilin proposed stackforge/rally: Add Python 3.4 as supported interpreters  https://review.openstack.org/158743
20:26 openstackgerrit joined #openstack-rally
20:27 rook joined #openstack-rally
20:28 rook boris-42: ping
21:06 psd joined #openstack-rally
21:19 boris-42 joined #openstack-rally
21:30 psd_ joined #openstack-rally
21:36 meteorfox boris-42: I saw your comment on Andrey's patch. So, use an OrderedDict instead?
21:37 boris-42 meteorfox: gimme a moment
21:39 rook boris-42: question, doing a boot storm with Rally, i am seeing some weird things where Rally is requesting information on a nova id that doesn't exist
21:40 boris-42 rook: warnings durring cleanup?
21:41 boris-42 rook: show me the log
21:43 rook https://gist.github.com/jtaleric/dfede38fb32f561ae79f
21:43 rook example boris-42 ^
21:45 boris-42 rook:  Max retries exceeded with url:
21:45 boris-42 rook: this means API rate limits are too small
21:46 rook that is the first time i have ever heard that.
21:46 rook so, somethig with the nova api config?
21:50 meteorfox rook: I think with nova cli, you can do  $ nova rate-limits
21:51 meteorfox it should show what are your limits
21:51 rook i have none
21:51 rook returned a empty set
21:52 meteorfox rook: so they are probably globally set.
21:53 meteorfox rook: here are the defaults: http://docs.openstack.org/trunk/config-reference/content/configuring-compute-API.html
21:53 meteorfox boris-42: can these rate-limits be set with Rally? Like the same way we set quotas?
21:54 rook " (BoolOpt) Whether to use per-user rate limiting for the api. This option is only used by v2 api. Rate limiting is removed from v3 api."
21:54 rook so it might be.
21:58 meteorfox rook: what's your runner configuration?
22:02 boris-42 meteorfox: rook they can't be
22:02 boris-42 meteorfox: rook it's deployment configuration
22:02 boris-42 meteorfox: rook as far as I know it requires rebooting services
22:04 meteorfox boris-42: are we complete sure it's being rate limited? Should that return an HTTP status code of 429, or 413 or something like that. There error only says ConnectionError
22:04 meteorfox boris-42: and the BadStatusLine is empty
22:06 psd__ joined #openstack-rally
22:09 boris-42 meteorfox: it says this "Max retries exceeded with url:"
22:09 boris-42 meteorfox: I got the same error when I had api rate limits
22:13 meteorfox boris-42: right, but that doesn't necessarily means you are rate-limited. Max retries exceeded, is that you tried N times, and then gave up. The httplib under requests, I believe, takes an argument called 'retries', that specifies how many times you want to retry, if the http request fails
22:13 meteorfox boris-42: it could fail for many reasons, including rate-limits
22:14 boris-42 meteorfox: agree but for some reason I think this one is relate to ratelimit
22:15 meteorfox boris-42: yeah, I bet you are right. You've seen more Rally errors than me :)
22:16 meteorfox boris-42: I want to improve this. Can we open a bug or something to improve the error message?
22:16 boris-42 meteorfox: I don't know how to improve it
22:16 boris-42 meteorfox: it's just exception that we caught..
22:17 meteorfox boris-42: yeah, I think the error message is somewhere, we just need to bubble it up all the way
22:17 meteorfox boris-42: let me take a look to see if I can improve it somehow
22:18 boris-42 meteorfox: so rally captures whole exception trace
22:18 boris-42 meteorfox: it doesn't hide anything from you
22:18 boris-42 meteorfox: you won't get more info until you add it to python nova client
22:27 rook I have no ratelimiting set.
22:27 rook it is set to false in my nova config.
22:27 rook config*
22:27 rook meteorfox: boris-42 ^
22:28 rook is there a easy way to increase the retries?
22:28 rook also, i have the tcpdump
22:28 rook i can look for 429 or 413.
22:28 boris-42 rook: hm strange
22:29 rook ^KÕ<89>wXÅe)HTTP/1.1 404 Not Found^M
22:29 rook Content-Length: 73^M
22:29 rook Content-Type: application/json; charset=UTF-8^M
22:29 rook X-Compute-Request-Id: req-cb63456a-c0f8-4645-acb1-d834a7b97362^M
22:29 rook Date: Thu, 05 Mar 2015 20:56:06 GMT^M
22:29 rook ^M
22:30 rook {"itemNotFound": {"message": "Instance could not be found", "code": 404}
22:30 rook i see a ton of these
22:30 rook no 413 or 429
22:31 boris-42 rook: hm rate limits are in api-paste ini as far as I know
22:31 rook i have none set
22:31 boris-42 rook: so you have to see 404 Not Found
22:31 boris-42 rook: it's normal
22:32 rook ok - but the /v2/key/servers issue with bad stauts
22:32 rook status**
22:32 boris-42 rook: because when we send request to delete VM, we are wating 404 not found
22:32 boris-42 rook: so it's okay to see 404
22:32 rook alright
22:32 rook but what rally bombs out on isn't so good.
22:32 boris-42 rook: but this stuff that you paste beofre is unclear
22:33 rook HTTPConnectionPool(host='20.0.0.39', port=8774): Max retries exceeded with url: /v2/c3bce63cba464ed09a87e15998d1157c/servers (Caused by <class 'httplib.BadStatusLine'>: ''
22:33 rook ^ that?
22:33 rook that is from Rally
22:34 boris-42 rook: it's not from Rally
22:35 boris-42 rook: it's from nova python client
22:35 rook well it is really from httplib
22:38 meteorfox boris-42: what's weird is that python-novaclient, raises a proper exception if indeed the request returned 429? http://docs.openstack.org/developer/python-novaclient/ref/exceptions.html  and here https://github.com/openstack/python-novaclient/blob/ac6636a54d72ba76d0adca76e07d1d26d9ea35c3/novaclient/client.py#L385-L386
22:38 psd_ joined #openstack-rally
22:40 meteorfox rook: did you filter your tcp dump for dest=20.0.0.39
22:40 meteorfox ?
22:40 rook yes
22:40 boris-42 meteorfox: not sure https://github.com/openstack/python-novaclient/blob/ac6636a54d72ba76d0adca76e07d1d26d9ea35c3/novaclient/exceptions.py#L217-L256
22:40 boris-42 meteorfox: let me analyze this code
22:42 rook boris-42: meteorfox if i wnated to change hte nova-api IP in a rally deployment, how could i do that
22:43 meteorfox boris-42: based on the message, and status code, it looks in a map, for the proper exception, that's how I understood it
22:45 boris-42 meteorfox: rook this may help http://stackoverflow.com/questions/23013220/max-retries-exceeded-with-url
22:46 boris-42 meteorfox: rook seems like server is kicking your request even before it goest to nova
22:46 rook is there a way to change the nova-api ip within a deployment?
22:48 boris-42 rook: I belive if you try to run one more time rally it will work perfectly fine
22:48 rook ive tried hundreds of times... no bueno
22:49 boris-42 rook: so it just doens't work ?
22:50 boris-42 rook: you need to change rally machine IP not nova IP machine as far as I understand
22:50 meteorfox rook: depends on how you are deploying openstack, but usually in the nova.conf there's property where you set in which interface to listen on, and that will 'change' your IP address. But I doubt that will fix the problem
22:51 meteorfox rook: what kind of deployment do you have? are there proxies in front of it?
22:51 rook Yes, HAProxy - which is why I want to change nova-api's ip
22:51 rook :)
22:51 rook point directly instead of at the Proxy IP
22:51 meteorfox rook: like boris-42 suggested these other services could be killing your require even before it hits the nova api
22:52 boris-42 rook: restart HAProxy
22:52 boris-42 rook: that may help lol
22:52 meteorfox rook: ah ok. look at where nova-api is listening on
22:53 rook yup
22:53 boris-42 rook: could you try to restart haproxy
22:54 meteorfox rook: if you are in linux, you could  try something like ' netstat -tulpn ' and grep for nova
22:55 meteorfox rook: if it's listening on 0.0.0.0, then you are set, all you have to do is hit the servers IP, at the right port
22:56 meteorfox rook: if it's something else and is not 127.0.0.1: then use that as the ip
22:56 meteorfox rook: if it's 127.0.0.1: then you'll have to modify nova.conf and set it to listen to a different net iface
22:57 jaypipes joined #openstack-rally
22:58 rook meteorfox hm i know the IP, however rally continues to hit the haproxy ip
22:58 rook instead of directly
22:59 meteorfox rook: did you update your keystone url in the deployment configuration for Rally
23:00 rook i need to chane the keystone endpoint url for nova-api
23:00 meteorfox rook: ooh, wait no
23:00 meteorfox brain fart
23:00 meteorfox it should pick it up from the catalog
23:00 rook right
23:01 boris-42 rook: yep rally will pick it from keystone catalog
23:01 rook yeah, i need to muck with some msyql
23:01 rook mysql**
23:01 boris-42 rook: mysql?)
23:02 rook to change the endpoint url
23:04 meteorfox rook: before you go in that direction? did you check if your HAprox is configured to rate-limit ?
23:04 rook not it isn't
23:05 rook maxconn  10000
23:05 rook is the only 'limit" i see
23:07 meteorfox rook: there's nothing like this "stick-table type ip size 1m expire 30s store gpc0,http_req_rate(10s),http_err_rate(10s) store conn_cur store conn_rate(3s)"
23:07 meteorfox ?
23:08 rook nope
23:08 meteorfox hmm
23:10 meteorfox rook: are you doing a large workload? Large enough that network saturates, and drops packets?
23:16 meteorfox rook: also, did you try running with the debug flag --debug and --verbose
23:16 meteorfox rook: I doubt that will show anything else, but just in case
23:20 albertom joined #openstack-rally
23:32 openstackgerrit Mikhail Dubov proposed stackforge/rally: Unify RPS and constant runners and tune constant runner  https://review.openstack.org/155225
23:48 boris-42 meteorfox: btw could you re-review new format?
23:48 boris-42 rook: as well could you make a review for new rally format?

| Channels | #openstack-rally index | Today | | Search | Google Search | Plain-Text | summary