Perl 6 - the future is here, just unevenly distributed

IRC log for #openstack-rally, 2015-04-22

| Channels | #openstack-rally index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:04 apal boris-42: ?
00:26 davideagnello joined #openstack-rally
00:37 yingjun joined #openstack-rally
00:50 rook_ joined #openstack-rally
01:26 openstackgerrit Boris Pavlovic proposed openstack/rally: Fix rally plugins relative imports  https://review.openstack.org/176153
01:29 vkmc joined #openstack-rally
02:52 rook_ joined #openstack-rally
02:53 davideagnello joined #openstack-rally
02:54 echoingumesh joined #openstack-rally
03:16 prashantS joined #openstack-rally
03:18 echoingu_ joined #openstack-rally
03:20 openstackgerrit Boris Pavlovic proposed openstack/rally: Fix rally plugins relative imports  https://review.openstack.org/176153
03:22 prashantS joined #openstack-rally
03:24 prashantS joined #openstack-rally
03:28 pradeep joined #openstack-rally
03:31 prashantS joined #openstack-rally
03:31 prashantS joined #openstack-rally
03:33 nkhare joined #openstack-rally
03:41 prashantS joined #openstack-rally
03:46 prashantS joined #openstack-rally
03:57 echoingumesh joined #openstack-rally
03:57 rook_ joined #openstack-rally
04:00 prashantS joined #openstack-rally
04:38 panbalag left #openstack-rally
04:43 tfreger joined #openstack-rally
04:50 kiran-r joined #openstack-rally
05:12 prashantS joined #openstack-rally
05:17 neeti joined #openstack-rally
05:33 rdas joined #openstack-rally
05:40 prashantS joined #openstack-rally
05:41 echoingumesh joined #openstack-rally
05:46 rook_ joined #openstack-rally
05:48 rdas joined #openstack-rally
06:01 anshul joined #openstack-rally
06:18 echoingumesh joined #openstack-rally
06:22 openstackgerrit Mike Durnosvistov proposed openstack/rally: Server block migration scenario  https://review.openstack.org/175875
06:25 davideagnello joined #openstack-rally
06:57 ptoohill joined #openstack-rally
07:14 yingjun joined #openstack-rally
07:27 yingjun_ joined #openstack-rally
07:35 rook_ joined #openstack-rally
07:35 Miouge_ joined #openstack-rally
07:38 karmatronic joined #openstack-rally
07:41 arxcruz joined #openstack-rally
07:44 andreykurilin__ joined #openstack-rally
07:52 karmatronic joined #openstack-rally
07:58 pbandzi joined #openstack-rally
08:02 yingjun joined #openstack-rally
08:18 echoingumesh joined #openstack-rally
08:26 yfried|afk joined #openstack-rally
08:26 yingjun joined #openstack-rally
08:32 rook joined #openstack-rally
08:32 echoingumesh joined #openstack-rally
08:48 yfried|afk arxcruz: re https://review.openstack.org/#/c/146838/7/rally/benchmark/functional.py,cm
08:50 andreykurilin__ joined #openstack-rally
09:00 openstackgerrit Boris Pavlovic proposed openstack/rally: Fix rally plugins relative imports  https://review.openstack.org/176153
09:02 yingjun joined #openstack-rally
09:06 arxcruz yfried|afk, yeah, I think it would be better to make msg = default_msg + custom_msg
09:06 arxcruz yfried|afk, what do you think, all in one single line or default_msg in one line, and the custom message in another line?
09:07 yfried|afk arxcruz: could you plz explain why you had to implement the assertions at all?
09:07 arxcruz yfried|afk, i think boris-42 can gave you a better explanation since he was the one who request it :D
09:07 arxcruz I know it will be part of the functional framework, and we can't use assertX without inherit from TestCase
09:08 yfried|afk arxcruz: did he sepcifically ask you to WRITE code that already exist somewhere else
09:08 arxcruz yfried|afk, yes
09:08 arxcruz :)
09:08 arxcruz to be part of rally, so rally won't have external dependencies
09:08 yfried|afk arxcruz: then this definatly needs to be documented in commit message (if not in code)
09:09 yfried|afk arxcruz: well, then you could implement SSH and drop paramiko...
09:09 yfried|afk arxcruz: I'm not saying you don't have a good reason for that. I'm just saying the the reason should be well documented
09:10 arxcruz yfried|afk, sure, I will write down in the commit message don't worry :)
09:10 yfried|afk arxcruz: could you please look into that?
09:10 yfried|afk arxcruz: tnx
09:11 yfried|afk arxcruz: or maybe put it in release notes. if we are dropping a requirement...
09:11 echoingumesh joined #openstack-rally
09:11 yfried|afk arxcruz: ^ that's msdubov area
09:14 arxcruz yfried|afk, https://docs.google.com/spreadsheets/d/16DXpfbqvlzMFaqaXAcJsBzzpowb_XpymaK2aFY2gA2g/edit#gid=0
09:19 echoingumesh joined #openstack-rally
09:21 rook_ joined #openstack-rally
09:30 e0ne joined #openstack-rally
09:30 yfried|afk arxcruz: btw, another question
09:31 yfried|afk arxcruz: take assertEqual
09:31 yfried|afk arxcruz: you are doing if not a==b: raise RallyAssertionError
09:32 yfried|afk arxcruz: why not do "assert a==b" and get builtin assertion error?
09:34 arxcruz yfried|afk, https://docs.python.org/2/reference/simple_stmts.html#the-assert-statement
09:34 yfried|afk boris-42: andreykurilin__: any idea why this https://review.openstack.org/#/c/172831/ is unavailable?
09:35 arxcruz The current code generator emits no code for an assert statement when optimization is requested at compile time
09:35 arxcruz asserts are removed when you use optimization
09:35 arxcruz :)
09:37 yfried|afk well, will the testing use optimization? seems reasonable to assume they won't. but whatever - you get the same result with roughly the same amount of code, right?
09:38 arxcruz yfried|afk, I don't think use assert is a good practice
09:38 arxcruz even the TestCase assertXYZ code doesn't use it
09:39 yfried|afk arxcruz: really? I was sure the did. I remember seeing it once. I might be wrong
09:39 arxcruz yfried|afk, I check the source code recently :)
09:40 yfried|afk arxcruz: I trust you :)
09:42 anshul joined #openstack-rally
09:44 arxcruz yfried|afk, :)
09:49 tosky joined #openstack-rally
10:00 aix joined #openstack-rally
10:15 anshul joined #openstack-rally
10:50 anshul joined #openstack-rally
10:51 anshul joined #openstack-rally
11:07 nkhare joined #openstack-rally
11:09 rook_ joined #openstack-rally
11:15 anshul joined #openstack-rally
11:16 rdas joined #openstack-rally
11:30 pradeep joined #openstack-rally
11:31 boris-42 yfried__: ?
11:31 boris-42 yfried__: no asserts in production code
11:31 boris-42 yfried__: Rally is production code
11:32 boris-42 yfried__: it's okay to run it with -o
11:32 boris-42 yfried__: and I dislike idea of using unittest for anything except of unittests
11:32 boris-42 yfried__: and one more thing is that we can't extend default unittest exceptions
11:32 boris-42 yfried__: which will create issues
11:33 openstackgerrit Merged openstack/rally: Fix rally plugins relative imports  https://review.openstack.org/176153
11:34 boris-42 arxcruz: this is explanations ^
11:34 boris-42 arxcruz: of why we shouldn't use unittest framework parts
11:36 arxcruz yfried__, ^
11:38 openstackgerrit Sergey Skripnick proposed openstack/rally: [DO NOT MERGE] Testing coverage job  https://review.openstack.org/175558
11:38 openstackgerrit Sergey Skripnick proposed openstack/rally: Improve coverage job  https://review.openstack.org/175841
11:38 openstackgerrit Sergey Skripnick proposed openstack/rally: Fix creating user in keystone v3 wrapper  https://review.openstack.org/175504
11:40 rook_ joined #openstack-rally
11:41 softCloud joined #openstack-rally
11:57 e0ne joined #openstack-rally
11:58 aix joined #openstack-rally
11:59 msdubov_ joined #openstack-rally
11:59 openstackgerrit svasheka proposed openstack/rally: Add keystone update_user_password scenario  https://review.openstack.org/165422
12:02 openstackgerrit Mike Durnosvistov proposed openstack/rally: Add sample for the server block migration scenario  https://review.openstack.org/175875
12:07 openstackgerrit svasheka proposed openstack/rally: Replace _delete_service method with _resource_delete  https://review.openstack.org/176267
12:09 openstackgerrit Nikita Konovalov proposed openstack/rally: Wait for status  https://review.openstack.org/172466
12:11 yingjun joined #openstack-rally
12:15 jaypipes joined #openstack-rally
12:16 yingjun joined #openstack-rally
12:16 redixin joined #openstack-rally
12:18 cdent joined #openstack-rally
12:22 rook_ joined #openstack-rally
12:24 openstackgerrit Arx Cruz proposed openstack/rally: Adding FunctionalMixin class  https://review.openstack.org/146838
12:38 anshul joined #openstack-rally
13:06 pbandzi joined #openstack-rally
13:24 Miouge joined #openstack-rally
13:29 yfried__ boris-42: arxcruz: ok. please put this in commit msg, for the next one to ask the same thing
13:33 andreykurilin__ joined #openstack-rally
13:48 mwagner_lap joined #openstack-rally
13:59 openstackgerrit Nikita Konovalov proposed openstack/rally: [Sahara] Fix the config descriptions  https://review.openstack.org/176321
13:59 e0ne joined #openstack-rally
14:09 prashantS joined #openstack-rally
14:15 openstackgerrit Pavel Boldin proposed openstack/rally: Add function decorator `log_deprecated_args'  https://review.openstack.org/174453
14:15 openstackgerrit Pavel Boldin proposed openstack/rally: Refactor run_command_over_ssh, add script_args  https://review.openstack.org/173371
14:15 openstackgerrit Pavel Boldin proposed openstack/rally: Add `LogCatcher' context  https://review.openstack.org/174454
14:25 stpierre joined #openstack-rally
14:25 openstackgerrit Merged openstack/rally: Add sample for the server block migration scenario  https://review.openstack.org/175875
14:42 prashantS joined #openstack-rally
14:43 pradeep joined #openstack-rally
14:49 openstackgerrit Merged openstack/rally: Add keystone update_user_password scenario  https://review.openstack.org/165422
14:54 davideagnello joined #openstack-rally
15:17 davideagnello boris-42: is there a way to have scenario descriptions to show up in generated reports?
15:17 boris-42 davideagnello: not yet
15:18 boris-42 davideagnello: blocked by input task format
15:18 davideagnello boris-42: ok, something the new format would allow too
15:18 boris-42 davideagnello: it will allow bunch of stuff
15:19 davideagnello boris-42: yup, I have noticed :)
15:19 boris-42 davideagnello: I hope it will the last change in input format
15:19 boris-42 will be*
15:21 davideagnello boris-42: yeah, it gets more complicated each time.  looks like the new format is comprehensive to allow some critical functionality
15:27 boris-42 davideagnello: yep
15:27 boris-42 davideagnello: we didn't know about this functionallity year ago
15:27 boris-42 davideagnello: when we introduced the second format (current one)
15:27 boris-42 davideagnello: but it was amazing step forward=)
15:28 boris-42 davideagnello: because first format was really terrible=)
15:35 csoukup joined #openstack-rally
15:36 c_soukup joined #openstack-rally
15:37 csoukup joined #openstack-rally
15:37 e0ne joined #openstack-rally
15:38 openstackgerrit Sergey Skripnick proposed openstack/rally: Improve coverage job  https://review.openstack.org/175841
15:38 openstackgerrit Boris Pavlovic proposed openstack/rally: [DO NOT MERGE] Testing coverage job  https://review.openstack.org/175558
15:41 prashantS joined #openstack-rally
15:42 prashantS joined #openstack-rally
15:50 pbandzi joined #openstack-rally
16:24 coolsvap|afk joined #openstack-rally
16:30 tosky_ joined #openstack-rally
16:37 andreykurilin__ joined #openstack-rally
16:48 subscope_ joined #openstack-rally
17:07 subscope_ joined #openstack-rally
17:15 yfried|afk joined #openstack-rally
17:21 davideagnello joined #openstack-rally
17:32 prashantS joined #openstack-rally
17:45 davideagnello boris-42: when Rally runs a benchmark with a constant runner and concurrency greater than 1, it looks like it spawns a process for each concurrent scenario being executed?  these python processes run in parallel in order to simulate concurrent requests to your target cloud?
17:59 aix davideagnello, not really, basically in the reality the number of process is the number of available cpu cores
17:59 e0ne joined #openstack-rally
18:00 davideagnello ok, I noticed it takes a min between that times and concurrency
18:01 aix davideagnello, yes and concurrency is "divided" by number of processes and scenarios are being executed in threads
18:02 aix davideagnello, i.e you have 2 Cpu cores and 4 as concurrency, so 2 threads in 2 processes will be ran concurrently
18:03 davideagnello aix: ok makes sense.  so concurrency depends on the hardware you are running on top of
18:07 aix davideagnello, The number of concurrently ran scenarios is always the same on different hardware but the way how the concurrent scenarios are being started is different, maybe it could have performance impact, i.e. If you compare for example "starting 1000 concurrent scenarios on machine with 1 cpu core" vs. "starting 1000 concurrent scenarios on machine with 1000 CPU cores"
18:07 aix davideagnello, in the first case 1000 threads will be strated in one process, and in the second scenario one thread per each process will be started..
18:08 aix davideagnello, so 1000 processes with one thread (add 2. case)
18:09 davideagnello aix: ok, understood.  thank you for the details, very helpful :)
18:09 echoingumesh joined #openstack-rally
18:09 aix davideagnello, yw
18:13 boris-42 aix: thanks for helping davideagnello =)
18:17 prashantS joined #openstack-rally
18:18 softCloud1 joined #openstack-rally
18:50 openstackgerrit Merged openstack/rally: Improve coverage job  https://review.openstack.org/175841
18:53 davideagnello boris-42: thank you :)
18:54 davideagnello boris-42: when executing a rally task with a provided .yaml or json file, can you specify to run just one benchmark from this file?
19:21 openstackgerrit Sergey Skripnick proposed openstack/rally: Fix creating user in keystone v3 wrapper  https://review.openstack.org/175504
19:23 boris-42 davideagnello: so you can write such template
19:23 boris-42 davideagnello: actually rally input task are jinja2 templates
19:23 boris-42 davideagnello: take a look here
19:26 meteorfox boris-42: I was thinking about adding a new 'scenario' primitive, that could be useful. I want to know, what you think, and see if it's even a good idea
19:26 meteorfox boris-42: so, first the problem,
19:27 boris-42 davideagnello: https://rally.readthedocs.org/en/latest/tutorial/step_4_task_templates.html
19:27 boris-42 meteorfox: hm?
19:28 meteorfox boris-42: there are multiple scenarios available, but sometimes I want to run the same scenario multiple times, but with different arguments. For example concurrent builds, 10 times with 1 concurrent, 20 times with 2 concurrent, 40 times with 4 concurrent, .....
19:28 meteorfox boris-42:  so I was thinking of a scenario of scenarios
19:29 meteorfox boris-42: for example, a RepeatScenario, that can repeat a finite number of times a collections of scenarios
19:29 boris-42 meteorfox: hm not sure?
19:29 boris-42 meteorfox: why not just passing N times
19:29 boris-42 meteorfox: scenario?
19:30 meteorfox boris-42:  or a WeightedScenarios, runs certain scenarios  more than others based on a weight
19:30 boris-42 meteorfox: I dont' understand the use case of this
19:30 meteorfox boris-42: well, say, later when we have concurrent scenarios this could run in parallel
19:31 meteorfox boris-42: let me show you in another tool, the same kind of concept, where scenarios compose
19:31 boris-42 meteorfox: so you would like to have single iterations that run Scenario1 Scenario2 Secenaro3
19:32 meteorfox boris-42: yeah, something like that, or could be Scenario 1 multiple times with different arguments
19:32 boris-42 meteorfox: hm
19:32 meteorfox boris-42: I know today, you can repeat multiple 'args,runner,context' blocks, but it's too verbose
19:32 meteorfox boris-42: a lot of things are repeated
19:33 boris-42 meteorfox: it is okay
19:34 boris-42 meteorfox: so I would prefer to keep verbose mode
19:34 boris-42 meteorfox: like if you need to check for all images & flavors matrix
19:34 boris-42 meteorfox: that VMTask.boot_run_command works
19:35 meteorfox boris-42: hmm ok
19:35 boris-42 meteorfox: you should use something like this https://gist.github.com/boris-42/b2a457b4f781357bd07f
19:35 meteorfox boris-42: ooh that could work
19:35 boris-42 meteorfox: yep you can have arrays with args
19:35 meteorfox boris-42: I forget you can do those things with jinja
19:35 boris-42 ([flavor, image, times], .....)
19:36 boris-42 and run N times
19:36 boris-42 meteorfox: actually this is what I would like to see as certified task
19:36 meteorfox ok
19:37 davideagnello boris-42: ok, thank you.  looks like you can't specify a specific benchmark when running a rally task
19:37 boris-42 davideagnello: yep
19:37 boris-42 davideagnello: just if you write such task
19:38 boris-42 davideagnello: after finishing new format
19:38 boris-42 davideagnello: I am thinking about allowing to run specific group or benchmark name
19:39 boris-42 regexp/exact match
19:40 openstackgerrit Sergey Skripnick proposed openstack/rally: WIP: Add manifest file and split jobs  https://review.openstack.org/175973
19:40 shakamunyi joined #openstack-rally
19:40 openstackgerrit Sergey Skripnick proposed openstack/rally: Add new script for rally-gate jobs  https://review.openstack.org/175549
19:43 davideagnello boris-42: ok cool
19:44 boris-42 davideagnello: it will help if you write feature request
19:44 boris-42 davideagnello: https://github.com/openstack/rally/tree/master/doc/feature_request
19:44 boris-42 davideagnello: could you do it?
19:44 boris-42 davideagnello: so I won't forget to schedule it
19:44 davideagnello boris-42: what was the command to extract the results from previous test runs and get the $? populated?  thought I had it copied from before
19:44 davideagnello boris-42: yes, I will!
19:45 boris-42 davideagnello: ah
19:45 boris-42 davideagnello: rally task sla_check
19:45 davideagnello boris-42: great, got it.  thanks :)
19:46 boris-42 davideagnello: so please make feature request it shouldn't take too much time
19:47 echoingumesh joined #openstack-rally
19:50 echoingu_ joined #openstack-rally
19:55 openstackgerrit joined #openstack-rally
19:56 davideagnello boris-42: doing it now
20:03 psd joined #openstack-rally
20:05 psd does anyone here know Carlos L. Torres' irc nick?
20:06 boris-42 psd: meteorfox ^
20:07 meteorfox psd: that's me
20:08 psd meteorfox: hey!
20:08 meteorfox psd: hi
20:09 psd meteorfox: Not sure if you checked out https://review.openstack.org/#/c/169432/5/doc/specs/in-progress/add_cloud_info.rst
20:09 meteorfox psd: I saw your comment :)
20:10 openstackgerrit Davide Agnello proposed openstack/rally: Adding new feature request for Rally.  https://review.openstack.org/176460
20:12 psd meteorfox: do you want me to add/change something or just put up a new patch?
20:13 meteorfox psd: I was just asking for an example, of what's going to be the format of the JSON value for the environment_info
20:13 meteorfox psd: ideally, you could show an example in the spec
20:14 meteorfox psd: sorry, I have a meeting to go now
20:17 psd meteorfox: okay, I'll add one and put up a new patch. thanks :)
20:18 meteorfox psd: sure, I'll review it once is up. thanks!
20:25 echoingumesh joined #openstack-rally
20:46 openstackgerrit Alexander Gubanov proposed openstack/rally: Adds Nova floating IPs bulk tests  https://review.openstack.org/168054
21:00 davideagnello boris-42: Does Rally supports accessing service API's directly through REST API's?  useful for some negative testing
21:02 davideagnello boris-42: vs. writing your own rest client as part of your tests
21:19 echoingumesh joined #openstack-rally
21:25 boris-42 davideagnello: so not sure what to say
21:26 boris-42 davideagnello: we don't have anything done realted to this
21:26 boris-42 davideagnello: but you can use python request lib
21:27 davideagnello boris-42: ok thanks, will look into python's request lib
21:30 openstackgerrit joined #openstack-rally
21:30 boris-42 davideagnello: https://review.openstack.org/#/c/117705/
21:35 davideagnello boris-42: cool
21:36 davideagnello boris-42: can you capture logs of a specific test run after running a task for it?  similar to generating reports anytime after benchmark run
21:39 boris-42 davideagnello: rally logs?
21:40 davideagnello boris-42: logs captured from benchmark runs?
21:41 davideagnello boris-42: the ones that get generated when you run with -v and --debug
21:42 boris-42 davideagnello: so nope
21:44 davideagnello boris-42: ok so we should store the output as needed then when running benchmarks
21:46 boris-42 davideagnello: so you can add as well feature requrest
21:46 boris-42 davideagnello: regarding to this
21:46 boris-42 davideagnello: and we will try to find the way to implement it
21:46 davideagnello boris-42: ok :)
22:00 boris-42 davideagnello: btw please update this one https://review.openstack.org/#/c/176460/ =)
22:00 davideagnello boris-42: whoops, ok
22:05 andreykurilin__ joined #openstack-rally
22:11 openstackgerrit joined #openstack-rally
22:13 echoingumesh joined #openstack-rally
22:20 jaypipes joined #openstack-rally
22:32 openstackgerrit Merged openstack/rally: [Swift] Add base for Swift API Benchmarks: Patch-2  https://review.openstack.org/159258
22:33 openstackgerrit Merged openstack/rally: Add function decorator `log_deprecated_args'  https://review.openstack.org/174453
22:38 openstackgerrit Prabhjyot Singh Sodhi proposed openstack/rally: [SPEC] Create 'add cloud info' spec file  https://review.openstack.org/169432
22:39 karimb joined #openstack-rally
23:36 openstackgerrit Davide Agnello proposed openstack/rally: Adding new feature request for Rally.  https://review.openstack.org/176460
23:38 openstackgerrit Davide Agnello proposed openstack/rally: Adding new feature request for Rally.  https://review.openstack.org/176460
23:44 openstackgerrit Davide Agnello proposed openstack/rally: feature request to capture task logs  https://review.openstack.org/176551

| Channels | #openstack-rally index | Today | | Search | Google Search | Plain-Text | summary