Perl 6 - the future is here, just unevenly distributed

IRC log for #openstack-rally, 2016-02-03

| Channels | #openstack-rally index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:22 yingjun joined #openstack-rally
00:39 openstackgerrit Sergey Skripnick proposed openstack/rally: Add heat siege workload scenario  https://review.openstack.org/272510
01:05 openstackgerrit Julian Edwards proposed openstack/rally: Make Rally cope with unversioned keystone URL  https://review.openstack.org/274977
01:18 openstackgerrit Sergey Skripnick proposed openstack/rally: Add heat siege workload scenario  https://review.openstack.org/272510
01:27 baker joined #openstack-rally
01:27 baker_ joined #openstack-rally
01:41 openstackgerrit li,chen proposed openstack/rally: Using 'dt' as alias for datetime imports  https://review.openstack.org/269958
01:42 chenli joined #openstack-rally
01:46 baker joined #openstack-rally
01:58 chenli_ joined #openstack-rally
02:05 baohua joined #openstack-rally
02:10 baker joined #openstack-rally
02:18 baker joined #openstack-rally
02:21 Aish joined #openstack-rally
02:21 Aish left #openstack-rally
02:31 openstackgerrit Bo Chi proposed openstack/rally: Use dict and set comprehension instead of using dict() and set()  https://review.openstack.org/273449
02:51 openstackgerrit Julian Edwards proposed openstack/rally: Make Rally cope with unversioned keystone URL  https://review.openstack.org/274977
02:53 chenli joined #openstack-rally
02:57 harshs joined #openstack-rally
03:08 paco20151113 joined #openstack-rally
03:16 fsopenstack joined #openstack-rally
03:24 akrzos joined #openstack-rally
03:28 rook-laptop joined #openstack-rally
03:51 fsopenstack joined #openstack-rally
03:52 baker joined #openstack-rally
04:15 baker_ joined #openstack-rally
04:36 akrzos joined #openstack-rally
04:37 openstackgerrit Julian Edwards proposed openstack/rally: Make Rally cope with unversioned keystone URL  https://review.openstack.org/274977
04:39 harshs joined #openstack-rally
05:04 fsopenstack joined #openstack-rally
05:51 baker joined #openstack-rally
06:03 tfreger joined #openstack-rally
06:06 harshs joined #openstack-rally
06:24 neeti joined #openstack-rally
06:40 anshul joined #openstack-rally
06:45 tfreger1 joined #openstack-rally
06:47 openstackgerrit vigny_wu proposed openstack/rally: Add the scenario of list-flavors  https://review.openstack.org/275523
07:21 tfreger joined #openstack-rally
07:38 yfried joined #openstack-rally
07:45 neeti joined #openstack-rally
08:03 chandidas joined #openstack-rally
08:18 boris-42 joined #openstack-rally
08:20 amaretskiy joined #openstack-rally
08:27 neeti joined #openstack-rally
08:27 mvk_ joined #openstack-rally
08:48 yingjun joined #openstack-rally
09:00 yingjun joined #openstack-rally
09:13 tfreger joined #openstack-rally
09:22 openstackgerrit vigny_wu proposed openstack/rally: Add the scenario of list-flavors  https://review.openstack.org/275523
09:34 adiantum joined #openstack-rally
09:39 tfreger joined #openstack-rally
09:43 neeti joined #openstack-rally
09:45 e0ne joined #openstack-rally
09:59 yingjun joined #openstack-rally
10:27 rook-laptop joined #openstack-rally
10:31 neeti joined #openstack-rally
10:51 e0ne joined #openstack-rally
11:14 bigjools joined #openstack-rally
11:14 arxcruz joined #openstack-rally
11:18 psuriset joined #openstack-rally
11:55 ikhudoshyn boris-42 andreykurilin amaretskiy rvasilets guys I'd like to draw yr attention to the patch https://review.openstack.org/#/c/274977
11:55 amaretskiy ikhudoshyn: ok
11:55 ikhudoshyn pls take a look on the discussion in comments as well and share  yr thoughts
11:59 openstackgerrit Illia Khudoshyn proposed openstack/rally: Fix Ceilometer scenarios against Keystone API v3  https://review.openstack.org/275647
12:01 andreykurilin ikhudoshyn, I'll look at it a bit later
12:08 rook-laptop joined #openstack-rally
12:40 openstackstatus NOTICE: Infra running with lower capacity now, due to a temporary problem affecting one of our nodepool providers. Please expect some delays in your jobs. Apologies for any inconvenience caused.
12:41 Ritesh_ joined #openstack-rally
12:43 qwebirc71559 joined #openstack-rally
12:45 Sravani joined #openstack-rally
12:50 gubouvier joined #openstack-rally
12:51 akrzos joined #openstack-rally
12:56 openstackgerrit Piyush Raman Srivastava proposed openstack/rally: Add list subnets/routers/FIPs to NetworkWrapper  https://review.openstack.org/272251
12:57 openstackgerrit Illia Khudoshyn proposed openstack/rally: Fix Ceilometer scenarios against Keystone API v3  https://review.openstack.org/275647
12:58 Sravani Hi, Does Rally has REST APIs? If yes, would anyone please provide the documnetation links for the same? Thanks in advance.
13:03 openstackgerrit joined #openstack-rally
13:05 MaxPC joined #openstack-rally
13:08 rook-laptop joined #openstack-rally
13:17 tosky joined #openstack-rally
13:27 yfried amaretskiy: andreykurilin: re https://bugs.launchpad.net/rally/+bug/1510175
13:27 openstack Launchpad bug 1510175 in Rally "Rally task detailed 95%ile Response Times different from HTML graphics report" [High,Triaged]
13:27 yfried which value is the right one? HTML or JSON?
13:28 amaretskiy yfried: JSON shoudl have the same value as in HTML
13:29 amaretskiy yfried: because JSON data is calculated after we get all task results, but HTML calculates it on-the-fly, so this allows to process huge amount of data without high memory usage
13:30 yfried amaretskiy: the bug says they are different, so I'm wondering which is the right one
13:30 amaretskiy yfried: here is the place whare stats is calculated for html: https://github.com/openstack/rally/blob/master/rally/task/processing/plot.py#L28
13:31 amaretskiy yfried: truthfully speaking - JSON is more accurate, but it does not have streaming calculation, so HTML-way is better
13:31 amaretskiy yfried, the difference is rather small so HTML-way is the best
13:32 amaretskiy task results (JSON) uses deprecated way and should be improved to streaming processing (like "task report" does)
13:33 amaretskiy yfried https://github.com/openstack/rally/blob/master/rally/task/processing/plot.py#L72-L74
13:33 yfried rook: ^
13:34 rook amaretskiy: interesting, so automating the results is a bit difficult then since we cannot rely on the stdout results.
13:35 rook amaretskiy: to automate this work, we should pull the json, and calculate that way.
13:35 amaretskiy rook: the difference between non-streaming calculation and streaming one is small
13:35 rook amaretskiy: i disagree,.
13:35 rook amaretskiy we have seen quite the difference, as well as the success rate being totally incorrect.
13:35 amaretskiy rook: streaming processing can be improved to become very accurate
13:36 rook the stdout will show 100% for one test, but the HTML report will report 88% for the same test.
13:36 amaretskiy rook: in any case we MUST use streaming data processing because of possible memory usage issues while processing thousands of iterations
13:37 amaretskiy rook: this is a topic to improve streaming calculation, not switch back to old solution
13:38 amaretskiy rook: here is a streaming processing class for statistics https://github.com/openstack/rally/blob/master/rally/task/processing/charts.py#L351-L384
13:38 rook amaretskiy: so, it is recommended to ditch the stdout results, and use the JSON output
13:39 amaretskiy rook: discuss this with boris-42, I'm sure he will say that we must use streaming processing
13:40 rook amaretskiy: i am of the opinion to pull the raw data myself to avoid any changes that could occur within Rally.
13:41 rook amaretskiy -- which would mean i could dump the JSON and import that into python/perl/etc and calculate, right?
13:41 amaretskiy we have a feature of custom output tables/charts for that - you can generate arbitrary charts by the scenario
13:41 rook amaretskiy: example?
13:42 yingjun joined #openstack-rally
13:42 amaretskiy rook: http://logs.openstack.org/77/274977/5/check/gate-rally-dsvm-rally/34e285e/rally-plot/results.html.gz#/Dummy.dummy_output/output
13:42 amaretskiy rook: this is relatively new improvement :)
13:43 amaretskiy rook: her eis how these tables/charts are generated https://github.com/openstack/rally/blob/master/rally/plugins/common/scenarios/dummy/dummy.py#L84-L138
13:43 rook amaretskiy: how do I generate results like this?
13:44 rook amaretskiy: this is nice - would be great if we could start using this - instead of generating our graphs outside of Raly
13:44 openstackgerrit Steve Wilkerson proposed openstack/rally: Removed mutable default argument  https://review.openstack.org/275703
13:44 amaretskiy rook: there is an improvement upcoming: https://review.openstack.org/#/c/272989/2
13:44 rook amaretskiy can we do run to run compares?
13:45 amaretskiy rook: http://logs.openstack.org/89/272989/2/check/gate-rally-dsvm-rally/9e663dd/rally-plot/results.html.gz#/Dummy.dummy_output/output
13:45 amaretskiy rook: not clear, please explain
13:45 amaretskiy rook: compare scenarios runs?
13:46 andreykurilin amaretskiy: if the difference between streaming processing and old way is in 12%(even sometimes), our streaming algorithms su*k.
13:46 rook amaretskiy correct, lets say I run a scenario with OpenStack default config... Then I make a change to the OpenStack env, re-run the scenario - Now, I want to compare the two to show the difference.
13:47 amaretskiy rook: the feature you asking is on my development right now - it even possible that I submit a first patch set today :)
13:48 baohua joined #openstack-rally
13:48 rook amaretskiy cool
13:48 amaretskiy rook: streaming calculation must be improved, not thrown out
13:48 rook amaretskiy we would make use of that in Browbeat -- we are wrapping different Rally scenarios with changes to the infrastructure to show how those changes impact the env.
13:49 amaretskiy rook: here is a causer https://github.com/openstack/rally/blob/master/rally/common/streaming_algorithms.py#L154-L181
13:49 rook right now we are building the graphs external to Rally
13:49 rook but we have been using the stdout results...
13:49 rook which has been a thorn in our side.
13:50 amaretskiy rook: you need to have all the graphs in rally html report, right?
13:51 rook amaretskiy: nope, not necessarily.
13:52 openstackgerrit Chris St. Pierre proposed openstack/rally: Add wrapper for Glance API versions  https://review.openstack.org/264964
13:52 amaretskiy rook: feel free to discuss with me any problems/features related to reports
13:53 amaretskiy rook: right now I'm not planning to fix streaming calculation (working on another high priority task), but this is an important task for close future
13:56 abehl joined #openstack-rally
14:02 openstackgerrit Chris St. Pierre proposed openstack/rally: Create new types.convert()  https://review.openstack.org/249462
14:02 openstackgerrit Chris St. Pierre proposed openstack/rally: Convert ResourceType classes to plugins  https://review.openstack.org/249463
14:02 openstackgerrit Chris St. Pierre proposed openstack/rally: Convert existing scenarios to new types plugins  https://review.openstack.org/249464
14:07 rook amaretskiy can you please at least document the issue on readthedocs?
14:07 rook amaretskiy: and mention which source is the *right* source to use.
14:07 rook given the issue?
14:07 rook I think that might be a good first step.
14:08 amaretskiy rook: I think that this info is for launchpad bug, not docs
14:08 amaretskiy I will add a comment to a bug on launchpad
14:09 amaretskiy rook: maybe discuss this on our weekly rally meeting?
14:10 amaretskiy rook https://wiki.openstack.org/wiki/Meetings/Rally
14:10 rook amaretskiy i think it is important for users to understand that they *will* see a difference. it is totally confusing and gives users a lack of confidence in the numbers rally generates.
14:10 rook which, that is the most important thing to Rally, is the results it generates.
14:11 rvasilets joined #openstack-rally
14:16 rook amaretskiy do you disagree?
14:17 amaretskiy rook: agree about bug priority, not about mentioning the bug in documentation
14:18 amaretskiy rook: we have special resource for management bugs
14:19 amaretskiy rook: I've added a descriptive comment to bug
14:19 rook alright...
14:20 rook guess we can agree to disagree ;P
14:20 amaretskiy rook: :)
14:20 amaretskiy rook: let's talk about this on our weekly meeting
14:21 amaretskiy root: meeting is a great place because most members of rally team can participate
14:21 amaretskiy *rook
14:21 rook sure, ill think about joining :)
14:22 rook i used to join - but got buried with other things
14:22 amaretskiy rook: great :)
14:22 amaretskiy rook: great :)  <-- abiut joining
14:22 amaretskiy *about
14:33 absubram joined #openstack-rally
14:34 absubram_ joined #openstack-rally
14:35 openstackgerrit Piyush Raman Srivastava proposed openstack/rally: Add list subnets/routers/FIPs to NetworkWrapper  https://review.openstack.org/272251
14:57 redixin joined #openstack-rally
14:59 redixin hiyo
14:59 ikhudoshyn hi redixin
15:02 ylobankov joined #openstack-rally
15:16 boris-42 joined #openstack-rally
15:17 ylobanko_ joined #openstack-rally
15:27 baker joined #openstack-rally
15:30 harshs joined #openstack-rally
15:35 fsopenstack joined #openstack-rally
15:41 arnoldje joined #openstack-rally
16:08 ikhudoshyn boris-42: andreykurilin redixin amaretskiy rvasilets guys! this is to enable keystone v3 api by default and add dedicated v2 job https://review.openstack.org/#/c/274668/
16:09 ikhudoshyn please come and look! I'd like to be sure it's fine for us before I ping infra team
16:10 ikhudoshyn and pls don't forget https://review.openstack.org/#/c/274700/
16:12 yfried joined #openstack-rally
16:13 fsopenstack joined #openstack-rally
16:13 rvasilets ok
16:17 fsopenstack joined #openstack-rally
16:21 yfried joined #openstack-rally
16:22 fsopenstack joined #openstack-rally
16:25 baker joined #openstack-rally
16:30 agrebennikov joined #openstack-rally
16:31 rook-laptop joined #openstack-rally
16:34 shausy joined #openstack-rally
16:48 aarefiev joined #openstack-rally
16:49 NikitaKonovalov joined #openstack-rally
16:52 cmkirkla joined #openstack-rally
17:05 agrebennikov boris-42, any suggestions on my problem?
17:10 adiantum joined #openstack-rally
17:11 yfried_ joined #openstack-rally
17:14 absubram joined #openstack-rally
17:24 baker joined #openstack-rally
17:30 cmkirkla joined #openstack-rally
17:37 anshul joined #openstack-rally
17:37 abehl joined #openstack-rally
17:44 fsopenstack joined #openstack-rally
17:59 madhu_ak joined #openstack-rally
18:02 josdotso joined #openstack-rally
18:02 baker joined #openstack-rally
18:24 e0ne joined #openstack-rally
18:27 pvaneck joined #openstack-rally
18:37 rook-laptop joined #openstack-rally
18:41 mvk_ joined #openstack-rally
18:58 yfried_ joined #openstack-rally
19:00 harshs joined #openstack-rally
19:08 cmkirkla joined #openstack-rally
19:10 ylobankov joined #openstack-rally
19:37 rook-laptop joined #openstack-rally
19:44 kjorgensen joined #openstack-rally
19:44 ylobankov joined #openstack-rally
19:47 cmkirkla joined #openstack-rally
19:49 Aish joined #openstack-rally
20:01 absubram joined #openstack-rally
20:12 ningy joined #openstack-rally
20:21 ningy @here could anyone point me to an example of using ddt module?
20:31 akrzos_ joined #openstack-rally
20:35 psuriset joined #openstack-rally
20:36 redixin ningy: https://github.com/openstack/rally/blob/master/tests/unit/plugins/openstack/scenarios/neutron/test_network.py#L354-L371
20:37 redixin ningy: it will call test_create_and_list_floating_ips twice passing different arguments (listed in @ddt.data)
20:40 ningy thanks @redixin
20:45 Aish joined #openstack-rally
21:14 boris-42 joined #openstack-rally
21:37 psuriset joined #openstack-rally
21:52 ylobankov joined #openstack-rally
21:55 madhu_ak joined #openstack-rally
22:03 psuriset joined #openstack-rally
22:32 psuriset joined #openstack-rally
22:45 ylobankov joined #openstack-rally
22:55 rook-laptop joined #openstack-rally
23:07 kjorgensen joined #openstack-rally
23:09 absubram joined #openstack-rally
23:23 e0ne joined #openstack-rally
23:24 MaxPC joined #openstack-rally
23:35 openstackgerrit Merged openstack/rally: Removed mutable default argument  https://review.openstack.org/275703

| Channels | #openstack-rally index | Today | | Search | Google Search | Plain-Text | summary