Perl 6 - the future is here, just unevenly distributed

IRC log for #openstack-rally, 2015-04-20

| Channels | #openstack-rally index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:30 pradeep joined #openstack-rally
01:07 pradeep joined #openstack-rally
01:23 openstackgerrit Boris Pavlovic proposed openstack/rally: (WIP) sharing idea regarding to testing that cleanups works properly  https://review.openstack.org/175244
01:27 PerfBeing joined #openstack-rally
01:39 PerfBeing joined #openstack-rally
02:04 PerfBeing joined #openstack-rally
02:32 PerfBeing joined #openstack-rally
02:57 PerfBeing joined #openstack-rally
03:05 PerfBeing joined #openstack-rally
03:53 nkhare joined #openstack-rally
03:56 pradeep joined #openstack-rally
04:14 PerfBeing joined #openstack-rally
04:23 PerfBeing joined #openstack-rally
04:25 subscope joined #openstack-rally
04:28 tfreger joined #openstack-rally
04:32 aswadr joined #openstack-rally
04:58 PerfBeing joined #openstack-rally
05:15 _kiran_ joined #openstack-rally
05:21 _kiran_ Hello! :)
06:14 anshul joined #openstack-rally
06:19 yfried__ joined #openstack-rally
06:33 neeti joined #openstack-rally
06:40 anshul joined #openstack-rally
06:53 pbandzi joined #openstack-rally
06:59 akuznetsova joined #openstack-rally
07:04 anshul joined #openstack-rally
07:05 igormarnat joined #openstack-rally
07:08 PerfBeing joined #openstack-rally
07:31 e0ne joined #openstack-rally
07:38 e0ne joined #openstack-rally
07:38 kiranr joined #openstack-rally
07:45 arxcruz joined #openstack-rally
07:45 e0ne joined #openstack-rally
07:57 _kiran_ joined #openstack-rally
08:06 fhubik joined #openstack-rally
08:08 e0ne joined #openstack-rally
08:13 exploreshaifali joined #openstack-rally
08:30 dmellado joined #openstack-rally
08:32 openstackgerrit Swapnil Kulkarni proposed openstack/rally: Update stackforge to openstack  https://review.openstack.org/175302
08:39 andreykurilin__ joined #openstack-rally
08:40 Vishal_ joined #openstack-rally
08:40 karimb joined #openstack-rally
08:44 Vishal_ andreykurilin__: Hi..Please review https://bugs.launchpad.net/rally/+bug/1442124
08:46 anshul joined #openstack-rally
08:59 e0ne joined #openstack-rally
09:09 andreykurilin__ Vishal_: hi. I'll review it in several hours.
09:10 Vishal__ joined #openstack-rally
09:17 Vishal_ joined #openstack-rally
09:29 zerda joined #openstack-rally
09:30 pcaruana joined #openstack-rally
09:52 boris-42 joined #openstack-rally
10:00 openstackgerrit Arx Cruz proposed openstack/rally: Adding FunctionalMixin class  https://review.openstack.org/146838
10:01 aix joined #openstack-rally
10:01 neeti joined #openstack-rally
10:09 yfried__ boris-42: has rally repo changed path?
10:12 msdubov_ joined #openstack-rally
10:12 boris-42 yfried__: yep
10:13 boris-42 yfried__: we are not in openstack/* namespace
10:19 msdubov_ boris-42 andreykurilin yfried__ rvasilets guys let's discuss the 0.0.4 release.Any ideas what patches should be included in yhe next release?
10:20 rvasilets Existing users)?
10:20 boris-42 msdubov_: Existing Users
10:20 boris-42 msdubov_: Murano benchmarks
10:21 rvasilets https://review.openstack.org/#/c/148079/ streaming algorithms?
10:22 Guest3939 Anything being done with Swift?
10:22 msdubov_ rvasilets boris-42 thanks, yep, I think streaming algorithms & new sla should also be there, long on review
10:22 cdent joined #openstack-rally
10:23 msdubov_ maybe also some changes in Rally API? In case we will progress with my doc
10:23 pradeep msdubov_: ironic benchmarks
10:24 msdubov_ pradeep what's the current status of ironic benchmarks?
10:24 boris-42 pradeep: does anybody works on that?
10:24 pradeep msdubov_: 0 :)
10:24 pradeep msdubov_: boris-42: i have plans
10:24 andreykurilin Guest3939: https://review.openstack.org/#/c/159258/
10:24 msdubov_ pradeep then we'll have to wait with that, probably
10:25 rook_ andreykurilin: did anyone look at integrating/using ssbench w/ rally?
10:25 msdubov_ pradeep we have only 2-3 weeks till the next release
10:25 boris-42 andreykurilin: +1 for swift benchmarks
10:25 boris-42 rook_: pradeep one detail
10:25 pradeep msdubov_:  ok. i will try before that.
10:25 boris-42 rook_: pradeep we are not dicussing now long term plans
10:25 andreykurilin Also, I want to see https://review.openstack.org/#/c/162418/ in rally 0.0.4
10:25 boris-42 rook_: pradeep we have really rapid release strategy
10:25 pradeep boris-42: ok
10:25 boris-42 rook_: pradeep each 2-3 weeks release, so we need to plan
10:26 boris-42 rook_: pradeep work for next 2-3 weeks
10:26 boris-42 rook_: pradeep and have list of must have in next 2 releases
10:27 redixin joined #openstack-rally
10:27 openstackgerrit Merged openstack/rally: Update stackforge to openstack  https://review.openstack.org/175302
10:27 msdubov_ redizin any proposals for the next release?
10:28 msdubov_ redixin ^
10:28 redixin msdubov_, no
10:29 rook_ boris-42: makes sense.
10:29 rook_ boris-42 just want to see if there were plans for Swift workloads.
10:31 boris-42 rook_: so there is already benchmark on review
10:31 boris-42 rook_: and I believe it's already in our priority list
10:31 boris-42 msdubov_: what about plugin base ?
10:31 rook_ boris-42: yup, andreykurilin shared it.
10:32 rvasilets We need to order proposes
10:32 boris-42 rvasilets: ?
10:32 msdubov_ boris-42 have to review, but seems like you've done a lot of work on it?
10:32 msdubov_ boris-42 rvasilets priorities?
10:32 rvasilets what is the most important to review/merge, what is the second...
10:32 boris-42 msdubov_: so I am working on rebasing stuff
10:32 rvasilets yes
10:33 boris-42 msdubov_: btw where is doc?
10:33 boris-42 msdubov_: with list?
10:33 boris-42 msdubov_: I would like to put it in topic
10:33 boris-42 msdubov_: regarding to plugin base I am going to finish rebase today
10:34 boris-42 msdubov_: btw it will be nice if this get in next release https://review.openstack.org/#/c/162418/
10:44 oanufriev joined #openstack-rally
10:44 msdubov__ joined #openstack-rally
10:44 msdubov__ boris-42 Here it is https://docs.google.com/document/d/1TX5zpYcTX8AXm-K_h1lzUNVCMvbRgsjUKU-dEYNWLY8/edit
10:45 boris-42 msdubov__: make it shared
10:45 boris-42 msdubov__: everybody should be able to comment it and view it without any restrictions
10:45 msdubov__ It is shared for commenting for anyone with the link
10:46 msdubov__ boris-42, ^
10:46 msdubov__ boris-42, Do you have any problems?
10:47 boris-42 msdubov__: it is bad that only people that have link can view it
10:47 boris-42 msdubov__: what you are trying to hide ??
10:47 boris-42 msdubov__: chaged to public on web
10:47 msdubov__ boris-42, no problem, I made it public
10:47 msdubov__ boris-42, anyway you were going to post the link here
10:48 boris-42 msdubov__: I alreayd did that lol
10:49 msdubov__ boris-42, Can I send this doc now to the mailing list?
10:50 boris-42 msdubov__: nope
10:50 boris-42 msdubov__: you should put some information
10:50 msdubov__ boris-42, expected release date? or what?
10:50 boris-42 msdubov__: to this document
10:50 boris-42 msdubov__: first of all dates
10:51 boris-42 msdubov__: the second thing you should have at least one paragraph that explains purpose of this document
10:51 boris-42 rook_: ping
10:51 boris-42 rook_: did you see this https://bugs.launchpad.net/rally/+bug/1442124 ?
10:52 msdubov__ boris-42, Can we set as the expected release data May 4th?
10:54 msdubov__ boris-42, added some info
10:54 boris-42 msdubov__: yep let it be 4th May
10:55 anshul joined #openstack-rally
10:56 msdubov__ boris-42, ok now?
10:56 msdubov__ boris-42, I'm ready to send it to openstack-dev
10:59 msdubov__ boris-42 ?
11:00 rvasilets pradeep add tra before Rally
11:00 rvasilets in doc
11:00 rvasilets what is it?
11:01 boris-42 msdubov__: hm strange how he was able to do that
11:01 pradeep rvasilets: sorry. By mistake.
11:01 boris-42 pradeep: no worries
11:01 boris-42 pradeep: seems like it was just comment/suggestion
11:01 msdubov__ boris-42, Yep, for me the doc looks like open for view & comment only
11:02 msdubov__ boris-42, For you too?
11:02 rvasilets can you see test?
11:02 msdubov__ rvasilets, Yep
11:02 msdubov__ rvasilets, I can accept or decline it
11:03 rvasilets so I can edit too)
11:03 msdubov__ rvasilets, I have declined your test change
11:03 msdubov__ rvasilets, Nope
11:03 msdubov__ rvasilets, This is like a comment
11:03 rvasilets Okey
11:05 msdubov__ boris-42, rvasilets Okay I've sent a message to openstack-dev
11:05 rvasilets look good, but what about priorities? Do we need to add them?
11:06 boris-42 rook_: msdubov__ I believe we shouldn't
11:06 boris-42 rook_: msdubov__ just make on top more important *
11:10 openstackgerrit Sergey Skripnick proposed openstack/rally: WIP: super new rally_gate.py  https://review.openstack.org/163785
11:17 softCloud joined #openstack-rally
11:20 nkhare joined #openstack-rally
11:26 boris-42 #startmeeting Rally
11:27 boris-42 nope it doesn't work..
11:27 anshul joined #openstack-rally
11:32 openstackgerrit Wataru Takase proposed openstack/rally: [spec] Refactoring Rally cleanup  https://review.openstack.org/172967
11:36 tosky joined #openstack-rally
11:38 panbalag joined #openstack-rally
11:41 msdubov_ joined #openstack-rally
11:44 fhubik_afk joined #openstack-rally
11:46 fhubik_afk joined #openstack-rally
11:59 fhubik joined #openstack-rally
12:10 openstackgerrit Sergey Skripnick proposed openstack/rally: WIP: super new rally_gate.py  https://review.openstack.org/163785
12:15 exploreshaifali joined #openstack-rally
12:26 e0ne joined #openstack-rally
12:30 softCloud joined #openstack-rally
12:33 jaypipes joined #openstack-rally
12:42 arxcruz joined #openstack-rally
12:45 Vishal_ Hi..I have one question...can rally server be used by multiple people for different deployments at the same time....asking this because when I do rally deployment list...the * in the active column is in front of only one of the deployments??
12:57 boris-42 Vishal_: hi there
12:57 Vishal_ boris-42: Hi
12:57 boris-42 Vishal_: yep it can
12:58 Vishal_ boris-42: so is there any significance of * in the active column
12:58 Vishal_ boris-42: when we do rally deployment list
12:59 boris-42 Vishal_: it means that that deployment-uuid will be passed to all comands
13:00 boris-42 Vishal_: you can pass for any command "rally task start --deployment <UUID|NAME>"
13:02 boris-42 Vishal_: as well if you have different linux users then they have own directories ~
13:02 boris-42 Vishal_: and ~/.rally/
13:02 boris-42 Vishal_: where info about default deployment is stored
13:04 Vishal_ boris-42: thanks for the clarification
13:07 svasheka joined #openstack-rally
13:08 fhubik_ joined #openstack-rally
13:18 openstackgerrit Alexander Gubanov proposed openstack/rally: Adds Nova floating IPs bulk tests  https://review.openstack.org/168054
13:20 Vishal_ Hi all..would request everybody to review https://bugs.launchpad.net/rally/+bug/1442124
13:26 zerda joined #openstack-rally
13:27 boris-42 yfried__: ^ could you take a look?
13:31 openstackgerrit Alexander Gubanov proposed openstack/rally: Adds Nova floating IPs bulk tests  https://review.openstack.org/168054
13:41 yfried__ boris-42: at Vishal_'s bug?
13:42 rook joined #openstack-rally
13:44 boris-42 yfried__: there is alredy patch on review
13:44 pradeep1 joined #openstack-rally
13:56 yfried__ boris-42: posted my review. I think it has a flaw
13:57 openstackgerrit Sergey Skripnick proposed openstack/rally: WIP: super new rally_gate.py  https://review.openstack.org/163785
14:02 yfried__ boris-42: VIshal just dropped...
14:09 mwagner_lap joined #openstack-rally
14:37 Vishal_ joined #openstack-rally
14:38 yfried__ Vishal_: have you seen my comment?
14:38 karimb joined #openstack-rally
14:40 Vishal_ yfried__: yes, thanks for the review
14:41 yfried__ Did I miss anything? or is it really no handling the case of mulitple network per tenant
14:41 yfried__ ?
14:41 yfried__ Vishal_: ^
14:43 Vishal_ yfried__: you are right it does not handle multiple networks per tenant...but in the multiple case generate_cidr function will be changed completely or a new function has to be created
14:43 yfried__ Vishal_: the current status blocks you from testing certain configurations
14:44 yfried__ Vishal_: the new patch is opening the run for very probable errors
14:45 yfried__ Vishal_: the solution can either look into the network context, or do a try/except iterations in subnet-create until a suitable address is picked
14:46 Vishal_ yes..as an example the way vmware nsxv solution handles the overlapping ip's is completely different from the non-overlapping ones....Also having overlapping ip's is very common in cloud? what you say
14:46 yfried__ Vishal_: anyway, I'd suggest for the default mode to be "overlapping IPs = true" (with a fix like ^)
14:47 yfried__ Vishal_: working with neutron as default (ie devstack and most other installations I'm aware of) has overlapping IPs as default
14:47 Vishal_ yfried__: yes
14:47 yfried__ Vishal_: ^ that's the point of tenant isolation - you can't know other tenant's networks, but you certainly know your own
14:48 Vishal_ yfried__: sure will try to make the solution compatible to your suggestion
14:51 Vishal_ yfried__: thanks
14:53 cdent joined #openstack-rally
14:53 tosky joined #openstack-rally
14:57 yfried__ Vishal_: np
15:03 openstackgerrit Sergey Skripnick proposed openstack/rally: WIP: super new rally_gate.py  https://review.openstack.org/163785
15:17 jseutter joined #openstack-rally
15:19 openstackgerrit Sergey Skripnick proposed openstack/rally: WIP: super new rally_gate.py  https://review.openstack.org/163785
15:26 redixin joined #openstack-rally
15:33 PerfBeing joined #openstack-rally
15:35 darkhuy2 joined #openstack-rally
15:37 PerfBeing joined #openstack-rally
15:57 openstackgerrit Sergey Skripnick proposed openstack/rally: WIP: super new rally_gate.py  https://review.openstack.org/163785
16:12 karimb joined #openstack-rally
16:25 openstackgerrit Roman Vasilets proposed openstack/rally: Add Http Request Scenario  https://review.openstack.org/117705
16:36 exploreshaifali joined #openstack-rally
16:46 frobware_ joined #openstack-rally
17:20 davideagnello joined #openstack-rally
17:24 davideagnello boris-42: does the Rally framework have built in support for negative testing or tests?
17:24 boris-42 davideagnello: negative testing can be done with SLA checks
17:25 boris-42 davideagnello: like you can expect that something will fail
17:25 openstackgerrit Sergey Skripnick proposed openstack/rally: WIP: super new rally_gate.py  https://review.openstack.org/163785
17:25 boris-42 davideagnello: e.g. if you try to create more VMs that quotas allows you
17:25 davideagnello boris-42: using assert <<expression>>
17:26 boris-42 davideagnello: nope
17:27 boris-42 davideagnello: in scenario you can just raise exceptions
17:27 boris-42 davideagnello: and check that with SLA stuff
17:28 davideagnello boris-42: ok, would the SLA details be held by rally?  how would the test access this information?
17:28 boris-42 davideagnello: hm
17:29 boris-42 davideagnello: sla are plugins that analyze results and says yes-no
17:29 boris-42 davideagnello: like this https://github.com/openstack/rally/blob/master/rally/benchmark/sla/base.py#L155-L186
17:38 davideagnello boris-42: ok thank you, taking a look
17:39 openstackgerrit Sergey Skripnick proposed openstack/rally: Fix creating user in keystone v3 wrapper  https://review.openstack.org/175504
17:39 davideagnello boris-42: is there tests written in rally that run in Tempest that you can point me to?
17:45 boris-42 davideagnello: not sure that I fully get the question..
17:46 boris-42 davideagnello: rally can run tempest (but rally framework and tempest are fully different frameworks)
17:46 davideagnello boris-42: Rally supports executing Tempest test?  yes, looking for any rally tests that run tempest tests to see how it works
17:48 boris-42 davideagnello: there are 2 things
17:48 yfried__ joined #openstack-rally
17:48 boris-42 davideagnello: "rally verify" (so it actually runs testr)
17:49 boris-42 davideagnello: "rally task" (scenario that runs tempest)
17:49 boris-42 davideagnello: https://github.com/openstack/rally/tree/master/samples/tasks/scenarios/tempest-do-not-run-against-production
17:51 PerfBeing joined #openstack-rally
17:53 openstackgerrit Sergey Skripnick proposed openstack/rally: WIP: super new rally_gate.py  https://review.openstack.org/163785
17:59 openstackgerrit Sergey Skripnick proposed openstack/rally: WIP: super new rally_gate.py  https://review.openstack.org/163785
18:00 davideagnello boris-42: thank you!
18:01 boris-42 davideagnello: but do not run rally tempest scenario against proudction cloud
18:02 davideagnello boris-42: ok, just read the warning in the ReadMe :)
18:19 stpierre joined #openstack-rally
18:21 openstackgerrit Chris St. Pierre proposed openstack/rally: Update .gitreview to new project  https://review.openstack.org/175523
18:23 darkhuy joined #openstack-rally
18:27 openstackgerrit Chris St. Pierre proposed openstack/rally: Add scenario to modify Cinder volume metadata  https://review.openstack.org/174894
18:29 e0ne joined #openstack-rally
18:33 PerfBeing joined #openstack-rally
18:55 openstackgerrit Sergey Skripnick proposed openstack/rally: WIP: super new rally_gate.py  https://review.openstack.org/163785
18:55 andreykurilin__ joined #openstack-rally
18:58 e0ne joined #openstack-rally
19:06 davideagnello boris-42: if you have a set of rally tests that make use of one resource (e.g. set of VMs) can you specify the order of scenario execution?
19:08 boris-42 davideagnello: hm?
19:10 davideagnello boris-42: can execution order be specified in a set of scenario tests/benchmarks?
19:11 boris-42 davideagnello: I do not understand where
19:12 boris-42 davideagnello: you mean in rally input task or ?
19:13 davideagnello boris-42: in the configuration file (yaml/json) where tasks are defined
19:13 davideagnello boris-42: the order of tasks in this file doesn't constitute the order of execution
19:14 boris-42 davideagnello: yep
19:14 boris-42 davideagnello: that is well known issue
19:14 boris-42 davideagnello: so we are working on new input format
19:15 boris-42 davideagnello: you can take a look here https://github.com/openstack/rally/blob/master/doc/specs/in-progress/new_rally_input_task_format.rst
19:17 davideagnello boris-42: ok great, with this new format you can also control the execution order of your tasks
19:18 davideagnello boris-42: when do you think this new format will be enabled?
19:19 echoingumesh joined #openstack-rally
19:19 pbandzi joined #openstack-rally
19:20 boris-42 davideagnello: I really do not exactly. It requires a lot of changes. Like changing DB schema, and we are not able at this moment to change DB schema without migrations, and migrations mechanism requires upgrades
19:21 boris-42 davideagnello: as well we need to do REAL migrations not fakes like those sqla-migrate
19:26 openstackgerrit Sergey Skripnick proposed openstack/rally: Add new script for rally-gate jobs  https://review.openstack.org/175549
19:26 davideagnello boris-42: yeah, looks like it will require a lot of changes for this.  This restructuring would be really valuable :)
19:30 pbandzi joined #openstack-rally
19:31 boris-42 davideagnello: yep I stareted developing this new format in the middle of the Nov 2014 -)
19:31 boris-42 or October=)
19:33 boris-42 davideagnello: btw this will be as well great improvement https://review.openstack.org/#/c/175549/
19:34 boris-42 davideagnello: but it is more related to gates and quality of rally
19:57 davideagnello boris-42: cool, nice to see the quality bar is being kept high
20:00 andreykurilin__ joined #openstack-rally
20:02 openstackgerrit Mikhail Dubov proposed openstack/rally: [Docs] Various fixes  https://review.openstack.org/174639
20:03 openstackgerrit Boris Pavlovic proposed openstack/rally: Improve coverage job  https://review.openstack.org/175557
20:03 openstackgerrit Boris Pavlovic proposed openstack/rally: [DO NOT MERGE] Testing coverage job  https://review.openstack.org/175558
20:03 openstackgerrit Mikhail Dubov proposed openstack/rally: Add streaming algorithms and SLA to check for outliers  https://review.openstack.org/148079
20:11 davideagnello boris-42: has the rally repository moved?  I can't seem to access: https://git.openstack.org/openstack/rally
20:11 darkhuy davideagnello:rally has been moved to official openstack
20:11 darkhuy #openstack-rally:it is no longer in incubation
20:13 davideagnello darkhuy: that makes sense, great!
20:17 boris-42 davideagnello: maybe https://git.openstack.org/cgit/openstack/rally
20:17 andreykurilin__ joined #openstack-rally
20:17 boris-42 davideagnello: yep we were on stackforge but recently moved to openstack
20:17 boris-42 davideagnello: btw https://git.openstack.org/cgit/openstack/rally =)
20:17 boris-42 might of automation
20:18 davideagnello boris-42: ok I see that :)  nice
20:21 darkhuy boris-42:is there documentation somewhere or a good way to trace through rally execution code?
20:23 boris-42 darkhuy: heh so....
20:23 boris-42 darkhuy: https://github.com/openstack/rally/tree/master/rally/cmd/commands
20:23 boris-42 darkhuy: so these are CLI commands
20:24 boris-42 darkhuy: this is future rally as lib API https://github.com/openstack/rally/blob/master/rally/api.py
20:24 boris-42 darkhuy: so start from it =)
20:26 darkhuy boris-42: heh...ok...maybe i should start from the top and work down.....looking at those 3 files you told me to refactor....my brain almost explode trying to figure out what is doing what
20:27 boris-42 darkhuy: actually code in rally is quite straight =)
20:28 darkhuy boris-42:its also massiveeee
20:30 boris-42 darkhuy: it's mostly because of a lot of plugins
20:39 davideagnello boris-42: with the new input task format, can you explain what you mean by the third issue this is resolving:  No way to support multi scenario load, because we used scenario name as a identifier of single task
20:39 davideagnello boris-42: there is currently support to run multiple benchmarks in a single task
20:40 boris-42 davideagnello: scenario != benchmark
20:40 boris-42 davideagnello: there is no support to run simultaneously N scenarios in single benchmark
20:40 davideagnello boris-42: ok, a benchmark can be seen as a single execution of a scenario?
20:41 boris-42 davideagnello: nope benchmark is together (scenario + context + sla + runner)
20:41 boris-42 davideagnello: scenario is just method that is used to generate load
20:41 davideagnello boris-42: ok, but you can run multiple benchmarks in parallel?
20:41 davideagnello boris-42: ok
20:43 kevinbenton joined #openstack-rally
20:43 boris-42 davideagnello: nope
20:43 boris-42 davideagnello: you can't
20:43 boris-42 davideagnello: only if you run N times "rally task start"
20:43 boris-42 davideagnello: but it is mess
20:43 boris-42 davideagnello: and doesn't allow you to have single context for N scenarios
20:44 davideagnello boris-42: ok, so currently there is no way to execute multiple unrelated benchmarks in parallel?
20:44 kevinbenton joined #openstack-rally
20:45 boris-42 davideagnello: nope
20:45 boris-42 davideagnello: the new input format is mostly created to fix this
20:47 davideagnello boris-42: ah ok, looking forward to the new input format :)
20:51 davideagnello boris-42: are there other assert types supported in Rally.  Comparing to the python unit test framework, like assertEqual, assertTrue, etc..
20:53 boris-42 davideagnello: take a look at this patch
20:53 boris-42 https://review.openstack.org/#/c/146838/
20:55 davideagnello boris-42: nice
20:57 meteorfox boris-42: is the new coverage gate up?
20:57 boris-42 meteorfox: ?
20:58 rook_ joined #openstack-rally
20:58 meteorfox boris-42: the coverage job
20:59 boris-42 meteorfox: what do you mean by up?
20:59 boris-42 meteorfox: we merged one patch and going to merge one more to improve it
20:59 boris-42 meteorfox: https://review.openstack.org/#/c/175557/
20:59 meteorfox boris-42:  'up' as in working and merged
20:59 meteorfox :)
21:00 boris-42 meteorfox: so first part work and it is merged
21:00 boris-42 meteorfox: this just use another metric (missing lines instead of percentage)
21:00 boris-42 meteorfox: and shows diff
21:00 meteorfox boris-42: cool
21:02 meteorfox boris-42: unrelated topic. I ran a few comparisons using Rally, compared to Gatling (http://gatling.io). Obviously some discrepancies are expected, but I measured around 80% overhead on the reported response times
21:03 meteorfox boris-42: the scenario I used for the comparison, was  the NeutronNetworks.create_and_delete_networks
21:04 meteorfox boris-42: which pretty much just does the POST and DELETE but using the python-neutronclient
21:04 boris-42 meteorfox: so hm what concurrency?
21:05 meteorfox 10
21:06 meteorfox I tried to included everything Rally does with Gatling. Creating the tenants/users, the authenticating, and creating the networks
21:06 meteorfox boris-42: I don't think there are polling intervals in the python-neutronclient, I looked at the code, and it seems just a call to create, and then a call to delete
21:07 meteorfox boris-42: is there a sleep or something in the runner?
21:07 boris-42 meteorfox: so what did you mearued load duration
21:08 boris-42 meteorfox: or avarage
21:08 meteorfox response times
21:08 boris-42 meteorfox: duration?
21:08 boris-42 meteorfox: what means response times?
21:08 boris-42 meteorfox: you collected all logs?
21:09 meteorfox boris-42: so the report, shows the different aggregate statistics for create and delete response times
21:09 boris-42 meteorfox: sorry but you need to be more preciese in telling me what you compared=)
21:09 boris-42 otherwise it is not clear did you the same thing in both cases
21:10 boris-42 like in rally you took atomic actions?
21:10 meteorfox boris-42: ok, so the HTML report, reports the atomic_actions response times
21:10 boris-42 just duration of atomic_actions*
21:10 boris-42 okay
21:10 meteorfox yes
21:10 boris-42 and what did you in gatling.io
21:10 boris-42 ?
21:11 meteorfox I replicated the same scenario using Gatling
21:11 meteorfox create/delete networks
21:12 meteorfox boris-42: I know there will be an overhead of using Rally, because it uses the python-*client libraries, but I didn't expect this much
21:13 boris-42 meteorfox: so what was duration of single iteration?
21:13 meteorfox boris-42: ok, let me fetch data, one sce
21:13 meteorfox sec*
21:14 boris-42 meteorfox: I mean Rally is not good for short scenarios
21:14 boris-42 meteorfox: because you have various overheads and python is slow
21:14 meteorfox boris-42: yeah, I know. Probably, it will work better for long running atomic_actions
21:15 meteorfox like building servers and stuff
21:17 meteorfox boris-42: here's one iteration https://gist.github.com/meteorfox/340d260e1657e60223e8
21:17 PerfBeing joined #openstack-rally
21:17 PerfBeing joined #openstack-rally
21:18 meteorfox boris-42: actual response times for network create for example, are around 130 ms
21:19 boris-42 meteorfox: but you have forgot that you are not parsing HTTP output to python objects and vise versa
21:20 boris-42 =)
21:20 meteorfox boris-42: yeah, I understand. I always had that in mind :)
21:22 meteorfox boris-42: for most tests this might be good enough, but for others I'll have to use other tools with lower latency
21:22 openstackgerrit Sergey Skripnick proposed openstack/rally: Fix creating user in keystone v3 wrapper  https://review.openstack.org/175504
21:23 boris-42 meteorfox: heh
21:24 boris-42 meteorfox: not sure that this is enough important in case of openstack
21:25 meteorfox boris-42: so, one question, does the HTML report, combines the successful and failed atomic_actions ?
21:25 boris-42 meteorfox: what do you mean by combines?
21:26 meteorfox boris-42: so here's the data from the HTML report from Rally
21:26 meteorfox https://gist.github.com/meteorfox/faa8c67615c3e36e09f8
21:26 meteorfox are those number aggregating durations from 'failed' atomic_actions too?
21:26 boris-42 nope
21:26 boris-42 meteorfox: just from not failed
21:27 meteorfox boris-42: ok. For comparison here's what I get from Gatling
21:27 boris-42 meteorfox: btw did you generate the same load?
21:28 meteorfox boris-42: yes, of course. I made sure everything was as similar as possible
21:28 meteorfox creating users/tenants, the load, 10 concurrent users, a total 100 networks
21:29 boris-42 btw in rally we create users only once
21:29 boris-42 create users/tennts
21:29 boris-42 do the load
21:29 boris-42 delete users/tenants
21:30 e0ne joined #openstack-rally
21:31 meteorfox boris-42: I did the same :) create all users/tenants first, then up to 10 threads concurrently authenticates, then each users builds 10 networks, and then deletes them
21:31 meteorfox boris-42: here's the data for create_network that I got from Gatling, these are just 'success' actions
21:31 meteorfox https://gist.github.com/meteorfox/cfc256a3631c78baacda
21:31 meteorfox boris-42: also, is in milliseconds
21:32 openstackgerrit Chris St. Pierre proposed openstack/rally: Add scenario to modify Cinder volume metadata  https://review.openstack.org/174894
21:32 meteorfox boris-42: 95th percentile from Rally, shows about 1.4 seconds, while Gatling reports 259 ms
21:33 meteorfox boris-42: these are measuring the same operations, which is essentially  a POST to /v2.0/networks, except that Rally, uses the python-neutronclient
21:34 meteorfox boris-42: reposting, I had a typo in Max, Mean
21:35 meteorfox https://gist.github.com/meteorfox/cfc256a3631c78baacda
21:35 meteorfox these matches the neutron log
21:38 boris-42 meteorfox: soo it's quite interesting
21:39 boris-42 meteorfox: AHHH
21:40 boris-42 meteorfox:  I know where is the issue
21:40 boris-42 meteorfox: create_network is actually duration of (authentication + create_netowrk)
21:40 boris-42 meteorfox: you can easily check this create scenario that does next
21:41 boris-42 meteorfox: instead of those 2 lines https://github.com/openstack/rally/blob/master/rally/benchmark/scenarios/neutron/network.py#L69-L70
21:41 boris-42 meteorfox: put before them one more
21:42 boris-42 meteorfox:  self.clients("neutron")
21:42 boris-42 meteorfox: it's not python overhead it's keystone overhead
22:19 openstackgerrit Sergey Skripnick proposed openstack/rally: Add new script for rally-gate jobs  https://review.openstack.org/175549
22:19 pbandzi joined #openstack-rally
22:23 boris-42 meteorfox: so
22:24 boris-42 meteorfox: did you tried?
22:24 rook_ joined #openstack-rally
22:36 pbandzi_ joined #openstack-rally
22:37 jaypipes joined #openstack-rally
22:54 PerfBein_ joined #openstack-rally
22:58 meteorfox boris-42: I haven't tried it, but looking at the code from the python-neutronclient, seems it's just a post: https://github.com/openstack/python-neutronclient/blob/b978f909014cbc398c21bfae917fcece89a36e38/neutronclient/v2_0/client.py#L1205-L1207
22:59 boris-42 meteorfox: nope nope
22:59 boris-42 meteorfox: the reason is authentication
23:00 meteorfox boris-42: ok, so adding 'self.clients("neutron")' will make it authenticate first?
23:00 boris-42 meteorfox: yep
23:02 boris-42 meteorfox: just one line fix easy to test
23:02 meteorfox boris-42: ok I'll try it out tomorrow. How can we solve this generally? I was thinking we could collect the time at the lowest level, monkey patch or something to get just the http request/response time without auth, object marshalling/unmarshalling
23:02 meteorfox boris-42: I know. I have to go home now. My wife is waiting. :)
23:03 boris-42 meteorfox: =)
23:03 boris-42 meteorfox: see you
23:28 davideagnello boris-42: I am trying to use Rally's built-in osclients module and create_from_env(), but I am getting the following typeError exception:_init__() got an unexpected keyword argument 'admin_clients'
23:28 davideagnello in rally/rally/benchmark/runners/base.py
23:30 boris-42 davideagnello: hm
23:30 davideagnello boris-42: I have my sample tests which instantiate a keystone client itself, which worked fine.  I am trying to use Rally's built-in support for Keystone client and retrieve an authentication session object from it
23:30 boris-42 like here http://boris-42.me/the-simplest-way-to-use-openstack-python-clients/
23:31 boris-42 davideagnello: if you are trying to use inside the scenario create_from_env I can deff say that you are doing something wrong=)
23:31 davideagnello boris-42: ok, probably :)  I should provide this through a context?
23:32 boris-42 davideagnello: I am not sure what you are trying to do
23:33 davideagnello boris-42: so this is what my basic rally scenario plugin looks like right now:  https://review.openstack.org/#/c/175524/1/rally-scenarios/plugins/list_clusters.py
23:34 davideagnello I am instantiating a keystone client there, I am trying to use Rally's support for built-in osclients for keystone instead of instantiating my own
23:34 boris-42 davideagnello: use just osclients
23:35 boris-42 davideagnello: yep you shouldn't deff revinvent oslcients
23:35 davideagnello boris-42: yes, that is what I am trying to do, the issue is that our client (cue) is currently not in there
23:35 boris-42 davideagnello: so you can make simple patch in rally
23:36 boris-42 davideagnello: to add abbility to extend osclients outside of rally
23:36 boris-42 davideagnello: like register client
23:36 davideagnello boris-42: ok, is there a blueprint out for this feature?  would like to contribute for it
23:37 boris-42 davideagnello: we dislike beruacraccy
23:37 boris-42 davideagnello: so patch is enough
23:37 boris-42 davideagnello: but for now in your plugins
23:37 boris-42 davideagnello: just do next thing self.clients("keystone") -> this will return intialized plain keystone client
23:38 davideagnello boris-42: ok
23:38 boris-42 davideagnello: after that you can create session and init your client
23:39 davideagnello boris-42: was going to use this to get the keystone client:  ks = self.clients.keystone()  same?
23:39 boris-42 davideagnello: nope this won't work
23:39 boris-42 davideagnello: self.clients("keystone")
23:43 PerfBeing joined #openstack-rally
23:44 openstackgerrit Pavel Boldin proposed openstack/rally: Add function decorator `log_deprecated_args'  https://review.openstack.org/174453
23:44 openstackgerrit Pavel Boldin proposed openstack/rally: Refactor run_command_over_ssh, add script_args  https://review.openstack.org/173371
23:44 openstackgerrit Pavel Boldin proposed openstack/rally: Add `LogCatcher' context  https://review.openstack.org/174454
23:48 davideagnello boris-42: ok, why wouldn't that work?
23:48 exploreshaifali joined #openstack-rally
23:48 boris-42 why it should work?
23:49 boris-42 davideagnello: you are calling this method https://github.com/openstack/rally/blob/master/rally/benchmark/scenarios/base.py#L185
23:49 davideagnello boris-42: makes sense now
23:50 davideagnello boris-42: what endpoints would that keystone client use by default?
23:50 davideagnello boris-42: from env?
23:51 boris-42 davideagnello: nope
23:51 boris-42 davideagnello: rally creates own users/tenants
23:51 boris-42 davideagnello: on each iteration rally will pick randomly one user
23:51 davideagnello boris-42: what if the scenario needs a specific user?
23:52 boris-42 davideagnello: you mean that already exists in system?
23:52 davideagnello boris-42: yes, like admin?
23:52 boris-42 davideagnello: admin is accessable via self.admin_clients()
23:52 boris-42 davideagnello: and it is provided via "rally deployment create" stuff
23:53 boris-42 davideagnello: but in case if you need sepcifc user (that is not admin and already exist in system)
23:53 boris-42 davideagnello: then you need this patch https://review.openstack.org/#/c/168524/
23:55 davideagnello boris-42: ok
23:55 davideagnello boris-42: is there an example how the "rally deployment create" works?
23:56 boris-42 davideagnello: hm I can show plugin
23:56 boris-42 davideagnello: or how it looks for end user?
23:56 boris-42 davideagnello: https://rally.readthedocs.org/en/latest/tutorial/step_1_setting_up_env_and_running_benchmark_from_samples.html#registering-an-openstack-deployment-in-rally
23:57 PerfBeing joined #openstack-rally
23:57 davideagnello boris-42: oh ok, yes already done that
23:58 echoingumesh joined #openstack-rally
23:58 boris-42 davideagnello: so basically plugin has few methods and name
23:58 boris-42 davideagnello: and it's own args
23:58 openstackgerrit joined #openstack-rally
23:58 boris-42 davideagnello: the simplest one is existingcloud it does nothing
23:59 boris-42 davideagnello: https://github.com/openstack/rally/blob/master/rally/deploy/engines/existing.py#L134-L147
23:59 davideagnello when I create my own session, would I be able to use the Endpoint object values for auth.. session = ks.session.Session(auth=self.auth)

| Channels | #openstack-rally index | Today | | Search | Google Search | Plain-Text | summary