Perl 6 - the future is here, just unevenly distributed

IRC log for #openstack-rally, 2015-02-19

| Channels | #openstack-rally index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:32 dmorita joined #openstack-rally
02:49 ilbot3 joined #openstack-rally
02:49 Topic for #openstack-rally is now ☁ Rally RoadMap: http://goo.gl/JZkmwY ☁ Rally IRC chat logs http://irclog.perlgeek.de/openstack-rally ☁ Key persons to ask:  boris-42, msdubov, rediskin, andreykurilin, amaretskiy  ☁ Documentation: https://rally.readthedocs.org/en/latest/ ☁ To publish changes to Rally:  https://rally.readthedocs.org/en/latest/contribute.html
04:15 psd joined #openstack-rally
04:30 nkhare joined #openstack-rally
04:47 himangi joined #openstack-rally
05:13 rdas joined #openstack-rally
05:35 psd_ joined #openstack-rally
05:53 exploreshaifali joined #openstack-rally
06:23 pboldin joined #openstack-rally
06:41 psd__ joined #openstack-rally
06:52 psd joined #openstack-rally
06:56 pboldin joined #openstack-rally
07:07 openstackgerrit Pavel Boldin proposed stackforge/rally: Add FIO disk benchmark for VMs  https://review.openstack.org/152388
07:07 openstackgerrit Pavel Boldin proposed stackforge/rally: Add the generic script-based benchmark for the VMs  https://review.openstack.org/141671
07:07 openstackgerrit Pavel Boldin proposed stackforge/rally: Add the context benchmark_image  https://review.openstack.org/138466
07:07 pboldin addressed review comments
07:07 pboldin please rereview :)
07:27 anshul joined #openstack-rally
07:32 neeti joined #openstack-rally
07:39 nmagnezi joined #openstack-rally
08:07 psd_ joined #openstack-rally
08:08 arxcruz joined #openstack-rally
08:08 exploreshaifali joined #openstack-rally
08:22 amaretskiy joined #openstack-rally
08:29 andreykurilin_ joined #openstack-rally
08:35 openstackgerrit Kairat Kushaev proposed stackforge/rally: Add create and check stack scenario  https://review.openstack.org/155685
08:35 openstackgerrit Kairat Kushaev proposed stackforge/rally: Add create-suspend-resume stack scenario  https://review.openstack.org/155346
08:35 openstackgerrit Kairat Kushaev proposed stackforge/rally: Add list-stack-with-resources scenario  https://review.openstack.org/154463
08:39 tfreger joined #openstack-rally
08:39 tfreger left #openstack-rally
08:45 kairat_kushaev Hello, guys! Could you help me with me with some questions regarding rally philosophy?
08:45 kairat_kushaev If I understand correctly rally is a benchmark framework.
08:46 kairat_kushaev How it turned out that a lot of project specific tests were contributed to rally=)
08:46 kairat_kushaev Just curious
08:46 kairat_kushaev And why do we need such approach?
08:58 stannie joined #openstack-rally
09:14 openstackgerrit Ravikumar Venkatesan proposed stackforge/rally: Adds Nova keypair tests  https://review.openstack.org/156989
09:34 nkhare joined #openstack-rally
09:51 wtakase joined #openstack-rally
09:51 openstackgerrit Sergey Skripnick proposed stackforge/rally: Add new script for gate jobs  https://review.openstack.org/144633
09:54 andreykurilin kairat_kushaev: hi
09:56 kairat_kushaev andreykurilin: Hi
09:56 andreykurilin kairat_kushaev: python library "requests" was designed as "HTTP for humans". Rally has the same philosophy. We want to simplify usage of our product as much as possible.
09:58 andreykurilin kairat_kushaev: Rally contains a lot of scenarios, so end-users can just install Rally and start benchmarking without extra steps
10:13 kairat_kushaev andreykurilin: so supporting of rally samples and scenarios will be on rally-side
10:14 kairat_kushaev andreykurilin: and if rally will fail because of changes in one of components we need to wait for updates from component devs
10:16 andreykurilin kairat_kushaev: yes. we have such problems sometimes:( when it happens, we can two solutions: move failed scenarios to gate-rally-dsvm-neutron-unstable or move sla check, so gates will not failed due to failed scenario...
10:17 andreykurilin kairat_kushaev: also, rally supports plugins, so everything(scenarios, engines, context) can be delivered separately from rally repo
10:17 wtakase joined #openstack-rally
10:21 kairat_kushaev andreykurilin: Thanks for info! Yes, i know about this but I also can see a lot of tests in rally repo. It seems that community does not tend to use gates  a lot=). By the way, would you think about prohibiting of component changes at all in rally (like in other benchmark frameworks)? As an option, define separate repo for it?
10:21 boris-42 kairat_kushaev: nope we won't create separate repo
10:21 psd__ joined #openstack-rally
10:21 e0ne joined #openstack-rally
10:22 boris-42 kairat_kushaev: "why?" we should have 10 repos?
10:22 kairat_kushaev andreykurilin: I am asking because I contributed some code but i am not quite agree with current approach and would like to know reasons
10:22 boris-42 kairat_kushaev: with what parts you disagree?)
10:24 dmellado Hi, I'm getting an UnicodeDecodeError when trying to install tempest with rally-manage. I guess it'll have to do with my connection, but could anyone has a look at
10:24 kairat_kushaev boris-42: Why do we need component specific code in benchmark?) And why do I need to write additional tests to performance test scenario? If smth will change i need to make changes in func tests in component repo, in rally scenarios and rally unittests.
10:24 dmellado http://paste.openstack.org/show/QkskMtceX5o6kk48L8rd/
10:24 dmellado ?
10:25 boris-42 kairat_kushaev: to many questions..
10:25 boris-42 kairat_kushaev: let's start
10:25 boris-42 "Why you are writting unit tests for performance tests? " is the first?
10:25 boris-42 kairat_kushaev: ^
10:25 andreykurilin kairat_kushaev: prohibit of component changes? you mean use stable branches of OpenStack?
10:25 andreykurilin dmellado: hi
10:26 andreykurilin dmellado: looks like,  you trying to use old version of rally..
10:26 dmellado hi andreykurilin
10:26 dmellado mmmm it shouldn't, just cloned from the repo
10:26 dmellado let me check the git log
10:27 kairat_kushaev boris-42: ok, let this be the first:)
10:27 dmellado I'll reinstall it, just in case
10:27 dmellado but the git log's from yesterday
10:27 boris-42 kairat_kushaev: this is very important part
10:28 boris-42 kairat_kushaev: unit tests allows us to simplify development. Like during big refactorings, or changes in libs and so on
10:28 andreykurilin dmellado: sorry
10:29 andreykurilin dmellado: my fault:)
10:29 boris-42 kairat_kushaev: + when we are running unittests against python34 and functioanl tests against python27 we can assume
10:29 dmellado np andreykurilin :)
10:29 boris-42 kairat_kushaev: that maybe rally will work against python34
10:29 andreykurilin dmellado: it can occur in master
10:29 boris-42 kairat_kushaev: + as far as I saw for latests 2 years unit test has great impact on code quality
10:30 dmellado should i clone another branch/tag?
10:30 boris-42 kairat_kushaev: that is why we are writing unit tests
10:31 boris-42 kairat_kushaev: makes sense?
10:31 andreykurilin dmellado: no. we should find a fix for this bug
10:32 andreykurilin dmallado: btw, you can report it to launchpad ;)
10:32 andreykurilin dmellado:^
10:32 dmellado I'll do!
10:32 dmellado I've seen that already reported
10:32 dmellado so I'll just reopen the old one
10:33 andreykurilin dmellado: if you need hot-fix, I know it:)
10:33 dmellado oh, that'd be great
10:34 boris-42 kairat_kushaev: ?
10:36 pboros joined #openstack-rally
10:36 kairat_kushaev boris-42: I understand the goal of unit-tests but why do we need them for test scenarios? If somth was broken will scenario itself say that the test has broken?
10:36 kairat_kushaev boris-42: Just curious=)
10:38 boris-42 kairat_kushaev: I agree that we have good functional CI
10:38 boris-42 kairat_kushaev: and it will catch if scenario doesn't work
10:38 boris-42 kairat_kushaev: but one more time you can put python34 syntax error in it
10:39 boris-42 kairat_kushaev: it will pass functional CI for python2.7 and it will pas unit tests for python34 (cause you don't have unit tests for that particula scenario)
10:40 boris-42 kairat_kushaev: in case of having unit tests for this such code won't be merged
10:41 boris-42 kairat_kushaev: it doesn't give us 100% insurance but it's much bigger coverage
10:42 oanufriev joined #openstack-rally
10:44 kairat_kushaev boris-42: Thanks for feedback. Need to think more about it=)
10:44 boris-42 kairat_kushaev: I mean the biggest issue is next
10:44 boris-42 kairat_kushaev: we can't test everything functional in any combinations
10:45 boris-42 kairat_kushaev: like running all becnhmakrs against all OpenStack configurations against all python versions and all DB backends
10:45 boris-42 kairat_kushaev: so we need to find optimal testing matrix
10:45 boris-42 kairat_kushaev: that won't use too much resources and will be enough fast
10:45 boris-42 kairat_kushaev: so let's go to another questions
10:46 boris-42 kairat_kushaev: " Why do we need component specific code in benchmark?) " could you elobarate this?
10:46 boris-42 kairat_kushaev: what do you mean by component specific code?
10:51 kairat_kushaev boris-42: Ok, I mean test performance scenarios. What is the reason for them to be part of rally framework? You know that openstack is changing a lot and in this case these changes also affect rally.
10:52 kairat_kushaev boris-42: what is your opinion about it? I think that gates can be used a little bit more but I see that they are not because of some reason.
10:53 boris-42 kairat_kushaev: oh huge topic =)
10:53 boris-42 kairat_kushaev: first thing is speed of development
10:53 boris-42 kairat_kushaev: and the thing that only recently we started making releases
10:54 boris-42 kairat_kushaev: It means that I didn't want to block development of framework (so there were changes in it that required changes in benchmarks)
10:54 boris-42 kairat_kushaev: without having them in single repo it will be pain in ass to do
10:55 boris-42 kairat_kushaev: the second thing is that thing that framework and tests are in single repo doesn't mean that scenarios are part of framework
10:55 boris-42 kairat_kushaev: they are just in tree plugins
10:56 boris-42 kairat_kushaev: for benchmark engine there is no difference do you put in tree plugin or put in ~/.rally/plugins dir modules
10:57 boris-42 kairat_kushaev: the third reason is that Rally should be simple for Operators and QA and Devs
10:57 pboros_ joined #openstack-rally
10:57 boris-42 kairat_kushaev: simplicty means just run it and it works*
10:57 boris-42 kairat_kushaev: and adding 10 repost for plugins (is possible) but makes for end users troubles
10:58 exploreshaifali joined #openstack-rally
10:59 boris-42 kairat_kushaev: the forth reason is that Rally is going to maintain "parametrized cloud certification task" so single task will accept few arguments (images, flavors, amount of controlers, computes, ...) and generate proper big task that validates all services under proper load and criteria of success
11:00 boris-42 kairat_kushaev: that fifth reason when we are contributing to Rally we see limitations of framework and issues in architecture (so Rally team becomes first user of Rally framework) which is important for health
11:00 boris-42 of project
11:00 boris-42 kairat_kushaev: as well keeping in repo all utils allow you to write fast new benchmarks
11:01 boris-42 kairat_kushaev: just combining few methods that do a lot of stuff you are getting fast scenarios that you need
11:01 boris-42 kairat_kushaev: and sixth we don't force anybody to contribute to Rally repo, if you would like to have own repo with own benchmarks you are welcome
11:02 kairat_kushaev boris-42: Thanks for responses. Now I understand the architecture reasons more clearly=)
11:04 aix joined #openstack-rally
11:08 psd__ joined #openstack-rally
11:26 openstackgerrit Pavel Boldin proposed stackforge/rally: Add FIO disk benchmark for VMs  https://review.openstack.org/152388
11:26 openstackgerrit Pavel Boldin proposed stackforge/rally: Add the generic script-based benchmark for the VMs  https://review.openstack.org/141671
11:39 oanufriev joined #openstack-rally
11:40 msdubov_ joined #openstack-rally
11:56 rdas joined #openstack-rally
12:09 psd_ joined #openstack-rally
12:40 cdent joined #openstack-rally
12:43 openstackgerrit Roman Vasilets proposed stackforge/rally: Adding Multiple Request Scenario  https://review.openstack.org/117705
12:58 exploreshaifali joined #openstack-rally
12:59 msdubov_ joined #openstack-rally
13:06 psd__ joined #openstack-rally
13:32 openstackgerrit Vitaly Gusev proposed stackforge/rally: Add rally tests for Ceilometer  https://review.openstack.org/153994
13:37 openstackgerrit Merged stackforge/rally: Add aliases "rally task|deployment|verify use"  https://review.openstack.org/154471
13:45 nkhare joined #openstack-rally
13:48 yfried_ boris-42: were you looking for me?
14:01 aix joined #openstack-rally
14:10 nmagnezi joined #openstack-rally
14:22 psd_ joined #openstack-rally
14:33 psd boris-42, Regarding the NeutronSecgroup, you were saying everything will be similar to NovaSecGroup except : https://github.com/stackforge/rally/blob/master/rally/benchmark/scenarios/nova/security_group.py#L90
14:33 psd boris-42, so should I just delete this function?
14:37 boris-42 psd: I didn't say that it will be the same code
14:37 boris-42 psd: I said that it will cover the same functionallity
14:38 boris-42 psd: you shouldn't touch nova/security_group.py module
14:38 boris-42 psd: you should create new module in neutron
14:55 mwagner_lap joined #openstack-rally
15:05 exploreshaifali joined #openstack-rally
15:16 cdent joined #openstack-rally
15:21 nmagnezi joined #openstack-rally
15:25 msdubov_ joined #openstack-rally
15:27 boris-42 kairat_kushaev: could you please aovid changes in license header https://review.openstack.org/#/c/154463/9/rally/benchmark/scenarios/heat/utils.py ?
15:27 boris-42 kairat_kushaev: I mean it's not critical but just curious why?
15:35 frickler joined #openstack-rally
15:40 nmagnezi joined #openstack-rally
15:44 abhirc joined #openstack-rally
15:48 psd joined #openstack-rally
15:53 kairat_kushaev boris-42: It seems that pycharm auto-updates it because of some reason
15:54 kairat_kushaev boris-42: I'll check the settings because I didn't touch the headers
16:01 jaypipes joined #openstack-rally
16:05 himangi joined #openstack-rally
16:15 dpaterson joined #openstack-rally
16:20 nmagnezi joined #openstack-rally
16:25 openstackgerrit Jens Rosenboom proposed stackforge/rally: Fix security group setup when users_per_tenant is > 1  https://review.openstack.org/157433
16:29 kairat_kushaev left #openstack-rally
16:36 openstackgerrit joined #openstack-rally
16:40 verdurin joined #openstack-rally
16:41 verdurin I'm trying to run Rally on an RDO installation, and there have been some wrinkles.
16:41 verdurin Is it really intended for Ubuntu at the moment?
16:42 andreykurilin verdurin: hi. yes
16:43 verdurin Okay.
16:43 andreykurilin verdurin: what issue do you have?
16:43 verdurin andreykurilin: First one I've noticed is that the image name check in one of the sample
16:44 verdurin tasks is Ubuntu-specific
16:45 verdurin It looks for: ^cirros.*uec$
16:45 verdurin which fails with RDO.
16:46 andreykurilin verdurin: you can replace image name by image, which presents in your system:)
16:49 andreykurilin also, rally supports templates, so you can do like in our gates: https://github.com/stackforge/rally/blob/master/rally-jobs/rally.yaml#L867 and launch task as `rally task start task.yaml --task-args '{"image_name": "my_mega_image"}'`
16:49 andreykurilin verdurin: ^
16:49 verdurin andreykurilin: Ah, okay. Thanks.
16:49 verdurin It's my first time playing with Rally.
16:51 andreykurilin verdurin: sample dir include just samples. you can do with them whatever you want:)
16:51 verdurin I can certainly fix the image search string, but I wondered whether a more generic approach might be better.
16:53 psd boris-42, Yes, I have made a new module, my doubt is: does it make sense to make boot-and-delete-server-with-secgroup scenario for neutron?
16:54 andreykurilin verdurin: I don't know the way to write samples compatible for all versions of stacks. samples are only examples of how to be been configured tasks.
16:55 psd andreykurilin, can you help with ^^ ?
16:57 andreykurilin psd: hm... interested question
17:00 openstackgerrit Sergey Skripnick proposed stackforge/rally: Add new script for gate jobs  https://review.openstack.org/144633
17:01 andreykurilin psd: imo, no:)
17:01 psd andreykurilin, Thanks ;)
17:05 verdurin andreykurilin: the test worked with that small change, thanks.
17:09 andreykurilin verdurin: np
17:10 andreykurilin verdurin: feel free to ping me
17:10 * andreykurilin went home.
17:12 exploreshaifali joined #openstack-rally
17:13 afazekas joined #openstack-rally
17:17 openstackgerrit Jens Rosenboom proposed stackforge/rally: Fix security group setup when users_per_tenant is > 1  https://review.openstack.org/157433
17:21 openstackgerrit Mikhail Dubov proposed stackforge/rally: [WIP] Unify RPS and constant runners and tune constant runner  https://review.openstack.org/155225
17:23 openstackgerrit Sergey Skripnick proposed stackforge/rally: Add new script for gate jobs  https://review.openstack.org/144633
17:24 openstackgerrit Mikhail Dubov proposed stackforge/rally: [WIP] Unify RPS and constant runners and tune constant runner  https://review.openstack.org/155225
17:24 cdent joined #openstack-rally
17:29 boris-42 psd: nope you shouldn't do it
17:30 boris-42 psd: we will improve that benchmark
17:30 boris-42 psd: to support both
17:30 boris-42 verdurin: actually we should make all samples templated
17:30 boris-42 oh that will be huge work ...
17:31 boris-42 verdurin: btw do you know is there any chance to get rally automatically installed with RDO?)
17:36 openstackgerrit Prabhjyot Singh Sodhi proposed stackforge/rally: Adding Security group Scenarios for Neutron  https://review.openstack.org/157458
17:37 psd joined #openstack-rally
17:39 psd boris-42, ^, how do you want to support both?
17:41 boris-42 psd: step by step
17:41 boris-42 psd: let's work first on this one
17:41 psd Oh you meant in a different patch, sure :)
17:48 boris-42 psd: yep yep
17:49 boris-42 psd: baby steps
17:49 psd =)
17:58 amaretskiy1 joined #openstack-rally
18:04 psd_ joined #openstack-rally
18:09 harlowja joined #openstack-rally
18:35 psd joined #openstack-rally
19:06 e0ne joined #openstack-rally
19:31 himangi joined #openstack-rally
19:35 psd_ joined #openstack-rally
19:42 openstackgerrit Boris Pavlovic proposed stackforge/rally: Test novaclient  https://review.openstack.org/157502
19:49 jwang_ joined #openstack-rally
19:54 exploreshaifali joined #openstack-rally
20:00 psd boris-42, how many rally related ideas are you thinking of putting up at https://wiki.openstack.org/wiki/GSoC2015 ?
20:01 boris-42 psd: so for now two
20:01 boris-42 psd: your with info sutff
20:01 boris-42 psd: and exploreshaifali with task trends report
20:02 psd boris-42, oh nice :)
20:02 exploreshaifali boris-42, thanks :D
20:02 boris-42 psd: btw you should join #openstack-gsoc
20:03 psd boris-42, oh right, .. done!
20:14 jay_t_c joined #openstack-rally
20:14 jay_t_c boris-42: are you around today?
20:18 psd_ joined #openstack-rally
20:25 abhirc joined #openstack-rally
20:32 boris-42 jay_t_c: yep I am around
20:32 boris-42 jay_t_c: as usually 24/7 working
20:32 boris-42 time+)
20:33 boris-42 psd: so rally gates (and overall OpenStack gates are broken)
20:33 boris-42 pboros_: so we will need to wait unti things will be fixed
20:33 psd oh :(
20:35 jay_t_c boris-42: got everything set up for the bug i asked about yesterday regarding horizon and tempest config but looks like someone else grabbed it
20:36 jay_t_c boris-42: i am having trouble getting gerrit review to work from behind corporate proxy right now though
20:37 boris-42 jay_t_c: so it's not you https://bugs.launchpad.net/rally/+bug/1423358
20:37 boris-42 jay_t_c: heh ok
20:37 boris-42 jay_t_c: so I hope he will fix it soon...
20:38 jay_t_c boris-42: looks like someone named Nha Pham grabbed it
20:38 jay_t_c i hope also
20:38 boris-42 jay_t_c: so you can't run "git review -s"
20:38 jay_t_c boris-42: no
20:38 boris-42 jay_t_c: what it says?
20:39 jay_t_c boris-42: Exception: Could not connect to gerrit at ssh://jaycromer@review.openstack.org:29418/stackforge/rally.git
20:39 boris-42 jay_t_c: could you try to do next
20:40 boris-42 jay_t_c: ssh jaycorner@review.openstack.org
20:41 jay_t_c boris-42: ssh: Could not resolve hostname review.openstack.org: Name or service not known
20:44 boris-42 jay_t_c: can you ping it?
20:44 boris-42 jay_t_c: intersting intersting=)
20:44 jay_t_c boris-42: no, i am behind proxy
20:45 boris-42 jay_t_c: ah
20:45 boris-42 jay_t_c: heh..
20:54 psd_ joined #openstack-rally
21:01 boris-42 frickler: hi there
21:09 openstackgerrit Sergey Skripnick proposed stackforge/rally: Add new script for gate jobs  https://review.openstack.org/144633
21:17 abhirc joined #openstack-rally
21:38 openstackgerrit Sergey Skripnick proposed stackforge/rally: Add new script for gate jobs  https://review.openstack.org/144633
21:38 openstackgerrit Boris Pavlovic proposed stackforge/rally: Supress warning in cli methods that have sensetive output  https://review.openstack.org/157536
21:45 andreykurilin_ joined #openstack-rally
22:21 stannie left #openstack-rally
22:23 openstackgerrit Sergey Skripnick proposed stackforge/rally: Add new script for gate jobs  https://review.openstack.org/144633
22:26 exploreshaifali boris-42, what are we expecting final output of "generating trends from N tasks" ?
22:26 boris-42 уexploreshaifali html output that has trends
22:26 exploreshaifali that if a user use command *trend uudi1 uudi2 .. uuidN* he will able to see text result on terminal
22:27 exploreshaifali and for *trend report* the html output that has trends
22:27 exploreshaifali boris-42, ^
22:29 boris-42 exploreshaifali: nope
22:29 boris-42 exploreshaifali: there is no text ouput
22:29 boris-42 exploreshaifali: only HTML
22:29 exploreshaifali boris-42, :)
22:30 exploreshaifali boris-42, the parameters we will focus on - load duration, total duration, errors
22:37 exploreshaifali boris-42, for each parameter there will be a separate line
22:37 exploreshaifali ?
22:39 openstackgerrit Boris Pavlovic proposed stackforge/rally: Supress warning in cli methods that have sensetive output  https://review.openstack.org/157536
22:57 openstackgerrit Boris Pavlovic proposed stackforge/rally: Supress warning in cli methods that have sensetive output  https://review.openstack.org/157536
23:01 boris-42 ekarlso: nope just HTML
23:01 boris-42 ekarlso: oh sry
23:01 ekarlso boris-42: ...
23:01 boris-42 ekarlso: btw how are you?)
23:02 ekarlso boris-42: dont drink so much vodka, it confuses your eyes to type ek vs ex :P
23:02 andreykurilin_ ekarlso: :D
23:02 ekarlso boris-42: just got a baby so I'm doing superb :p
23:03 boris-42 ekarlso: actually I am typing only "e"
23:03 ekarlso even worse :D
23:04 boris-42 ekarlso: =)
23:56 abhirc joined #openstack-rally

| Channels | #openstack-rally index | Today | | Search | Google Search | Plain-Text | summary