Perl 6 - the future is here, just unevenly distributed

IRC log for #openstack-rally, 2015-02-11

| Channels | #openstack-rally index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:35 yingjun joined #openstack-rally
01:23 mga_ joined #openstack-rally
01:23 mga_ morning!
01:30 boris-42 mga_: it's evening lol
01:30 boris-42 mga_: hi
01:32 mga_ hey, well, for you :) I'm working in UTC+8h
01:33 mga_ so, 9.30am in Taiwan
01:40 mga_ boris-42: it's 7:30pm in your city?
01:40 boris-42 mga_: it's 4:40 am)
01:40 klindgren_ joined #openstack-rally
01:40 mga_ wtf
01:41 boris-42 mga_: it's called a lot of to do=)
01:41 mga_ where are you? russia?
01:41 boris-42 mga_: yep Moscow
01:42 mga_ I see.. poor you.
01:42 mga_ andrey is sleeping, I guess?
01:43 mga_ well, then I don't bother you by chatting. good luck! may the coffee stream never end. :)
01:43 boris-42 mga_: ya for now he is sleeping
01:43 boris-42 mga_: how can I help you?
01:45 mga_ my problem relates to a cleanup problem in tempest. I saw in the logs yesterday, that andrey pushed a change to git. https://review.openstack.org/#/c/154408/
01:45 boris-42 mga_: ya that thing is really bad bad
01:45 boris-42 mga_: it purge your cloud
01:47 mga_ really? I haven't seen that problem, yet. my cloud is still there. The problem is rather minor. Cleanup is called, but it cant get credentials, so it ends with an error.
01:48 mga_ sometimes, testobjects are not deleted. they are still there after test or benchmark.
01:48 boris-42 mga_: as far as I know it deletes all resource from cloud
01:48 boris-42 mga_: like all VMs and so on
01:49 mga_ that could be.. my testing cloud is emtpy anyway. I'll test this.
01:55 boris-42 mga_: heh and we are trying to make rally safe to run against production
01:55 boris-42 mga_: btw why do you generate load with tempest?
02:02 mga_ oh, nice, indeed, my VMs are gone.. :/
02:07 mga_ No, I run rally task, And then, tempest has a cleanup error: see here the log: http://paste.openstack.org/show/171091/
02:09 boris-42 mga_: why do you run rally task that use tempest to generate lod
02:09 boris-42 mga_: ?
02:09 boris-42 load*
02:09 boris-42 mga_: it's very bad way to generate load (It was made to show that it is very very bad way)
02:11 mga_ mh,.. what is a better solution?
02:13 mga_ I had to do research on Tempest, this thing doesn't work nicely, then I discovered Rally. Now i'm testing Rally with Tempest. What do you recommend?
02:19 boris-42 mga_: so in rally you have command "rally verify" that simplifies uage of tempst
02:19 boris-42 mga_: rally task start (that runs tempest scenario) is some kind of dirty hacks
02:20 boris-42 mga_: to run N instances of tempest simultenously to generate load
02:25 yingjun joined #openstack-rally
02:26 mwagner_lap joined #openstack-rally
02:26 mga_ yes, but In my opinion, verify runs only simple unit-tests, but no scenarios. I look for a way to create and run scenarios and generate load.
02:27 boris-42 mga_: HM??
02:28 boris-42 mga_: tempest is not about unit testing
02:28 boris-42 mga_: it is functioanl testing
02:28 boris-42 mga_: and rally runs it in rally verify command
02:30 mga_ sorry, you're right. I mean simple funtional tests like create and delete via API
02:31 boris-42 mga_: scenarios are the same simple scenarios
02:31 boris-42 mga_: I mean in directory "tempest/stress"
02:31 boris-42 mga_: they are just run in parallel
02:31 boris-42 mga_: actually the same thing you can do via rally=)
02:32 boris-42 mga_: without tempest=)
02:32 mga_ what I'm looking for is more complex scenario, not only restricted on an openstack component like "running all nova API tests" but also "run nova, create vm, do snapshot, make some ceilometer stuff and after all, shutdown and close verything properly. 100 times, for 50 users. that would be awesome.
02:33 boris-42 mga_: so why not using rally framework for that? Are you somehow limited?
02:33 mga_ but rally use tempest always, doesn't it? there's no way around.
02:34 boris-42 mga_: nope
02:35 boris-42 mga_: task that you run (that generates load with tempest) is done via rally framework
02:35 boris-42 mga_: so "rally verify' - runs tempest
02:35 boris-42 mga_: "rally task" - is fully rally framework based
02:35 mga_ I'm not sure whether I missed some wikipages to read..
02:36 boris-42 mga_: and there is one scenario that calls tempest in parallel
02:36 mga_ rally task start rally/samples/tasks/scenarios/tempest/single_test.json is  a task in a tempest folder
02:36 mga_ so where are the rally tasks?
02:37 boris-42 mga_: all other directories
02:37 boris-42 mga_: in  rally/samples/tasks/scenarios/*
02:37 mga_ oh..
02:38 boris-42 mga_: so tempest in scenarios is just few "benchmarks"
02:38 boris-42 mga_: to show that the framework is so flexible that it can genereate load even using tempest=)
02:39 mga_ :D okay
02:39 boris-42 mga_: so and that thin sux
02:39 mga_ I see
02:39 boris-42 mga_: if you want other scenarios (like you said)
02:39 boris-42 mga_:  you can create rally plugin for that
02:40 boris-42 mga_: https://rally.readthedocs.org/en/latest/plugins.html
02:40 boris-42 mga_: here is the wiki page related to that
02:43 mga_ but it doesn't exist so far? I mean a way to create a user story as a procedure,  I can run on rally? So handing over UUIDs from one task to another and clean up the whole thing later?
02:46 boris-42 mga_: I don't understand you
02:46 boris-42 mga_: Rally is framework to create such procedueres
02:46 boris-42 mga_: and it even supports plugins so you can easily write exactly what you need
02:46 boris-42 ,mga_ for example boot server from volume and delete https://github.com/stackforge/rally/blob/master/rally/benchmark/scenarios/nova/servers.py#L112-L138
02:47 boris-42 mga_: so you can find a lot of already scenarios and combine them with context classes and runners and their arugmnets
02:47 boris-42 mga_: plus you can code own stuff as a plugins
02:48 mga_ okay,  then I'll have a close look on it. haven't considered, that tempest is only one way to use rally.
02:49 boris-42 mga_: if we remove code "rally verify" it's really small part of rally)
02:49 boris-42 mga_: so "rally task" is ubsolutelly unrelated to tempest
02:51 mga_ ahh. ok. thanks for your time =) then I'll go through the code with a different view.
02:54 boris-42 mga_: yep forgot about tempest=)
05:02 klindgren__ joined #openstack-rally
05:05 rdas joined #openstack-rally
05:17 neeti joined #openstack-rally
05:33 tfreger joined #openstack-rally
05:59 nkhare joined #openstack-rally
07:11 nmagnezi joined #openstack-rally
07:22 mga_ joined #openstack-rally
07:40 Miouge joined #openstack-rally
07:41 msdubov joined #openstack-rally
07:42 exploreshaifali joined #openstack-rally
07:57 nkhare joined #openstack-rally
08:07 exploreshaifali joined #openstack-rally
08:10 e0ne joined #openstack-rally
08:21 openstackgerrit Mikhail Dubov proposed stackforge/rally: Stop scenario execution on SLA failure  https://review.openstack.org/152459
08:25 nmagnezi_ joined #openstack-rally
08:28 oanufriev joined #openstack-rally
08:31 openstackgerrit Mikhail Dubov proposed stackforge/rally: Stop scenario execution on SLA failure  https://review.openstack.org/152459
08:33 openstackgerrit Mikhail Dubov proposed stackforge/rally: Stop scenario execution on SLA failure  https://review.openstack.org/152459
08:35 openstackgerrit Roman Vasilets proposed stackforge/rally: Add murano benchmarks  https://review.openstack.org/137661
08:35 openstackgerrit Roman Vasilets proposed stackforge/rally: Add murano base for murano benchmarks  https://review.openstack.org/137650
08:37 andreykurilin_ joined #openstack-rally
08:41 openstackgerrit Kairat Kushaev proposed stackforge/rally: Add update + add/delete resource cases  https://review.openstack.org/152886
08:41 openstackgerrit Kairat Kushaev proposed stackforge/rally: Add "update-inplace/replace resource" cases  https://review.openstack.org/152885
08:41 nkhare joined #openstack-rally
08:53 andreykurilin_ joined #openstack-rally
08:55 andreykurilin_ joined #openstack-rally
08:59 openstackgerrit Andrey Kurilin proposed stackforge/rally: Remove all non-incubated oslo code  https://review.openstack.org/152847
09:01 nmagnezi joined #openstack-rally
09:04 andreykurilin_ mga_: hi
09:04 andreykurilin_ mga_: do you have any questions?
09:15 arxcruz joined #openstack-rally
09:17 openstackgerrit Mikhail Dubov proposed stackforge/rally: Stop scenario execution on SLA failure  https://review.openstack.org/152459
09:20 mga_ joined #openstack-rally
09:21 mga_ hi andreykurilin. boris recommended me to not use tempest scenarios. since then, I didn't have problems :P
09:21 abehl joined #openstack-rally
09:22 mga_ But two feature questions I have:
09:22 amaretskiy joined #openstack-rally
09:24 mga_ 1) my openstack machine is limited on resources, so when I run a scenario, sometimes, there were error messages. These messages were not displayed to me and the scenario finishes with 50% success. After the run, I didn't have the chance to get any details or logs from the run. the only option was to rerun the task using -v verbose mode. is there no other way?
09:28 mga_ 2) Using "rally info find" I can look for descriptions of the task, but from this description, there is no direct way, to copy something, to run this scenario directly. I have to know about the folder structure. Is it possible, that I can use the scenario namespace directly? Like "rally task start  NovaServers.boot_and_bounce_server" ?
09:38 openstackgerrit Roman Vasilets proposed stackforge/rally: Add murano benchmarks  https://review.openstack.org/137661
09:41 psd joined #openstack-rally
09:45 e0ne joined #openstack-rally
09:53 andreykurilin_ mga_: 1) please look at "rally task report" command. It can generate html page for you. in this report you can found all errors for each iteration.
09:54 andreykurilin_ mga_: also, "rally task results" can help you
09:54 kairat_kushaev joined #openstack-rally
09:56 andreykurilin_ mga_: no, you cant launch rally like "rally task start  NovaServers.boot_and_bounce_server". Each sceanario can be and should be configured. Rally provides samples which helps to understand, how scenarios can be configured
09:59 nmagnezi joined #openstack-rally
10:01 openstackgerrit Mikhail Dubov proposed stackforge/rally: Stop scenario execution on SLA failure  https://review.openstack.org/152459
10:06 nkhare joined #openstack-rally
10:06 openstackgerrit Vitaly Gusev proposed stackforge/rally: Add rally test for nova notifications  https://review.openstack.org/154803
10:15 openstackgerrit Kairat Kushaev proposed stackforge/rally: Add context that pre-creates heat stacks  https://review.openstack.org/154438
10:15 openstackgerrit Kairat Kushaev proposed stackforge/rally: Add list-stack-with-resources scenario  https://review.openstack.org/154463
10:21 openstackgerrit Roman Vasilets proposed stackforge/rally: Add murano benchmarks  https://review.openstack.org/137661
10:34 msdubov mga_: You can take a look at these samples: https://github.com/stackforge/rally/tree/master/samples/tasks/scenarios
10:35 tfreger joined #openstack-rally
10:37 openstackgerrit Mikhail Dubov proposed stackforge/rally: Stop scenario execution on SLA failure  https://review.openstack.org/152459
10:57 openstackgerrit Nir Magnezi proposed stackforge/rally: Adds run_bash_command function, to vm utils.  https://review.openstack.org/154825
11:00 openstackgerrit Olga Kopylova proposed stackforge/rally: Add aliases "rally task use" and "rally deployment use"  https://review.openstack.org/154471
11:02 openstackgerrit Oleh Anufriiev proposed stackforge/rally: Don't create task db record during validation  https://review.openstack.org/153037
11:09 oanufriev1 joined #openstack-rally
11:09 e0ne_ joined #openstack-rally
11:11 rvasilets_ joined #openstack-rally
11:16 openstackgerrit Mikhail Dubov proposed stackforge/rally: Stop scenario execution on SLA failure  https://review.openstack.org/152459
11:20 exploreshaifali joined #openstack-rally
11:20 openstackgerrit Merged stackforge/rally: Add create-update-delete stack scenario  https://review.openstack.org/152557
11:29 ilbot3 joined #openstack-rally
11:29 Topic for #openstack-rally is now ☁ Rally RoadMap: http://goo.gl/JZkmwY ☁ Rally IRC chat logs http://irclog.perlgeek.de/openstack-rally ☁ Key persons to ask:  boris-42, msdubov, rediskin, andreykurilin, amaretskiy  ☁ To publish changes to Rally:  https://wiki.openstack.org/wiki/Rally/Develop#How_to_contribute   ☁ Rally weekly updates:  https://wiki.openstack.org/wiki/Rally/Updates
11:32 openstackgerrit Nir Magnezi proposed stackforge/rally: Modifies _boot_server() to handle instances with multiple networks.  https://review.openstack.org/154833
11:39 openstackgerrit Nir Magnezi proposed stackforge/rally: Modifies _boot_server() to handle instances with multiple networks.  https://review.openstack.org/154833
11:40 yfried_ joined #openstack-rally
11:43 openstackgerrit Nir Magnezi proposed stackforge/rally: Adds run_bash_command function, to vm utils  https://review.openstack.org/154825
11:44 openstackgerrit Nir Magnezi proposed stackforge/rally: Modifies _boot_server() to handle instances with multiple networks  https://review.openstack.org/154833
11:45 nmagnezi joined #openstack-rally
11:48 Miouge_ joined #openstack-rally
11:48 cdent joined #openstack-rally
12:01 psd joined #openstack-rally
12:06 openstackgerrit Andrey Kurilin proposed stackforge/rally: Remove all non-incubated oslo code  https://review.openstack.org/152847
12:12 nkhare joined #openstack-rally
12:17 exploreshaifali joined #openstack-rally
12:24 Miouge joined #openstack-rally
12:28 psd_ joined #openstack-rally
12:28 boris-42 joined #openstack-rally
12:31 tellesnobrega hi, does the create-and-list-tenant.json works with keystone v3?
12:38 nkhare joined #openstack-rally
12:44 openstackgerrit Andrey Kurilin proposed stackforge/rally: Remove all non-incubated oslo code  https://review.openstack.org/152847
12:48 openstackgerrit Andrey Kurilin proposed stackforge/rally: Remove all non-incubated oslo code  https://review.openstack.org/152847
12:49 Miouge joined #openstack-rally
12:49 boris-42 tellesnobrega: nope
12:49 boris-42 tellesnobrega: it's only for keystone v2
12:50 tellesnobrega boris-42, thanks
12:50 boris-42 tellesnobrega: probably we need special set of benchamrks
12:50 boris-42 tellesnobrega: for Keystone V3
12:50 tellesnobrega boris-42, would be a good idea
12:51 boris-42 tellesnobrega: wanna work on this?
12:52 tellesnobrega boris-42, i wish i could. but i dont have the time right now, swamped with sahara and university work
12:53 boris-42 tellesnobrega: it's actually Quite simple
12:53 boris-42 tellesnobrega: just few lines of code
12:53 tellesnobrega boris-42, authenticate and create-and-delete-user.json worked fine with v3
12:53 boris-42 tellesnobrega: tenants were removed from keystone v3
12:54 tellesnobrega boris-42, the idea is just change the api to use project api?
12:59 boris-42 tellesnobrega: nope
12:59 boris-42 tellesnobrega: just to have keystone v3 benchmarks
12:59 boris-42 kairat_kushaev: that will init keystone v3 client
13:00 openstackgerrit Andrey Kurilin proposed stackforge/rally: Remove all non-incubated oslo code  https://review.openstack.org/152847
13:01 tellesnobrega boris-42, i see, well. I will try to find some time to do it
13:02 boris-42 tellesnobrega: let me point you to the code
13:02 tellesnobrega sure
13:03 nmagnezi joined #openstack-rally
13:03 mkoderer_cloud joined #openstack-rally
13:03 openstackgerrit Andrey Kurilin proposed stackforge/rally: Remove all non-incubated oslo code  https://review.openstack.org/152847
13:06 tfreger left #openstack-rally
13:12 tfreger joined #openstack-rally
13:22 jjmb1 joined #openstack-rally
13:22 boris-42 tellesnobrega: https://github.com/stackforge/rally/blob/master/rally/benchmark/scenarios/keystone/basic.py#L21 here is the code
13:22 boris-42 tellesnobrega: for current keystone benchmarks
13:22 boris-42 tellesnobrega: so you need to do similar for keystone v3
13:23 tellesnobrega boris-42, ok
13:23 tellesnobrega boris-42, i will try to do it as fast as i can
13:45 e0ne joined #openstack-rally
13:55 rook joined #openstack-rally
13:58 nmagnezi boris-42, hi
14:03 openstackgerrit Andrey Kurilin proposed stackforge/rally: Remove all non-incubated oslo code  https://review.openstack.org/152847
14:09 mwagner_lap joined #openstack-rally
14:18 openstackgerrit Nir Magnezi proposed stackforge/rally: Modifies NeutronWrapper class and Network context with multiple nets and IPv6  https://review.openstack.org/154885
14:25 rook joined #openstack-rally
14:41 nmagnezi andreykurilin, hi there
14:43 openstackgerrit Andrey Kurilin proposed stackforge/rally: Remove all non-incubated oslo code  https://review.openstack.org/152847
14:45 openstackgerrit Wataru Takase proposed stackforge/rally: Add EC2 API test scenario  https://review.openstack.org/147550
14:48 andreykurilin_ nmagnezi: hi
14:50 nmagnezi andreykurilin, i'm working on some modifications to NetworkWrapper. stuff related to IPv6 and multiple networks in benchmark scenarios.
14:50 nmagnezi andreykurilin, i've added you as reviewed for one of the patches (modifications to _boot_server)
14:50 nmagnezi as reviewer*
14:51 andreykurilin_ nmagnezi: yeah, I saw your request
14:52 andreykurilin_ nmagnezi: I'll do a review a little bit later
14:52 psd joined #openstack-rally
14:53 nmagnezi andreykurilin, thanks! is it ok to put you as reviewer for the test of this work? I was told that you handle networking stuff in Rally :)
14:54 openstackgerrit Ivan Kolodyazhny proposed stackforge/rally: Implement create_and_upload_volume_to_image scenario for Cinder  https://review.openstack.org/152670
14:58 andreykurilin_ nmagnezi: heh)
15:03 stannie joined #openstack-rally
15:30 e0ne joined #openstack-rally
15:38 exploreshaifali joined #openstack-rally
15:40 dmellado hey here, is there any way to tell rally to skip tempest tests when launching them?
15:44 openstackgerrit Mikhail Dubov proposed stackforge/rally: Mention Rally ReadTheDocs in README  https://review.openstack.org/154447
15:57 openstackgerrit Nir Magnezi proposed stackforge/rally: Modifies NeutronWrapper class and Network context with multiple nets and IPv6  https://review.openstack.org/154885
16:10 andreykurilin_ joined #openstack-rally
16:11 anshul joined #openstack-rally
16:20 andreykurilin_ joined #openstack-rally
16:37 openstackgerrit Pavel Boldin proposed stackforge/rally: Add FIO disk benchmark for VMs  https://review.openstack.org/152388
16:37 openstackgerrit Pavel Boldin proposed stackforge/rally: Add the generic script-based benchmark for the VMs  https://review.openstack.org/141671
16:37 openstackgerrit Pavel Boldin proposed stackforge/rally: Add floating-IP aware VM boot/delete utils  https://review.openstack.org/151384
16:37 openstackgerrit Pavel Boldin proposed stackforge/rally: Add the context benchmark_image  https://review.openstack.org/138466
16:37 openstackgerrit Pavel Boldin proposed stackforge/rally: Store keypair and secgroup data on per-user basis  https://review.openstack.org/149771
16:55 coolsvap joined #openstack-rally
16:57 jjmb joined #openstack-rally
17:03 openstackgerrit Roman Vasilets proposed stackforge/rally: Add murano benchmarks  https://review.openstack.org/137661
17:03 openstackgerrit Roman Vasilets proposed stackforge/rally: Add murano base for murano benchmarks  https://review.openstack.org/137650
17:19 psd joined #openstack-rally
17:30 openstackgerrit Alexander Maretskiy proposed stackforge/rally: (WIP) Add chunks support and chart `Parallel iterations' to report  https://review.openstack.org/146814
17:47 exploreshaifali joined #openstack-rally
17:58 openstackgerrit Kairat Kushaev proposed stackforge/rally: Add context that pre-creates heat stacks  https://review.openstack.org/154438
17:58 openstackgerrit Kairat Kushaev proposed stackforge/rally: Add list-stack-with-resources scenario  https://review.openstack.org/154463
17:58 psd_ joined #openstack-rally
18:05 masa_ joined #openstack-rally
18:18 masa__ joined #openstack-rally
18:22 masa__ left #openstack-rally
18:31 masa__ joined #openstack-rally
19:02 mwagner_lap joined #openstack-rally
19:07 psd_ joined #openstack-rally
19:17 tellesnobrega joined #openstack-rally
20:00 afazekas joined #openstack-rally
20:04 psd_ joined #openstack-rally
20:06 ningy joined #openstack-rally
20:06 ningy anyone noticed the ssl error when using rally to run authenticate test for validate_heat?
20:07 ningy looks like the test is looking for ca-certificate.crt in system /etc/ssl/certs location instead of the location specified in rally.conf file
20:08 ningy is there an existing bug for this?
20:12 e0ne joined #openstack-rally
20:58 psd_ joined #openstack-rally
21:49 masa___ joined #openstack-rally
21:50 cdent joined #openstack-rally
21:51 exploreshaifali joined #openstack-rally
22:04 masa__ joined #openstack-rally
22:11 jjmb joined #openstack-rally
22:48 cdent joined #openstack-rally
23:22 openstackgerrit Mikhail Dubov proposed stackforge/rally: Stop scenario execution on SLA failure  https://review.openstack.org/152459
23:40 psd_ joined #openstack-rally
23:59 openstackgerrit Mikhail Dubov proposed stackforge/rally: Fix rally task detailed --iterations-data  https://review.openstack.org/155114

| Channels | #openstack-rally index | Today | | Search | Google Search | Plain-Text | summary