Perl 6 - the future is here, just unevenly distributed

IRC log for #openstack-rally, 2015-04-01

| Channels | #openstack-rally index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:03 harlowja joined #openstack-rally
00:04 nkhare joined #openstack-rally
00:09 vipul boris-42: can rally be run as a library, where i just want to run a bunch of tests, don't care about the measurements, etc.
00:10 vipul or does it need to be installed as a full-blown service, with a db, etc.
00:10 boris-42 vipul: rally is not a service yet. You can run it from cli
00:11 boris-42 vipul: and use sqlite db backend (by default)
00:11 boris-42 vipul: so it is quite easy
00:11 boris-42 vipul: to use
00:11 boris-42 vipul: rally as a python lib is still in progress
00:12 boris-42 vipul: but it will collect the same data as rally cli collects (no difference)
00:12 boris-42 vipul: so just run install_rally.sh script and you are ready to go
00:12 vipul boris-42: Ok thanks.  this is good info.
00:12 boris-42 vipul: btw as well "rally task sla_check"
00:13 boris-42 will return code 1 if there are failures in sla
00:13 boris-42 and code 0 if everything passed
00:13 vipul boris-42: We are looking at adding integration tests to a stackforge project, and want something lightweight that address integration and scenario tests
00:13 boris-42 vipul: so it is rally =)
00:14 vipul instaed of just saying let's go with tempest.. want to look at both and pick one that solves both
00:14 boris-42 vipul: actually we are continue working on adding to rally assertations stuff
00:14 boris-42 let me find the patch
00:15 boris-42 like this https://review.openstack.org/#/c/146838/ vipul
00:16 vipul boris-42: yea this is great.. really would make rally more of a framework that provides the building blocks
00:19 vipul boris-42: any projects that are writing Rally plugins today?
00:23 yingjun joined #openstack-rally
00:24 dmorita joined #openstack-rally
00:28 openstackgerrit Merged stackforge/rally: Load OpenStack clients only if those are used  https://review.openstack.org/168887
00:28 openstackgerrit Merged stackforge/rally: [Sahara] Improved provisioning plugin support  https://review.openstack.org/167651
00:31 boris-42 vipul: so
00:32 boris-42 vipul: usually we are trying to merge them in rally source tree
00:32 boris-42 vipul: that simplify shipping to ops them
00:33 boris-42 vipul: these are scenarios that are in upstream https://github.com/stackforge/rally/tree/master/rally/benchmark/scenarios
00:33 boris-42 vipul: so a lot of projects is covered
00:34 vipul i asumme that is regarless of whether the project is stackforge vs openstack?
00:34 vipul also how does that impact the velocity, if all changes need ot be approved by the Rally team
00:34 boris-42 vipul: Rally team doesn't care
00:34 boris-42 vipul: so you can use in-tree rally plugins
00:34 boris-42 vipul: if you don't want to deal with rally team
00:35 vipul Ok got it..
00:35 boris-42 vipul: just put plugins in this dir https://github.com/openstack/cinder/tree/master/rally-jobs/plugins
00:35 boris-42 vipul: they will be autodiscovered in rally gate job for cinder patches
00:35 boris-42 vipul: but as I said Rally team is helping others to provide high quality well tested plugins
00:35 boris-42 vipul: that is our goal
00:36 vipul boris-42: great.. i think at least initially in-tree makes sense while we're getting started
00:39 boris-42 vipul: on what project are you working?
00:39 vipul https://github.com/stackforge/cue
00:40 boris-42 vipul: so does it have some HTTP api python clients and so on?
00:40 vipul boris-42: yep - python-cueclient
00:41 boris-42 vipul: so I can help you guys to setup dsvm job with rally
00:42 boris-42 so you will get such http://logs.openstack.org/24/169124/3/check/gate-rally-dsvm-cinder/37f4984/ results
00:42 vipul boris-42: Yea that would be awesome.. we
00:42 vipul we've got devstack plugins that should just work
00:42 boris-42 vipul: and this is HTML report http://logs.openstack.org/24/169124/3/check/gate-rally-dsvm-cinder/37f4984/rally-plot/results.html.gz
00:42 boris-42 vipul: okay that is the biggest art
00:42 boris-42 vipul: I can make infra job
00:42 vipul output is sweet!
00:43 boris-42 vipul: yep but it will be better in future=)
00:43 boris-42 vipul: actually I had to go to sleep it's 3:43 a.m.
00:43 boris-42 vipul: but in any case nice to meet you=)
00:44 vipul boris-42: yea you as well -- good night..
00:44 vipul we'll talk soon to get all this going
00:44 boris-42 vipul: yep sure
00:44 boris-42 vipul: it's actually quite simple if you have devstack plugin
00:44 boris-42 so see you=)
00:45 vipul ok
01:06 openstackgerrit Oleh Anufriiev proposed stackforge/rally: Removed task validation from api.Task.start  https://review.openstack.org/158899
01:15 panbalag joined #openstack-rally
01:39 mwagner_lap joined #openstack-rally
01:45 openstackgerrit pradeep proposed stackforge/rally: Add "suspend and resume" resume scenario for Nova  https://review.openstack.org/169150
02:19 mwagner_lap joined #openstack-rally
02:33 aswadr joined #openstack-rally
02:35 yingjun joined #openstack-rally
02:50 openstackgerrit pradeep proposed stackforge/rally: Add "pause and unpause" scenario for Nova  https://review.openstack.org/169598
03:47 nkhare joined #openstack-rally
04:08 openstackgerrit pradeep proposed stackforge/rally: Add "pause and unpause" scenario for Nova  https://review.openstack.org/169598
04:27 rvcehimanshu joined #openstack-rally
04:33 yingjun joined #openstack-rally
04:36 yingjun joined #openstack-rally
04:39 nkhare_ joined #openstack-rally
04:39 rdas joined #openstack-rally
04:45 tfreger joined #openstack-rally
04:48 rvcehimanshu joined #openstack-rally
05:39 garyk joined #openstack-rally
05:41 yingjun joined #openstack-rally
06:29 abehl joined #openstack-rally
06:39 _kiran_ joined #openstack-rally
06:41 kiran-r Hello fellow openstackers!
06:41 kiran-r =)
06:47 yfried joined #openstack-rally
06:50 yfried_ joined #openstack-rally
06:56 yfried_ boris-42: re https://review.openstack.org/#/c/154885/14
06:56 yfried_ boris-42: n-net has no structure that mirrors neutron routers
06:59 neeti joined #openstack-rally
07:07 openstackgerrit Mikhail Dubov proposed stackforge/rally: [Docs] Add tutorial chapter about task templates  https://review.openstack.org/168867
07:18 kiran-r Is there a hard and fast rule that the person who has been assigned in the blueprint should only upstream the code?
07:23 yfried_ kiran-r: not at all. any specific bp in mind?
07:24 kiran-r yes. Here is the blueprint whose code hasnt been upstreamed yet. https://blueprints.launchpad.net/rally/+spec/rally-lbaas-scenarios
07:24 yfried_ kiran-r: though it helps to make sure ppl aren't duplicating effort or doing conflicting work
07:24 kiran-r I have written the scenarios for loadbalancer. So Can I upstream it?
07:25 yfried_ kiran-r: I see no related patches. IMO you can feel free to push
07:25 yfried_ kiran-r: tag bp assignee as reviewer for your patches
07:25 yfried_ kiran-r: and tag me as well :)
07:27 kiran-r Sure.
07:27 kiran-r Also, I have submitted 2 more blueprints. May I get your approval as well?
07:27 kiran-r approval for the 2?
07:28 kiran-r https://blueprints.launchpad.net/rally/+spec/rally-neutron-metering-scenarios
07:29 kiran-r and this one https://blueprints.launchpad.net/rally/+spec/firewall-rally-scenarios
07:31 nkhare joined #openstack-rally
07:32 yfried_ kiran-r: I don't have approval privileges on LP. that's boris-42's thing
07:32 arxcruz joined #openstack-rally
07:32 yfried_ kiran-r: but you shouldn't worry about this
07:32 yfried_ kiran-r: go ahead and do your work.
07:33 kiran-r Thanks. =)
07:33 nmagnezi joined #openstack-rally
07:33 yfried_ kiran-r: since you are only extending scenarios, you don't really need to submit bp
07:33 yfried_ kiran-r: it only helps to connect and track multiple patches on the same subject
07:34 kiran-r oh okay. :)
07:39 aswadr joined #openstack-rally
07:44 yfried_ andreykurilin: re https://review.openstack.org/#/c/168849/4/rally/api.py,cm
07:44 yfried_ andreykurilin: I don't understand what you mean
08:12 asrangne joined #openstack-rally
08:15 yingjun joined #openstack-rally
08:16 e0ne joined #openstack-rally
08:16 nmagnezi boris-42, ping
08:17 rvcehimanshu joined #openstack-rally
08:19 amaretskiy joined #openstack-rally
08:33 aswadr joined #openstack-rally
08:36 kiran-r How do I fetch the arguments in rally input json/yaml file?
08:40 garyk1 joined #openstack-rally
08:45 garyk joined #openstack-rally
08:46 garyk joined #openstack-rally
08:47 rdas joined #openstack-rally
08:47 nmagnezi joined #openstack-rally
08:57 nkhare joined #openstack-rally
08:58 amaretskiy kiran-r hi
08:58 amaretskiy kiran-r please explain the problem - it's not clear
08:59 andreykurilin_ joined #openstack-rally
09:00 kiran-r In few testcases like create_and_delete_subnets, the subnet args like cidr, subnets per network etc are extracted from the json/yaml inout file. How to implement the same for other scenarios.
09:08 rvcehimanshu joined #openstack-rally
09:28 openstackgerrit Yair Fried proposed stackforge/rally: Adds endpoint to --fromenv deployment creation  https://review.openstack.org/166607
09:28 yfried_ boris-42: ^ please take a look
09:30 neeti joined #openstack-rally
09:40 amaretskiy kiran-r: imput "args" in scenario is scenario method's arguments, you can even use just **kwargs and get the kwargs dict with all specified data in input file "args"
09:40 openstackgerrit pradeep proposed stackforge/rally: Add "suspend and resume" scenario for Nova  https://review.openstack.org/169150
09:41 Miouge joined #openstack-rally
09:41 amaretskiy kiran-r: look at https://github.com/stackforge/rally/blob/master/rally/benchmark/scenarios/neutron/network.py#L76-L80
09:42 amaretskiy kiran-r also https://github.com/stackforge/rally/blob/master/rally/benchmark/scenarios/nova/servers.py#L45-L46
09:44 kiran-r How are the kwargs resolved?
09:44 kiran-r Thanks.
09:58 neeti joined #openstack-rally
09:59 openstackgerrit pradeep proposed stackforge/rally: Add "pause and unpause" scenario for Nova  https://review.openstack.org/169598
10:02 cdent joined #openstack-rally
10:04 kiran-r How is the trigger the cleanup in rally.?
10:12 openstackgerrit Sergey Skripnick proposed stackforge/rally: WIP: super new rally_gate.py  https://review.openstack.org/163785
10:19 Miouge joined #openstack-rally
10:20 msdubov joined #openstack-rally
10:21 olkonami joined #openstack-rally
10:26 pboros joined #openstack-rally
10:29 oanufriev joined #openstack-rally
10:31 rvcehimanshu1 joined #openstack-rally
10:36 aarefiev joined #openstack-rally
10:40 openstackgerrit Sergey Skripnick proposed stackforge/rally: Make tempest out of rally  https://review.openstack.org/169690
10:42 redixin joined #openstack-rally
11:01 andreykurilin yfried_" hi
11:02 rook__ joined #openstack-rally
11:03 andreykurilin yfried_: auto_generated config file is stored as ~/.rally/tempest/for-deployment-<uuid>/tempest.conf , so copy of path to this file will not save config while re-install.
11:03 andreykurilin yfried_: you should copy it to some temp dir, uninstall&install tempest and than, return it back:)
11:04 yfried__ joined #openstack-rally
11:13 rvcehimanshu joined #openstack-rally
11:25 openstackgerrit pradeep proposed stackforge/rally: Add "suspend and resume" scenario for Nova  https://review.openstack.org/169150
11:28 openstackgerrit pradeep proposed stackforge/rally: Add "pause and unpause" scenario for Nova  https://review.openstack.org/169598
11:36 openstackgerrit pradeep proposed stackforge/rally: Add "pause and unpause" scenario for Nova  https://review.openstack.org/169598
11:44 yfried__ joined #openstack-rally
11:48 yfried joined #openstack-rally
11:49 psd joined #openstack-rally
11:55 yfried boris-42: are you around?
11:55 yfried <kiran-r> How is the trigger the cleanup in rally.?
11:55 yfried kiran-r: ^ could you please explain your question
11:59 openstackgerrit pradeep proposed stackforge/rally: Add "suspend and resume" scenario for Nova  https://review.openstack.org/169150
12:08 panbalag joined #openstack-rally
12:09 mwagner_lap joined #openstack-rally
12:13 _kiran_ joined #openstack-rally
12:16 panbalag joined #openstack-rally
12:29 openstackgerrit Sergey Skripnick proposed stackforge/rally: WIP: super new rally_gate.py  https://review.openstack.org/163785
12:39 yfried redixin: andreykurilin: I am seeing a weird issue
12:39 andreykurilin yfried: ?
12:39 yfried I executed this task: http://paste.openstack.org/show/197772/
12:40 yfried but in output html, I see this under "Input file": http://paste.openstack.org/show/197771/
12:40 yfried andreykurilin: please note "resource_management_workers" detail
12:40 yfried which wasn't in original input
12:41 yfried andreykurilin: or the fact that the order of some values has changed
12:42 Miouge joined #openstack-rally
12:42 andreykurilin yfried: change of order is not strange. json or yaml loader reads config file in dict and as you know, dict doesn't save the order:)
12:43 yfried andreykurilin: I meant that I assumed the Input file would be the actual input *file* and not something that have been parsed
12:43 openstackgerrit Olga Kopylova proposed stackforge/rally: Lazy import of keystoneclient in osclients  https://review.openstack.org/169729
12:43 yfried andreykurilin: I'm more concerned about fields that I didn't specify
12:46 andreykurilin yfried: input file cann't be the actual input *file*, because actual input file is a template, which can contain some variables like {{image}} and etc, so `rally task start` loads given file and extend it with parameters from --task-args.
12:48 yfried andreykurilin: but what about resource_management_workers that doesn't exist in input file?
12:49 andreykurilin yfried: I'm just looking at code and can't find a place where it can be added...need more time
12:52 yfried andreykurilin: looks like it's taking the value from CONF.users_context.resource_management_workers
12:52 yfried rally/benchmark/context/users.py L90
12:54 andreykurilin yfried: I suppose, default values from user context are added while task validation
12:55 yfried andreykurilin: tnx
12:57 andreykurilin yfried: hm...strange, validation happens after printing input file
13:05 svasheka boris-42:  https://review.openstack.org/#/c/165450/14/rally/benchmark/scenarios/keystone/utils.py if I use resourse_delete, will I get results for how much time did it took to delete one?
13:12 dpaterson joined #openstack-rally
13:16 yfried svasheka: ^ yes
13:16 svasheka yfried: ty
13:16 yfried svasheka: notice "with base.AtomicAction(self, r):"
13:17 yfried svasheka: means that the delete call has a timer attached to it
13:18 svasheka yfried: yeah my bad, should've read more carefully
13:18 yfried svasheka: np
13:28 openstackgerrit Roman Vasilets proposed stackforge/rally: Add murano benchmarks  https://review.openstack.org/137661
13:29 boris-42 yfried: hi therer
13:29 yfried boris-42: Hi. Been looking for you all day :)
13:30 boris-42 yfried: was sleeping whole day=)
13:30 yfried boris-42: seems legit...
13:30 boris-42 yfried: soo regarding to your question
13:30 boris-42 yfried: it is better to change interface and raise not implemented error
13:31 boris-42 yfried: then to do changes only in one class
13:31 openstackgerrit Sergey Skripnick proposed stackforge/rally: WIP: super new rally_gate.py  https://review.openstack.org/163785
13:31 yfried boris-42: ok, I'm not sure if I agree, but please state so in comments for nmagnezi to follow
13:32 openstackgerrit Roman Vasilets proposed stackforge/rally: Add murano benchmarks  https://review.openstack.org/137661
13:32 boris-42 yfried: so but I am a bit worried about whole situtation
13:32 yfried nmagnezi: boris-42: so no need to touch novaWrapper
13:32 boris-42 yfried: there is ^
13:33 boris-42 yfried: as I said I dislike making classes that should inherit one intereface different
13:33 boris-42 yfried: and in case of nova network that class may stop work and raise bad exceptions
13:33 boris-42 yfried: I am more and more thinking about creating separated NeutronNetworkContext
13:33 boris-42 yfried: that will assume that is run only against neutron
13:34 boris-42 yfried: the goal of NetwrokContext is to work in both Nova Network and Neutron world
13:35 yfried boris-42: maybe your way is better, but it shouldn't block the current patch, as uses existing framwork to the best way available
13:35 boris-42 yfried: nope it doesn't use it well
13:35 boris-42 yfried: changes in wrapper are inproper
13:35 yfried boris-42: I'm open to better ideas that don't require complete rally refactor
13:35 boris-42 yfried: the idea in wrapper is to NOT care which network backedn you use
13:36 boris-42 yfried: I already said
13:36 boris-42 yfried: make a separated NeutronNetworkContext
13:36 boris-42 yfried: and use directly neutron client do all the things that you need
13:37 yfried boris-42: ^ that's a complete network refactor
13:37 boris-42 yfried: hm where?
13:37 boris-42 yfried: it's just 1 more plugin in rally
13:37 yfried boris-42: make a separated NeutronNetworkContext
13:37 yfried boris-42: that's out of scope for current patch
13:37 boris-42 yfried: that doesn't affect anything else
13:38 boris-42 yfried: it is not
13:38 boris-42 yfried: you are making from NetworkContext that should work in the same way in Neutron and NovaNetwork
13:38 boris-42 yfried: something that doesn't work in NovaNetwork
13:38 boris-42 yfried: if some arguments are specified
13:39 boris-42 as well you are breaking idead of wrapper in root
13:39 boris-42 that are just done to hide difference between network dirver
13:40 yfried boris-42: you'd need to define clearly the different use cases between the wrapper and the context
13:46 anshul joined #openstack-rally
13:52 * boris-42 yfried: me back
13:52 boris-42 yfried: so
13:54 boris-42 yfried: so we are wrapping osclients and returning instance with strict interface that will work in our cloud
13:54 svasheka boris-42: https://review.openstack.org/#/c/165422/9/tests/unit/benchmark/scenarios/keystone/test_basic.py if it return fake_user, then no additional comments?
13:55 boris-42 yfried: and it doesn't matter what network backend do you have
13:55 boris-42 yfried: and it is used internally where we just want results (and don't care too much about how it is done)
13:56 boris-42 yfried: Network context is plugin that is used by end users, it goal is to create network and do all required setups no matter what Network backend we are using
13:56 baker joined #openstack-rally
13:56 boris-42 yfried: that patch change both Wrapper and Network context in such way that they become useless
13:57 boris-42 yfried: aka the major rule is not fulfilled
13:57 boris-42 yfried: if we need something that is only presented in Neutron and can be done only in Neutron
13:57 boris-42 yfried: it is candidate for separated context class
13:58 boris-42 yfried: for example we can do common part inside network context and setting up dualip in separated context
13:58 boris-42 yfried: does that make sense?
14:02 anshul joined #openstack-rally
14:06 openstackgerrit svasheka proposed stackforge/rally: Add keystone create_user_update_password scenario  https://review.openstack.org/165422
14:10 openstackgerrit svasheka proposed stackforge/rally: Add keystone benchmark scenarios for roles  https://review.openstack.org/165409
14:11 nkhare joined #openstack-rally
14:24 openstackgerrit Sergey Skripnick proposed stackforge/rally: WIP: super new rally_gate.py  https://review.openstack.org/163785
14:28 nkhare joined #openstack-rally
14:28 openstackgerrit svasheka proposed stackforge/rally: Add unit test for keystone._get_service_by_name  https://review.openstack.org/169034
14:31 boris-42 yfried: so?
14:32 nkhare_ joined #openstack-rally
14:33 nkhare joined #openstack-rally
14:37 cdent_ joined #openstack-rally
14:39 exploreshaifali joined #openstack-rally
14:46 yfried boris-42: sorry. I had to drop offline. I'll go over your comment later. sorry
14:46 boris-42 yfried: ok sure
15:03 openstackgerrit Prabhjyot Singh Sodhi proposed stackforge/rally: Create 'add cloud info' spec file  https://review.openstack.org/169432
15:03 rvcehimanshu joined #openstack-rally
15:10 openstackgerrit Sergey Skripnick proposed stackforge/rally: WIP: super new rally_gate.py  https://review.openstack.org/163785
15:22 openstackgerrit Alexander Maretskiy proposed stackforge/rally: [Reports] Add classes for benchmark HTML report charts  https://review.openstack.org/159458
15:35 prashantS joined #openstack-rally
15:41 arxcruz joined #openstack-rally
15:49 openstackgerrit Sergey Skripnick proposed stackforge/rally: WIP: super new rally_gate.py  https://review.openstack.org/163785
15:53 kiran joined #openstack-rally
16:07 openstackgerrit Alexander Maretskiy proposed stackforge/rally: [Reports] Add chunks support to report  https://review.openstack.org/146814
16:16 openstackgerrit Alexander Maretskiy proposed stackforge/rally: [Reports] Add Load Profile chart to report  https://review.openstack.org/169828
16:20 openstackgerrit Alexander Maretskiy proposed stackforge/rally: [Vm] Restore fixed IP support in boot_runcommand_delete  https://review.openstack.org/161699
16:21 andreykurilin_ joined #openstack-rally
16:25 exploreshaifali joined #openstack-rally
16:26 openstackgerrit Sergey Skripnick proposed stackforge/rally: WIP: super new rally_gate.py  https://review.openstack.org/163785
16:34 rediskin joined #openstack-rally
16:48 openstackgerrit Boris Pavlovic proposed stackforge/rally: Script that profiles imports  https://review.openstack.org/169837
16:59 rvcehimanshu joined #openstack-rally
17:07 kun_huang boris-42:  https://review.openstack.org/#/c/169837/1/rally/__init__.py
17:07 e0ne joined #openstack-rally
17:07 kun_huang boris-42:  I have a test, with a result 39216-lines json file
17:10 openstackgerrit Yuriy Nesenenko proposed stackforge/rally: Add Cinder create_backup volume scenario  https://review.openstack.org/167171
17:11 davideagnello joined #openstack-rally
17:11 boris-42 лkun_huang нуз
17:11 boris-42 kun_huang: yep
17:12 boris-42 kun_huang: we have a LOT OF imports calls
17:13 kun_huang boris-42: seeing the 100_longest one, top 10 are rally import calls LOL
17:16 boris-42 kun_huang: it's not the true
17:16 boris-42 kun_huang: the thing is when you are importing the module that will import other modules
17:16 boris-42 kun_huang: you will get duration of import of first module bigger
17:17 boris-42 kun_huang: I will try to make tree of imports
17:17 kun_huang boris-42: yep, I know that
17:17 kun_huang boris-42:  the tree imports make sense
17:22 tosky joined #openstack-rally
17:30 davideag_ joined #openstack-rally
17:34 e0ne joined #openstack-rally
17:44 meteorfox boris-42: is there a 'OpenStack way' of adding/writing microbenchmarks? My concern is that the patch for chunk support for reports hasn't been tested with large data size, so how can we be sure it does the right thing
17:54 prashant_ joined #openstack-rally
17:59 e0ne joined #openstack-rally
18:14 yfried joined #openstack-rally
18:15 boris-42 meteorfox: what do you mean by openstack way?
18:15 boris-42 meteorfox: openstack way is rally=)
18:15 boris-42 meteorfox: we will test these chunks on larger sets of data when we finish everything
18:15 boris-42 meteorfox: including db changes
18:15 boris-42 kun_huang: so it's not so hard
18:18 boris-42 meteorfox: as well we will use rally for rally testing and benchmarking in future
18:28 e0ne joined #openstack-rally
18:32 zerda joined #openstack-rally
18:36 openstackgerrit Sergey Skripnick proposed stackforge/rally: WIP: super new rally_gate.py  https://review.openstack.org/163785
18:39 cdent joined #openstack-rally
18:52 meteorfox boris-42: I meant, if there a library or module to do python microbenchmarks. I don't think Rally would be a good fit for what I'm thinking about.
18:52 meteorfox boris-42: for example, in Java, there's JMH and Google  Caliper
18:52 meteorfox boris-42: in Go, there's the -bench flag
18:53 meteorfox boris-42: I know there are some python libraries, like heapy (guppy), and timeit
18:53 meteorfox boris-42: I was just wondering if there was a library that brought those kind of utilities together in Python
18:54 meteorfox boris-42: the idea is that I can run a function, or method, with a know data set, and get heap diff, and time it
18:58 prashantS joined #openstack-rally
19:07 openstackgerrit Sergey Skripnick proposed stackforge/rally: WIP: super new rally_gate.py  https://review.openstack.org/163785
19:15 boris-42 meteorfox: we will use just rally
19:15 boris-42 meteorfox: for that
19:16 meteorfox boris-42: wouldn't that be scope creep. I feel that's not really Rally's domain. What I'm asking for is not even openstack specific, it's just python microbenchmarks
19:17 boris-42 meteorfox: so let me expalin
19:17 boris-42 meteorfox: we will write rally plugins that are calling rally
19:18 boris-42 meteorfox: ant there will be tests with large set
19:18 boris-42 meteorfox: and we will get overall perforomance
19:18 boris-42 meteorfox: and do this in gates
19:18 meteorfox boris-42: ok, I see.
19:18 boris-42 meteorfox: micro-benchmarking i believe is something that we should avoid
19:18 boris-42 =)
19:19 boris-42 meteorfox: because users are not interested of how fast works some piece of rally
19:19 boris-42 meteorfox: they want to know that we can create 100k tasks
19:19 boris-42 meteorfox: and that there are no races if we are running multple tasks simultaneously
19:19 boris-42 meteorfox: and that they can generate report with 1000k iterations
19:20 boris-42 and this should be voting and run on every patch*
19:20 boris-42 not just one time action
19:20 meteorfox boris-42: right, but what I'm suggesting wouldn't be for the users, but for us.  Writing simple benchmarks in our tests, that we could run with tox, to make sure we can deliver that performance
19:21 meteorfox boris-42: here's what the Falcon guys/gals did. https://github.com/falconry/falcon/blob/master/falcon/bench/bench.py
19:32 meteorfox boris-42: a little toolkit, or framework like that, could easily be a separate stackforge project, which different projects could use to write little micro-benchmarks as a regression-suite
19:34 meteorfox boris-42: think of it as performance unit tests
19:58 davideagnello joined #openstack-rally
19:58 davideagnello joined #openstack-rally
20:00 psd joined #openstack-rally
20:12 baker joined #openstack-rally
20:44 baker joined #openstack-rally
20:49 openstackgerrit Sergey Skripnick proposed stackforge/rally: WIP: super new rally_gate.py  https://review.openstack.org/163785
20:52 e0ne joined #openstack-rally
20:58 e0ne joined #openstack-rally
21:05 psd joined #openstack-rally
21:35 openstackgerrit Sergey Skripnick proposed stackforge/rally: WIP: super new rally_gate.py  https://review.openstack.org/163785
21:52 baker joined #openstack-rally
21:58 boris-42 joined #openstack-rally
22:00 openstackgerrit Sergey Skripnick proposed stackforge/rally: WIP: super new rally_gate.py  https://review.openstack.org/163785
22:14 mwagner_lap joined #openstack-rally
22:55 andreykurilin_ joined #openstack-rally
23:11 prashant_ joined #openstack-rally
23:32 harlowja joined #openstack-rally
23:50 harlowja joined #openstack-rally
23:51 nkhare joined #openstack-rally

| Channels | #openstack-rally index | Today | | Search | Google Search | Plain-Text | summary