Perl 6 - the future is here, just unevenly distributed

IRC log for #openstack-rally, 2015-03-23

| Channels | #openstack-rally index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:44 PerfBeing joined #openstack-rally
00:49 PerfBeing joined #openstack-rally
01:01 baker joined #openstack-rally
01:27 panbalag joined #openstack-rally
01:39 yingjun joined #openstack-rally
02:37 baker joined #openstack-rally
03:29 baker joined #openstack-rally
04:00 nkhare joined #openstack-rally
04:39 PerfBeing joined #openstack-rally
04:45 PerfBeing joined #openstack-rally
05:24 rdas joined #openstack-rally
05:27 rdas joined #openstack-rally
05:33 tfreger joined #openstack-rally
06:03 zerda joined #openstack-rally
06:04 neeti joined #openstack-rally
06:09 anshul joined #openstack-rally
06:12 yfried|afk joined #openstack-rally
06:25 nmagnezi joined #openstack-rally
06:27 yfried|afk joined #openstack-rally
06:53 yfried|afk joined #openstack-rally
06:58 pcaruana joined #openstack-rally
07:00 anshul joined #openstack-rally
07:05 anshul joined #openstack-rally
07:07 anshul joined #openstack-rally
07:11 yfried joined #openstack-rally
07:18 yfried joined #openstack-rally
07:39 yfried joined #openstack-rally
07:47 anshul joined #openstack-rally
07:54 msdubov joined #openstack-rally
08:14 fhubik_afk joined #openstack-rally
08:19 amaretskiy joined #openstack-rally
08:28 yfried_ joined #openstack-rally
08:30 pboros joined #openstack-rally
08:37 yfried_ joined #openstack-rally
08:40 andreykurilin_ joined #openstack-rally
08:44 yfried__ joined #openstack-rally
09:02 e0ne joined #openstack-rally
09:04 anshul joined #openstack-rally
09:06 psd joined #openstack-rally
09:09 fhubik joined #openstack-rally
09:40 psd joined #openstack-rally
09:44 andreykurilin yfried: no. `rally verify compare` works only with two verification results
09:47 openstackgerrit Alexander Maretskiy proposed stackforge/rally: [Vm] Restore fixed IP support in boot_runcommand_delete  https://review.openstack.org/161699
09:51 frobware are there are rally nova scenarios where I can customize the amount of work that is done between boot and delete?
09:54 e0ne joined #openstack-rally
09:58 amaretskiy frobware: if you want to run custom shell script on VM, then use https://github.com/stackforge/rally/blob/master/rally/benchmark/scenarios/vm/vmtasks.py#L46
09:59 amaretskiy frobware: otherwise implement your own scenario - look at https://github.com/stackforge/rally/blob/master/rally/benchmark/scenarios/nova/servers.py#L102-L104
10:00 amaretskiy frobware: it's rather simple to make own scenario based on existent code
10:00 frobware amaretskiy, how does the script get "into" the VM?
10:01 amaretskiy there is a SSH connection established
10:01 frobware amaretskiy, and 'rally task ...' will copy it there?
10:05 amaretskiy frobware: patho to shell script should be specified via task arguments in input json/yaml file
10:05 amaretskiy *path
10:06 amaretskiy this script will be copied into each vm, and its json output will be available in task results
10:07 frobware amaretskiy: thanks; will try the dd sample.
10:14 e0ne joined #openstack-rally
10:18 cdent joined #openstack-rally
10:26 openstackgerrit Alexander Maretskiy proposed stackforge/rally: [Reports] Add classes for benchmark HTML report charts  https://review.openstack.org/159458
10:29 tfreger joined #openstack-rally
10:30 msdubov joined #openstack-rally
10:45 panbalag joined #openstack-rally
10:46 panbalag joined #openstack-rally
10:57 tosky joined #openstack-rally
11:03 openstackgerrit Alexander Maretskiy proposed stackforge/rally: [Reports] Add chunks support and chart `Parallel iterations' to report  https://review.openstack.org/146814
11:10 anshul joined #openstack-rally
11:27 openstackgerrit Roman Vasilets proposed stackforge/rally: Add murano benchmarks  https://review.openstack.org/137661
11:27 openstackgerrit Roman Vasilets proposed stackforge/rally: Add base for murano benchmarking  https://review.openstack.org/137650
11:29 openstackgerrit Roman Vasilets proposed stackforge/rally: Adding Multiple Request Scenario  https://review.openstack.org/117705
11:47 soumit joined #openstack-rally
11:52 rvcehimanshu joined #openstack-rally
12:03 soumit joined #openstack-rally
12:11 frobware I was looking at https://www.mirantis.com/blog/rally-as-an-openstack-performance-dashboard/ and wondered how the results page has more than one task listed?  When I run 'rally task report' I only get the last test I ran. Is there a means to get all the tasks in one HTML report file?
12:11 oanufriev joined #openstack-rally
12:15 psd joined #openstack-rally
12:23 mwagner_lap joined #openstack-rally
12:33 fhubik joined #openstack-rally
12:34 fhubik_afk joined #openstack-rally
12:48 boris-42 frobware: hi
12:48 frobware boris-42, hello
12:51 frobware boris-42, I also noticed the tempest output (HTML) - how does one generate that with rally?
12:54 dpaterson joined #openstack-rally
12:55 nmagnezi joined #openstack-rally
13:04 boris-42 frobware: just a sec
13:04 boris-42 frobware: so you can put rally task report <uuid1> <uuid2> <uuid3> ....
13:05 frobware boris-42: is that how the report in the blog was generated?
13:07 boris-42 frobware: what blog?
13:07 boris-42 frobware: ah no nope
13:07 boris-42 frobware: it just had a lot of benchs
13:08 frobware boris-42: in a single file (json/yaml)?
13:09 fhubik_afk joined #openstack-rally
13:09 fhubik_lunch joined #openstack-rally
13:10 boris-42 frobware: yep but the result will look the same in both cass
13:10 dpaterson joined #openstack-rally
13:10 boris-42 frobware: 1 task with a lot of benchmarks and N tasks with 1 benchmark
13:11 frobware boris-42: so this (http://people.linaro.org/~andrew.mcdermott/task.json) is valid for multiple tasks?
13:12 boris-42 frobware: just a sec
13:13 baker joined #openstack-rally
13:15 boris-42 frobware: yep it should be valid
13:15 dpaterson joined #openstack-rally
13:16 frobware boris-42: thanks, make sense now.
13:18 frobware boris-42, with regard to the tempest html table; is there some additional post-processing that you did from a 'rally verify detailed'?
13:20 boris-42 frobware: ?)
13:20 frobware boris-42, fig 2 in https://www.mirantis.com/blog/rally-as-an-openstack-performance-dashboard/
13:21 boris-42 frobware: https://lh4.googleusercontent.com/bzG5aTmf8OfoHBKmC5Bix5wiunRyw1lUwfLaHR8nHczoWummrpoqAS9luL5uHnFFsJ9gpBF7uajnQkT4ywajXJNJ86NxY5EGbTKnusBYGRe1_gpYoHsUc8tEOAnWX5JjJ5xM8_A
13:22 frobware boris-42: is that output generated from within rally?
13:23 boris-42 frobware: yep "rally verify results --html "
13:23 nmagnezi_ joined #openstack-rally
13:24 * frobware blushes - I see it now. :)
13:25 * frobware has more questions...
13:26 frobware boris-42, is it possible to get a list of the uuids without all the tables formatting? I wanted to do something like: rally task report --tasks $(rally task list) --out foo.html
13:36 boris-42 frobware: weshould extend
13:36 boris-42 frobware: that
13:37 dpaterson joined #openstack-rally
13:39 baker joined #openstack-rally
13:41 dpaterson joined #openstack-rally
13:43 nmagnezi__ joined #openstack-rally
13:45 exploreshaifali joined #openstack-rally
13:50 openstackgerrit Nikita Konovalov proposed stackforge/rally: [Sahara] Suppurt batch job execution  https://review.openstack.org/161188
13:52 e0ne joined #openstack-rally
14:08 nmagnezi_ joined #openstack-rally
14:10 boris-42 yfried__: hi there
14:18 yfried__ boris-42: hi
14:19 coolsvap joined #openstack-rally
14:20 nkhare joined #openstack-rally
14:25 frobware for the tempest html generation is it possible to point rally at an existing tempest run and generate the same report?
14:28 frobware I have an existing testrepository with tempest results, it would be handy to generate a report from that using rally.
14:31 yfried__ dpaterson: ^
14:32 dpaterson frobware: I am adding an import feature to do just that.
14:32 frobware dpaterson: \o/
14:32 dpaterson https://blueprints.launchpad.net/rally/+spec/verification-import
14:37 baker joined #openstack-rally
14:39 boris-42 frobware: yep
14:40 boris-42 frobware: ah no=)
14:40 boris-42 frobware: but if dpaterson implement it I will be happy to merge it=)
14:40 boris-42 frobware: btw do you want to try to extend list commnad?
14:40 boris-42 frobware: to accept output formats?
14:41 frobware boris-42: yes, I could look at that.
14:41 boris-42 frobware: so the idea is like we have in "rally verify results"
14:41 boris-42 --html/--json/....
14:41 boris-42 we can have here --oneline --json or by default table
14:42 boris-42 it will be quite common
14:43 dpaterson boris-42: I was on Red Hat Rally syncup this morning and I was made aware that we are supposed to put new features in docs/specs?
14:43 openstackgerrit Nikita Konovalov proposed stackforge/rally: [Sahara] Improve Image context  https://review.openstack.org/166859
14:43 dpaterson Do I need to add a spec for import feature or is bluerprint enough?
14:45 yfried__ dpaterson: what kind of tempest output do you plan to process? subunit? junit?
14:45 dpaterson same stream that is created when you run verify
14:45 dpaterson subunit.stream
14:47 boris-42 dpaterson: nope
14:47 dpaterson nope what?
14:47 boris-42 dpaterson: you don't need to create spec for that
14:47 dpaterson ah cool :)
14:48 boris-42 dpaterson: so I believe that you should do something like we have for "rally task report"
14:48 boris-42 dpaterson: you can specify file with result of "rally task results"
14:48 dpaterson boris-42: I will look at that feature, never have.
14:48 boris-42 and it will generate HTML
14:49 dpaterson boris-42: okay, will plagerize
14:49 boris-42 dpaterson: btw it will be nice to have in "rally verify results --subunit"
14:49 boris-42 ouput
14:49 boris-42 for integration between Rally & Jenkins
14:49 dpaterson yes
14:50 dpaterson can add as seperate patch
14:50 boris-42 that one doesn't require specs as well=)
14:50 dpaterson right
14:50 boris-42 dpaterson: yep
14:50 dpaterson no blueprint at all right?
14:50 boris-42 dpaterson: just for BIG features
14:50 dpaterson right
14:50 boris-42 dpaterson: and we are doing now specs
14:50 boris-42 not BPs
14:50 dpaterson yeah, makes sense
14:51 dpaterson boris-42: fyi I am going to NYC to work with the QA folks, specifically around tempest cli stuff on my end.  Anything you'd like to bring to table?
14:54 rdas joined #openstack-rally
14:55 boris-42 dpaterson: why not rally?)
14:55 boris-42 dpaterson: for func testing?)
14:55 boris-42 dpaterson: and cli testing?)
14:57 neeti joined #openstack-rally
14:57 dpaterson boris-42: sorry I don't quite understand your questions
14:58 nmagnezi_ joined #openstack-rally
14:58 boris-42 dpaterson: ah you mean adding cli interface for tempest
14:59 dpaterson da
14:59 boris-42 dpaterson: I thought using tempest for project cli testing
14:59 boris-42 dpaterson: so i have
14:59 boris-42 dpaterson: http://boris-42.me/rally-verfiy-as-the-control-plane-for-gabbi-tempest-in-tree-functional-tests/ did you see this ?
15:00 dpaterson boris-42: sorry for confusion.  I'll check it out now
15:03 boris-42 dpaterson: so mabye that effort is more common that tempest cli
15:07 dpaterson boris-42: Yeah where tests are moving out of tempest and into projects testing by project will be good.
15:13 yfried joined #openstack-rally
15:14 yfried joined #openstack-rally
15:17 baker joined #openstack-rally
15:18 boris-42 dpaterson: yep
15:31 openstackgerrit Sergey Skripnick proposed stackforge/rally: WIP: super new rally_gate.py  https://review.openstack.org/163785
15:31 PerfBeing joined #openstack-rally
15:32 boris-42 frobware: btw do you need any help with contributing stuff?
15:32 boris-42 frobware: did you contribute before to OpenStack proejcts?)
15:33 frobware boris-42: probably - this would be my first OS patch
15:36 exploreshaifali joined #openstack-rally
15:42 jaypipes joined #openstack-rally
15:42 meteorfox boris-42: hey boris-42 !, you are alive! I thought something happened to you. I didn't see you online last Thursday, and Friday =)
15:45 boris-42 meteorfox: I don't want talk about that=)
15:45 boris-42 meteorfox: but now I am back= )
15:45 boris-42 frobware: so
15:45 boris-42 frobware: this page may help you https://rally.readthedocs.org/en/latest/contribute.html
15:46 meteorfox boris-42: ah, no problem man. I'm glad you are back.
15:46 boris-42 frobware: this is the command that you should fix https://github.com/stackforge/rally/blob/master/rally/cmd/commands/task.py#L447
15:46 boris-42 meteorfox: actually I was working
15:46 boris-42 meteorfox: hard working=)
15:46 frobware boris-42, think I'm mostly there for the --oneline part
15:46 boris-42 not like now=)
15:46 boris-42 frobware: so that part should be fixed
15:46 frobware boris-42: quick questions: github PRs, or something else... ?
15:47 boris-42 frobware: PR?
15:47 frobware boris-42, pull request
15:47 boris-42 frobware: nope not github pull request
15:47 boris-42 frobware: openstack uses gerrit and own stuff
15:48 boris-42 github repo is mirror
15:48 frobware ok
15:48 boris-42 frobware: so you should take a look at this page https://rally.readthedocs.org/en/latest/contribute.html =)
15:48 boris-42 frobware: everything is expalined
15:49 frobware boris-42, Might have to get a company sign-off account...
15:49 frobware boris-42, for the CLA
15:49 boris-42 frobware: uhoh
15:49 boris-42 frobware: btw are you working togehter with dpaterson ?)
15:49 boris-42 meteorfox: so did you have a chance to think about various reports?
15:50 frobware boris-42, no. or not that I'm aware of. :)
15:51 frobware boris-42, how do I get "joined" to either the openstack or rally in LP?
15:53 meteorfox boris-42: not really, but right now the only thing that comes to mind is two things. Response Time Histogram (maybe with log x-scale, might not be needed), API response time metrics for each atomic action, and a timeseries of atomic_actions per (second|minute)
15:54 meteorfox boris-42: three things actually, last one I just came up with it
15:54 boris-42 frobware: hm?
15:54 boris-42 frobware: so you should register to OpenStack.org
15:54 boris-42 frobware: after that to lauchpad
15:54 frobware boris-42, already registered in LP
15:55 meteorfox boris-42: let me show you some example of what I mean. These are taken from Gatling, and HTTP load generator framework
15:56 boris-42 frobware: so can you open https://review.openstack.org/#/ and try to login?
15:56 meteorfox https://usercontent.irccloud-cdn.com/file/7CQ0k6s0/response_time_series.png
15:56 boris-42 meteorfox: samples are very welcome
15:56 frobware boris-42, need to go through the CLA process first... just investigating internally to see if we have already signed a corporate version.
15:57 boris-42 frobware: I signed individual
15:57 meteorfox https://usercontent.irccloud-cdn.com/file/9owOMrCU/density_plot.png
15:57 boris-42 dpaterson: but we don't have latency stuff
15:57 boris-42 dpaterson: sry
15:57 boris-42 meteorfox: ^
15:57 frobware boris-42, I can sign into review.o.o OK
15:58 meteorfox boris-42: latency == response time
15:58 meteorfox boris-42: so, the boot times could be ploted in a histogram/density plot
15:58 meteorfox for example
15:59 meteorfox https://usercontent.irccloud-cdn.com/file/5KfsxjVJ/reqs_per_seconds.png
16:00 meteorfox boris-42: also something like that ^ (but requests per seconds, instead of responses) to see load being injected over time
16:05 meteorfox boris-42: So a lot of the performance tests I run, are measuring the time it takes to do X under certain conditions, or parameters. E.g Boot times with X flavor, and Y image. Boot times with volumes, Concurrent Boot times (1 to N concurrent). Upload images of X size or format. etc
16:06 frobware boris-42, good news, already signed.
16:06 meteorfox boris-42: so having  a histograms, shows the distribution of the times. Also the time series would show me the throughput over time
16:07 meteorfox boris-42: finally, the response times time series, will show me for each transaction how the response time of the operation is behaving over time. e.g. increase, or spikes, etc
16:07 boris-42 frobware: great=)
16:08 boris-42 meteorfox: so we have already histograms
16:08 boris-42 meteorfox: some kind of them
16:08 meteorfox meteorfox: right, I saw that in the patch
16:08 boris-42 meteorfox: I mean not only in patch
16:08 boris-42 meteorfox: we already have them
16:08 openstackgerrit Sergey Skripnick proposed stackforge/rally: WIP: super new rally_gate.py  https://review.openstack.org/163785
16:09 meteorfox boris-42: yeah, I think I know the one you are referring too
16:09 boris-42 meteorfox: so it will be nice to prepare spec
16:09 boris-42 meteorfox:yo ucan do it step by step
16:10 meteorfox boris-42: I'm not sure if that style of histogram will work. :/ I need something like this
16:10 meteorfox https://usercontent.irccloud-cdn.com/file/diZQdRtf/build_times_histograms.png
16:10 meteorfox boris-42: just one of the graphs is OK
16:10 meteorfox boris-42: I mean from that histogram
16:11 boris-42 so we have one line
16:11 boris-42 of that graph
16:11 meteorfox boris-42: right
16:11 meteorfox boris-42: that's all I need
16:11 boris-42 but it's alerady implementd
16:12 boris-42 meteorfox: http://logs.openstack.org/42/160142/8/check/gate-rally-dsvm-rally/76fecde/rally-plot/results.html.gz#/KeystoneBasic.create_and_list_users bottom right
16:12 boris-42 meteorfox: histogram
16:12 meteorfox boris-42: ok, what was weird for me using that histogram, its the scales can they be padded so it doesn't look like a bar chart
16:12 meteorfox ?
16:13 boris-42 meteorfox: ?
16:14 boris-42 meteorfox: hm not sure that I get=)
16:14 meteorfox boris-42: what I mean, is that it needs padding on both scales
16:14 meteorfox boris-42:  they way is rendered looks like a barchart
16:16 boris-42 meteorfox: hm I really don't see any difference...
16:16 boris-42 meteorfox: except that they have factor stuff and N graphs instead of our 1
16:16 frobware boris-42, to ensure we're singing from the same hymn sheet: http://people.linaro.org/~andrew.mcdermott/0001-rally-add-a-oneline-option-to-rally-task-list.patch
16:17 boris-42 frobware: just run "git review"
16:17 boris-42 frobware: and let's discuss it using std review process
16:17 frobware boris-42, sure
16:17 boris-42 frobware: it will be simpler for everybody \
16:18 boris-42 meteorfox: so what you would like to scale?
16:18 boris-42 meteorfox: on bottom we have duration (like in that chart)
16:18 meteorfox boris-42: "they" is actually me. I built that last chart. But, like you say there's no difference (ignore the factors), I meant your histogram needs padding on each side of the scale, it's weird that it takes the whole graphing area
16:19 boris-42 meteorfox: hm
16:19 boris-42 meteorfox: but why it should not take whole graphing area?
16:19 meteorfox boris-42: give a few mins, let me see if I can edit it on JS/CSS to show you what I want
16:20 boris-42 meteorfox: ok that makes sense=)
16:21 baker joined #openstack-rally
16:35 meteorfox boris-42: I couldn't edit the JS, because it's embedded on HTML, and I don't SVG enough, to edit manually. Anyway, I hope this can help. imho, is that adding padding (white space) to each side of the chart, will help the users to interpret the charts as histogram, and see how it's is distributed, especially if it starts from 0. Here's an example, using d3js.
16:35 meteorfox http://bl.ocks.org/mbostock/raw/3048450/
16:36 boris-42 meteorfox: so you add whitespaces from letf and right?
16:36 openstackgerrit Roman Vasilets proposed stackforge/rally: Add murano benchmarks  https://review.openstack.org/137661
16:36 openstackgerrit Roman Vasilets proposed stackforge/rally: Add base for murano benchmarking  https://review.openstack.org/137650
16:36 meteorfox boris-42: the changes would be, make both scales start from 0, and add some extra padding at the end of X, and Y scales
16:38 meteorfox boris-42: currently, your histogram's X scale behaves like a "discrete" scale, where each number is  a tick/label, and it doesn't show the continuity of the scale. Also, it doesn't start from 0
16:39 boris-42 meteorfox: so it starts from zero *
16:39 meteorfox boris-42: probably, is because it's actually a multi-bar chart, and not histogram.
16:39 meteorfox boris-42: only Y-scale starts from 0
16:40 boris-42 meteorfox: but what about case when you have duration between
16:40 boris-42 2k seconds and 3k seconds
16:40 boris-42 why we should start histogram from zero?
16:41 meteorfox boris-42: usually this is handle with a log scale in X axis
16:41 meteorfox handled*
16:41 openstackgerrit svasheka proposed stackforge/rally: Add keystone create_and_delete_service scenario  https://review.openstack.org/165468
16:44 boris-42 meteorfox: not fully convinced with this
16:44 boris-42 meteorfox: what are benifits?
16:45 meteorfox boris-42: yeah, I know. the benefits are mostly in the UX. Data won't change
16:49 meteorfox boris-42: do you have histogram, with more than 10 count? I just want to see how it looks when there's more disperse data
16:49 openstackgerrit Wataru Takase proposed stackforge/rally: Add boot server scenario for EC2 API  https://review.openstack.org/147550
16:51 meteorfox boris-42: nevermind found one: http://logs.openstack.org/42/160142/8/check/gate-rally-dsvm-rally/76fecde/rally-plot/results.html.gz#/Authenticate.keystone
16:54 meteorfox boris-42: well, ok so let's forget about histograms. I think yours already provides what I need :)
16:55 meteorfox boris-42: now you'r total durations chart, is kind of what I need. But it'd be cool if it shows a stacked decomposition of each atomic action duration, instead of total
16:56 meteorfox boris-42: do you think that's reasonable?
17:00 rdas joined #openstack-rally
17:05 exploreshaifali_ joined #openstack-rally
17:12 openstackgerrit svasheka proposed stackforge/rally: Add keystone benchmark scenarios for roles  https://review.openstack.org/165409
17:12 openstackgerrit svasheka proposed stackforge/rally: Add keystone update_and_delete_tenant scenario  https://review.openstack.org/165450
17:12 openstackgerrit svasheka proposed stackforge/rally: Add keystone update_user_password scenario  https://review.openstack.org/165422
17:12 openstackgerrit svasheka proposed stackforge/rally: Add keystone create_and_delete_service scenario  https://review.openstack.org/165468
17:21 PerfBeing joined #openstack-rally
17:24 boris-42 meteorfox: it's on another page...
17:24 boris-42 meteorfox: http://logs.openstack.org/42/160142/8/check/gate-rally-dsvm-rally/76fecde/rally-plot/results.html.gz#/NovaKeypair.create_and_delete_keypair/details
17:27 harlowja joined #openstack-rally
17:31 soumit joined #openstack-rally
17:32 dpaterson joined #openstack-rally
17:33 boris-42 meteorfox: or you mean histogram one?
17:35 meteorfox boris-42: yeah that's what I meant
17:35 boris-42 meteorfox: so yep current one is bad
17:48 abehl joined #openstack-rally
17:53 PerfBeing joined #openstack-rally
17:57 openstackgerrit Prasanth Anbalagan proposed stackforge/rally: Rally verify support for any in-tree Openstack project functional tests  https://review.openstack.org/166487
18:05 openstackgerrit Prasanth Anbalagan proposed stackforge/rally: Rally verify support for any in-tree Openstack project functional tests  https://review.openstack.org/166487
18:08 meteorfox boris-42: do you think we should merge the median change after, the reports stuff gets merged in? They modified benchmark/utils.py, and I'll need to rebase
18:11 meteorfox boris-42: or is there a way to make a patch depend on the changes of that?
18:18 boris-42 meteorfox: I think we can get it merged before reports
18:18 boris-42 meteorfox: and rebase reprots
18:18 meteorfox boris-42: cool. I'll work on it today, and see if I can get it merged in before tomorrow
18:21 harlowja joined #openstack-rally
18:25 openstackgerrit Mike Fedosin proposed stackforge/rally: Add the ability to get versions from context  https://review.openstack.org/165423
18:26 meteorfox boris-42: so going back to the report charts. We have histograms, and we have the stacked time series for the atomic_action durations. Last one is a throughput time series. The idea is that it will show atomic_actions per (second|minute) over time. It might not be useful for small runs, but concurrent workloads, that run for several minutes to hours, it
18:26 meteorfox would be useful
18:27 boris-42 hm
18:28 boris-42 is it similar to the graph that Alex is doing
18:28 boris-42 that show amount of concurrent iterations?
18:28 boris-42 over the time
18:30 meteorfox boris-42: maybe, I'll have to see it
18:31 boris-42 meteorfox: take a look http://logs.openstack.org/14/146814/35/check/gate-rally-dsvm-neutron-rally/093539d/rally-plot/results.html.gz#/NeutronNetworks.create_and_delete_networks
18:36 meteorfox boris-42: well, kind of. I was looking for an aggregation of all the actomic_actions per minute that were fired/completed. One chart for fired, and one for completed
18:37 meteorfox boris-42: if we know the timestamps of when atomic_actions were fired, and completed, we could then aggregate in blocks of 1 minute, and show that as a time series
18:38 meteorfox boris-42: then we could have a total for all atomic_actions combined, and a detailed one. One per atomic_action
18:46 openstackgerrit Sergey Skripnick proposed stackforge/rally: WIP: super new rally_gate.py  https://review.openstack.org/163785
18:47 boris-42 meteorfox: so keep in mind that this stuff should work for 10 iterations and for 10kk iterations for 10 seconds benchmarks and 10 days
18:47 boris-42 meteorfox: without any extra config params
18:50 meteorfox boris-42: hmm... I see. That's true
18:52 meteorfox boris-42: the problem I see is that parallel working iterations, doesn't really show what's going on, it only show how many iterations are active, but not how many actions are being completed over time
19:01 meteorfox https://usercontent.irccloud-cdn.com/file/MHo9Agby/throughput_over_time.png
19:02 meteorfox boris-42: ^^
19:03 boris-42 meteorfox:  we can have one graph
19:03 boris-42 with 2 lines
19:03 boris-42 finished over time and parallel overtime
19:03 boris-42 the algorithm for generation is almost the same
19:03 meteorfox boris-42: right
19:03 boris-42 just need to count amount of iteration in every point
19:03 boris-42 (point is interval of time)
19:04 meteorfox boris-42: exactly
19:04 boris-42 it's even simpler=)
19:04 frickler joined #openstack-rally
19:04 rvasilets joined #openstack-rally
19:04 boris-42 so that can be done when we get those 2 patches in
19:04 meteorfox boris-42:  do you think we could get a trace for each atomic action?
19:06 meteorfox https://usercontent.irccloud-cdn.com/file/JVUTyy9g/testcase_throughput.png
19:07 meteorfox boris-42: ^^ like this, each line would be an atomic_action
19:07 meteorfox boris-42: they don't need to an area chart, or to be stacked. I would actually prefer to be a regular time series plot
19:08 boris-42 meteorfox: it shoul be on other page
19:09 boris-42 meteorfox: "detailed"
19:09 boris-42 meteorfox: on page with overview
19:09 boris-42 we don't need
19:09 meteorfox boris-42: oh no. I meant for the detailed tab
19:09 boris-42 meteorfox: yep
19:10 boris-42 meteorfox: actually it will be better to write all this in spec
19:10 meteorfox boris-42: ok, will do. I write a sped just for the throughput time series, ok?
19:10 meteorfox i'll*
19:10 meteorfox spec*
19:10 meteorfox I can't english
19:14 boris-42 =)
19:14 boris-42 meteorfox: put all your idea
19:14 boris-42 ideas
19:14 boris-42 meteorfox: "Moving forward stuff"
19:16 meteorfox boris-42: ok
19:16 meteorfox boris-42: just the report stuff right?
19:18 boris-42 meteorfox: everything that you think we should imporve in graphs
19:18 meteorfox boris-42: alright, cool.
19:18 boris-42 meteorfox: new graphs / improve graphs / some other info
19:21 boris-42 meteorfox: here is tempalte https://github.com/stackforge/rally/blob/master/doc/specs/template.rst
19:23 meteorfox boris-42: thanks, I'll talk to my team, and see if they have other ideas.
19:23 boris-42 meteorfox: yep that is the idea of specs
19:24 boris-42 to discuss together such big sutff
19:24 meteorfox cool
19:24 boris-42 collect all opinions
19:24 boris-42 and make code only once
19:24 boris-42 =)
19:31 panbalag boris-42, Hi
19:31 boris-42 panbalag: hi there
19:32 panbalag boris-42, In the template https://raw.githubusercontent.com/stackforge/rally/master/doc/specs/template.rst, do the headings and format need to be exactly same when we upload specs?
19:35 boris-42 panbalag: yep
19:40 boris-42 panbalag: you should run locally
19:40 boris-42 panbalag: "tox"
19:42 baker_ joined #openstack-rally
19:44 panbalag boris-42, yeah I did that. but I only did tox -epy27 and it passed..but the unit tests failed because I had my own headings :)
19:45 yfried joined #openstack-rally
19:47 baker joined #openstack-rally
19:51 baker joined #openstack-rally
19:54 baker_ joined #openstack-rally
19:56 baker joined #openstack-rally
19:57 boris-42 panbalag: yep )
19:57 boris-42 meteorfox: btw http://openstacksummitmay2015vancouver.sched.org/event/ec4f18c7ff2c0614d2ecff15251e237f#.VRBwJBCUeQz
20:03 openstackgerrit Prasanth Anbalagan proposed stackforge/rally: Rally verify support for any in-tree Openstack project functional tests  https://review.openstack.org/166487
20:05 openstackgerrit Sergey Skripnick proposed stackforge/rally: WIP: super new rally_gate.py  https://review.openstack.org/163785
20:14 andreykurilin_ joined #openstack-rally
20:19 andreykurilin__ joined #openstack-rally
20:21 e0ne joined #openstack-rally
20:32 meteorfox boris-42: cool. I wish I could go
20:33 boris-42 meteorfox: you are not going to attend?
20:34 meteorfox boris-42: they won't pay it for it, plus I don't think I can travel this year (personal reasons)
20:36 meteorfox boris-42: is there a place where is documented how to use Rally for "production". Like how to setup a proper database for it, hw requirements, etc. The install_rally.sh scripts uses sqlite, I want to set up a proper db for it.
20:37 meteorfox boris-42: I want to install it in our lab, where it will tests several releases
20:37 andreykurilin_ joined #openstack-rally
20:38 meteorfox boris-42: I actually want to include rally as part of the os-ansible project: https://github.com/stackforge/os-ansible-deployment
20:38 boris-42 meteorfox: hm
20:38 meteorfox boris-42: that way users can just install their openstack deployment along with rally
20:38 boris-42 meteorfox: we are working on new install script
20:39 boris-42 meteorfox: why not just using ansible instead of install-rally.sh ?
20:39 boris-42 meteorfox: for that project?
20:39 meteorfox boris-42: right, that's the idea. I just want to know what configuration I need to set
20:39 boris-42 meteorfox: https://github.com/stackforge/rally/blob/master/etc/rally/rally.conf.sample#L387-L392
20:39 boris-42 meteorfox: ^ this basically
20:41 boris-42 meteorfox: and logs
20:41 meteorfox boris-42: what about number of file descriptors and other OS tunings? have you had problems with those?
20:42 boris-42 meteorfox: hm?
20:42 boris-42 meteorfox:  we don't have issues with that
20:42 meteorfox boris-42: ok
20:42 boris-42 meteorfox: I am trying to make something that is easy to use, not that requires magic to run it succesffully
20:43 meteorfox boris-42: yeah.
20:43 boris-42 if something is complicates or required tuning that should be done in upstream
20:44 meteorfox boris-42: cool. so the os-ansible project above, actually will deploy openstack across your infrastructure. I want to see if I can get Rally there and add its DB to the GalerDB cluster
20:46 boris-42 meteorfox: so part of our functional gates are using mysql
20:46 boris-42 meteorfox: so it will deff work with mysql backend
20:46 boris-42 meteorfox: you need just to put connection string proper (actually like you are doing for other services in openstack)
20:46 meteorfox boris-42: we use Maria actually
20:46 boris-42 meteorfox: who cares lolll
20:47 boris-42 meteorfox: they did something like sed -i "s/mysql/maria/g" *
20:47 boris-42 meteorfox: =)
20:48 meteorfox boris-42: lol
20:52 jaypipes joined #openstack-rally
21:09 panbalag andreykurilin_, Hi
21:09 meteorfox boris-42: it looks like hugh saunders, started something in the lines I was thinking of, but this really old now: https://github.com/hughsaunders/rally-jenkins
21:09 andreykurilin_ panbalag: hey!
21:10 panbalag andreykurilin_, looks like pep8 test failed. But its strange that the local run passed.
21:12 andreykurilin_ panbalag: it looks like hacking library released with new rules:(
21:12 andreykurilin_ boris-42: http://logs.openstack.org/87/166487/4/check/gate-rally-pep8/985aaef/console.html
21:13 andreykurilin_ boris-42: it seems to me that we have broken gates due to new rules
21:16 boris-42 panbalag: ^ wanna fix this
21:16 boris-42 andreykurilin_: heh it happens=)
21:17 andreykurilin_ boris-42: http://risovach.ru/upload/2014/06/mem/zhencshiny_52996641_orig_.jpeg
21:17 panbalag boris-42, how do I fix it?
21:18 boris-42 andreykurilin_: LOL
21:18 boris-42 panbalag: so first of all you should switch to master branch
21:18 boris-42 panbalag: after that run git remote update && git pull
21:19 boris-42 panbalag: after that create new branch fix_pep_crap
21:19 boris-42 panbalag: run "tox -e pep8"
21:19 boris-42 panbalag: after that you will get bunch of errors that should be addressed
21:19 panbalag boris-42, ok. let me try that.
21:20 boris-42 panbalag: thanks
21:21 panbalag boris-42, any idea if this is a know issue with python 43 gate test - "UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 1070: ordinal not in range(128)"...other python version tests passed.
21:21 panbalag boris-42, http://logs.openstack.org/87/166487/4/check/gate-rally-python34/29f3b25/console.html
21:24 boris-42 andreykurilin_: ^ any ideas
21:24 boris-42 panbalag: thats python 3.4 no jokes!
21:24 boris-42 =)
21:24 andreykurilin_ heh
21:24 andreykurilin_ one moment
21:24 boris-42 andreykurilin_: maybe file is not in UTF8?
21:25 andreykurilin_ maybe:D
21:25 andreykurilin_ boris-42: I'll try to execute it locally
21:27 andreykurilin_ boris-42: btw, failed test is yours
21:27 andreykurilin_ )
21:28 boris-42 andreykurilin_: ya me not py34 guru
21:29 andreykurilin_ boris-42: oh...I need to recreate env:(
21:34 andreykurilin_ boris-42: I can repeat this issue and it's quite strange. panbalag's patch doesn't change template which are tested by failed tests
21:35 boris-42 andreykurilin_: that's intersting
21:35 andreykurilin_ boris-42: also, tests are finished successfully on master branch:D
21:35 andreykurilin_ wtf?!
21:36 boris-42 andreykurilin_: magic people voodoo people
21:36 exploreshaifali_ joined #openstack-rally
21:36 andreykurilin_ :D
21:37 andreykurilin_ мамба мамба **ямда)
21:38 boris-42 andreykurilin_: tssss they can translate that
21:38 boris-42 =)
21:38 andreykurilin_ boris-42: :)
21:44 andreykurilin_ boris-42: I was wrong and you was right:)
21:44 andreykurilin_ boris-42: I missed with line
21:44 andreykurilin_ boris-42: problem in encoding of panbalag's file
21:45 boris-42 andreykurilin_:  heh
21:45 andreykurilin_ boris-42: possible solution is http://paste.openstack.org/show/195578/
21:45 andreykurilin_ or change encoding of new file
21:46 andreykurilin_ panbalag: ^
21:49 panbalag andreykurilin_, I'll try setting the encoding to utf8 on vim and see if that helps.
21:55 panbalag andreykurilin_, ok. I changed it "[root@localhost in-progress]# file common-rally-verify.rst .... common-rally-verify.rst: UTF-8 Unicode text". Lets see if this works
21:57 openstackgerrit Prasanth Anbalagan proposed stackforge/rally: Rally verify support for any in-tree Openstack project functional tests  https://review.openstack.org/166487
21:58 andreykurilin_ boris-42: btw, do we really need milestone section in specs?
21:58 andreykurilin_ *milestone which relates to OpenStack releases?
22:00 boris-42 andreykurilin_: I was going to remove it in my spec
22:00 boris-42 andreykurilin_: regarding to new format
22:00 andreykurilin_ :)
22:01 andreykurilin_ panbalag: encoding with UTD-8 will not help
22:02 andreykurilin_ panbalag: `open` function uses encoding 'ANSI_X3.4-1968' as default.
22:02 andreykurilin_ panbalag: so you can try to encode your file with this codec or use this solution http://paste.openstack.org/show/195578/
22:04 andreykurilin_ panbalag, boris-42: setting UTF-8 as coder is preferable way for me
22:04 andreykurilin_ boris-42: what you think about it?
22:08 panbalag andreykurilin_, is it ok to modify the code test_specs and check it in.. or does it need to go through review too?
22:08 panbalag andreykurilin_, I mean putting in your solution
22:09 andreykurilin_ panbalag: you can modify and check all locally(using `tox -epy34` command ) :) anf if it works - send to review
22:09 panbalag andreykurilin_, ok
23:07 panbalag boris-42, andreykurilin, pep8 test passed in the last commit https://review.openstack.org/#/c/166487/
23:07 panbalag boris-42, andreykurilin, Its strange because I had not made any changes.
23:12 PerfBeing joined #openstack-rally
23:13 boris-42 panbalag: maybe theydisabled that rule by default
23:33 mwagner_lap joined #openstack-rally
23:38 dmorita joined #openstack-rally
23:54 panbalag boris-42, ok.

| Channels | #openstack-rally index | Today | | Search | Google Search | Plain-Text | summary