Perl 6 - the future is here, just unevenly distributed

IRC log for #openstack-rally, 2016-02-15

| Channels | #openstack-rally index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:09 klindgren_ joined #openstack-rally
00:24 openstackgerrit Yaroslav Lobankov proposed openstack/rally: [Verify] Don't create new flavor when flavor already exists  https://review.openstack.org/274483
00:36 klindgren__ joined #openstack-rally
00:49 XuChao918 joined #openstack-rally
00:59 ylobankov joined #openstack-rally
01:07 klindgren_ joined #openstack-rally
01:12 baker joined #openstack-rally
01:24 openstackgerrit Yaroslav Lobankov proposed openstack/rally: [Verify] Adding docs [part 1]  https://review.openstack.org/280025
01:30 baohua joined #openstack-rally
01:34 ylobankov joined #openstack-rally
01:53 klindgren joined #openstack-rally
02:51 klindgren_ joined #openstack-rally
02:56 ylobankov joined #openstack-rally
03:07 klindgren__ joined #openstack-rally
03:17 akrzos joined #openstack-rally
03:31 klindgren joined #openstack-rally
03:54 ylobankov joined #openstack-rally
03:55 baker joined #openstack-rally
04:01 openstackgerrit Yaroslav Lobankov proposed openstack/rally: [Verify] Removing redundant arg from 'reinstall' command  https://review.openstack.org/280057
04:19 kbyrne joined #openstack-rally
04:46 klindgren_ joined #openstack-rally
04:56 shausy joined #openstack-rally
05:04 rdas joined #openstack-rally
05:29 klindgren joined #openstack-rally
05:32 openstackgerrit Yaroslav Lobankov proposed openstack/rally: [Verify] Adding docs [part 1]  https://review.openstack.org/280025
05:36 klindgren_ joined #openstack-rally
05:42 tfreger joined #openstack-rally
05:54 ylobankov joined #openstack-rally
06:07 klindgren joined #openstack-rally
06:09 klindgren_ joined #openstack-rally
06:09 yfried joined #openstack-rally
06:19 klindgren joined #openstack-rally
06:38 harshs left #openstack-rally
07:04 frickler joined #openstack-rally
07:07 aarefiev joined #openstack-rally
07:15 psuriset joined #openstack-rally
07:17 saurabh__ joined #openstack-rally
07:22 saurabh__ Hi there,
07:23 saurabh__ rally task start failing with error-> OperationalError: (sqlite3.OperationalError) unable to open database file [SQL: u'INSERT INTO task_results, given around 26 scenario in single json file and after 10 hours it failed, so any idea on this?
07:25 saurabh__ If executing only single scenario then it is working fine
07:26 saurabh__ can also see some warnings there
07:26 saurabh__ 2016-02-13 00:16:54.625 3493 WARNING rally.common.broker [-] Failed to consume a task from the queue: [Errno 24] Too many open files 2016-02-13 00:16:54.624 3493 WARNING rally.common.broker [-] Failed to consume a task from the queue: Unable to establish connection to https://192.169.123.50:5000/v2.0
07:38 yfried joined #openstack-rally
08:34 rcernin joined #openstack-rally
08:49 arxcruz joined #openstack-rally
09:00 amaretskiy joined #openstack-rally
09:16 psuriset joined #openstack-rally
09:16 saurabh__ joined #openstack-rally
09:16 yfried joined #openstack-rally
09:17 rcernin joined #openstack-rally
09:22 adiantum joined #openstack-rally
09:28 klindgren_ joined #openstack-rally
09:29 mvk joined #openstack-rally
09:29 bapalm joined #openstack-rally
09:59 e0ne joined #openstack-rally
10:09 andreykurilin saurabh__: gi! can you share a full log?
10:09 andreykurilin *hi
10:13 saurabh__ andreykurilin: http://paste.openstack.org/show/486959/
10:17 andreykurilin saurabh__: I saw 2 issues. The first one is a problem with keystone connectivity. Does `rally deployment check` work? The second one relates to inability to change db. Maybe something wrong with rights or there is no enough space at the disk
10:17 rvasilets_ saurabh__, I have no idea what is it yet. Could you share also task file that you run?
10:20 saurabh__ andreykurilin: there is 201GB available on disk
10:22 saurabh__ andreykurilin: and yes rally deployment works before starting test and after failing to connect to keystone, rally tried to put sla failed information into DB
10:24 andreykurilin saurabh__: "Unable to establish connection to https://192.169.123.50:5000/v2.0" says that something gone wrong while connecting keystone. Does 192.169.123.50 is an ip specified in deployment and can you access to it? or Is it private keystone ?
10:29 saurabh__ andreykurilin: yeah if that time keystone was not accessbile so why it was failed to updated DB, for keystone I'll check what happened that time becuase test was runing since 10 hours so may something went wrong with keystone
10:31 saurabh__ andreykurilin: in past I have tried the same test and in this case, rally results just shows that SLA faliled but here it seems rally itself crashing and not able to complete the scenario execution
10:31 andreykurilin it's strange
10:32 andreykurilin saurabh__: what rally release is used?
10:34 saurabh__ andreykurilin: around 6 monts ago updated using master branch
10:34 saurabh__ month*
10:35 andreykurilin saurabh__: so error with db appeared after upgrading to latest master, right?
10:36 saurabh__ andreykurilin: it was working fine just from last 2-3 executions facing this issue
10:36 andreykurilin hm...
10:37 saurabh__ rvasilets_: task file http://paste.openstack.org/show/486964/
10:40 andreykurilin saurabh__: I suppose, we changed something in db schema(I can't remember what it was) before we support db migrations. Do you have a lot of results in your db?
10:42 andreykurilin saurabh__: let's compare db schemas with `sqldiff` tool.
10:42 saurabh__ andreykurilin: yes too many
10:42 andreykurilin saurabh__: ok
10:43 andreykurilin saurabh__: wait a moment please, I'll make a clear sqlite db from the master and we will be able to compare your db schema with the latest one :)
10:43 saurabh__ andreykurilin: yeah sure
10:48 andreykurilin saurabh__: http://andreykurilin.me/share/20160216_clear_rally_db.sqlite
10:51 saurabh__ andreykurilin: could you also help and share some sqlite commands can be used to compare as used sqlite around 3 years ago... :)
10:51 andreykurilin saurabh__: sure
10:52 andreykurilin saurabh__: https://www.sqlite.org/download.html here you can find the one of existing tools, which allow to compare sqlite databases.
10:52 andreykurilin saurabh__: https://www.sqlite.org/sqldiff.html - can compare only schemas. something like `sqldiff --schema database1.sqlite database2.sqlite`
10:53 andreykurilin saurabh__: also, I'm recommend you to make a copy of your db and execute all command over it:)
10:53 andreykurilin *I recommend
10:57 andreykurilin saurabh__: http://manpages.ubuntu.com/manpages/lucid/man1/sqlt-diff.1p.html another tool :)
11:06 openstackgerrit Merged openstack/rally: Remove unnecessary second argument from get method  https://review.openstack.org/279607
11:09 andreykurilin saurabh__: both of tools creates a migration script, which displays a difference:)
11:10 andreykurilin *create
11:26 saurabh__ andreykurilin: sorry couldn't get the time to check... may be after some time will check... I just executed rally giving json only contains single sceanrio and it worked
11:26 saurabh__ scenario*
11:31 andreykurilin saurabh__: I suppose it will fail again in future while trying to update some "field". I recommend you to check the difference of db schemas when you have free time:) Also, db migrations were merged in master several weeks ago, so it will be easier in future to determine db changes and make migrations.
11:34 andreykurilin saurabh__: Btw, maybe, problem was not in db at all
11:36 saurabh__ andreykurilin: yeah problem may not be with DB but there is some problem :(
11:37 saurabh__ andreykurilin: anyway I'll check DB schema... Thanks for you help.
11:37 andreykurilin saurabh__: while keystone was dead, keystoneclient opened too many sockets and python was unable to open one more
11:37 andreykurilin so rally was unable to open db
11:37 andreykurilin 2016-02-13 00:16:54.625 3493 WARNING rally.common.broker [-] Failed to consume a task from the queue: [Errno 24] Too many open files
11:38 saurabh__ andreykurilin: yes that can be the case but I checked ulimit on rally machine and it is unlimited
11:40 saurabh__ andreykurilin: so any workaround possible of this problem if any?
11:41 andreykurilin saurabh__: based on some answers from http://stackoverflow.com, I can assume that ulimit displays not all limits. I'll think about "cheat" for this issue.
11:42 andreykurilin saurabh__: thanks for raising this issue, it looks interesting and not "a usual simple bug":)
11:43 saurabh__ andreykurilin: :-)
12:15 abalutoiu joined #openstack-rally
12:27 openstackgerrit_ joined #openstack-rally
12:30 akrzos joined #openstack-rally
12:35 arxcruz joined #openstack-rally
12:55 damia_pi joined #openstack-rally
13:51 stpierre joined #openstack-rally
13:58 psd joined #openstack-rally
14:48 openstackgerrit Merged openstack/rally: Make install_rally.sh --branch accept git tree-ish  https://review.openstack.org/278837
14:52 moravec joined #openstack-rally
15:04 baker joined #openstack-rally
15:08 adiantum joined #openstack-rally
15:10 arnoldje joined #openstack-rally
15:12 arxcruz joined #openstack-rally
15:16 ylobankov joined #openstack-rally
15:16 psuriset joined #openstack-rally
15:33 psd joined #openstack-rally
15:43 openstackgerrit Chris St. Pierre proposed openstack/rally: Add wrapper for Glance API versions  https://review.openstack.org/264964
16:05 ylobankov joined #openstack-rally
16:12 moravec joined #openstack-rally
16:52 ylobankov joined #openstack-rally
16:52 baker joined #openstack-rally
16:58 baker joined #openstack-rally
17:06 openstackgerrit Roman Vasilets proposed openstack/rally: [WIP] Change structure of rally plugin and plugin references page  https://review.openstack.org/280327
17:24 rcernin joined #openstack-rally
17:46 openstackgerrit joined #openstack-rally
17:59 psuriset joined #openstack-rally
18:13 e0ne joined #openstack-rally
18:16 yfried joined #openstack-rally
18:50 openstackgerrit Srinivas Sakhamuri proposed openstack/rally: Ensure cleanup context use the supplied api_versions in the task  https://review.openstack.org/279694
19:00 pvaneck joined #openstack-rally
19:16 KimJ joined #openstack-rally
19:20 stpierre are other people seeing these kinds of failures: https://rally-ci.tk/logs/_rci_task_OAIDbSXNnJ/rally-pg-py27-unit/console.log
19:20 stpierre i've rechecked and got the same thing, so if it's ephemeral it seems frequent
19:22 baker joined #openstack-rally
19:42 trams Hi, trying to boot vms with rally, but nova complains about not having a networkID specified. Can't seem to find a way to do that? am i blind or have i missunderstood something? :>
19:53 stpierre there are several ways to do that; the best depends on your network setup
19:53 stpierre if you're lucky, you can just use the auto_assign_nic setting, e.g.: https://github.com/openstack/rally/blob/master/rally-jobs/rally-neutron.yaml#L634
19:55 stpierre in some cases you really do have to specify a network id by passing the network id in as part of the 'nics' argument to booting the server, e.g.: "nics": {"net-id": "abcd-some-net-id"}
19:55 stpierre different scenarios take auto_assign_nic and nics in different ways, so you'll need to check each one that's failing before adding them
19:59 e0ne joined #openstack-rally
20:02 trams stpierre: thanks!
20:11 trams stpierre: ok, pases the syntax check, but seem to get a stacktrace. maybe i still missunderstand something. json: https://paste.fedoraproject.org/323178/
20:13 trams stacktrace,  https://paste.fedoraproject.org/323181/55671881/
20:17 openstackgerrit_ joined #openstack-rally
20:18 openstackgerrit_ joined #openstack-rally
20:26 stpierre oh, sorry, nics is supposed to be a list
20:26 stpierre "nics": [{"net-id": "some-net-id"}]
20:28 trams ah thanks!
20:44 openstackgerrit Srinivas Sakhamuri proposed openstack/rally: Ensure cleanup context uses the supplied api_versions in the task  https://review.openstack.org/279694
21:52 srini joined #openstack-rally
21:53 srini https://review.openstack.org/#/c/279694/   is failing on mirantis rally check, because of failing to install a dependency. Any idea how to fix this error?
21:54 stpierre i think everyone's getting the same thing :(
21:54 stpierre i asked about it earlier, no response
22:05 srini ok thanks, hopefully it gets fixed soon.
22:27 itsuugo joined #openstack-rally
22:30 adiantum joined #openstack-rally

| Channels | #openstack-rally index | Today | | Search | Google Search | Plain-Text | summary