Perl 6 - the future is here, just unevenly distributed

IRC log for #gluster-dev, 2017-12-04

| Channels | #gluster-dev index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:21 pkalever joined #gluster-dev
00:27 shyam joined #gluster-dev
01:01 pkalever joined #gluster-dev
01:18 gyadav joined #gluster-dev
02:57 ilbot3 joined #gluster-dev
02:57 Topic for #gluster-dev is now Gluster Development Channel - https://www.gluster.org | For general chat go to #gluster | Patches - https://review.gluster.org/ | Channel Logs - https://botbot.me/freenode/gluster-dev/ & http://irclog.perlgeek.de/gluster-dev/
03:30 psony joined #gluster-dev
03:43 susant joined #gluster-dev
03:44 nbalacha joined #gluster-dev
03:59 msvbhat joined #gluster-dev
04:14 atinm joined #gluster-dev
04:16 itisravi joined #gluster-dev
04:23 major joined #gluster-dev
04:25 vishnu_sampath joined #gluster-dev
04:38 jiffin joined #gluster-dev
04:40 hgowtham joined #gluster-dev
04:42 kmohanan joined #gluster-dev
04:47 Manikandan joined #gluster-dev
04:52 sunnyk joined #gluster-dev
04:53 itisravi joined #gluster-dev
04:54 hgowtham joined #gluster-dev
05:11 poornima_ joined #gluster-dev
05:15 ppai joined #gluster-dev
05:16 ndarshan joined #gluster-dev
05:21 itisravi__ joined #gluster-dev
05:26 skumar joined #gluster-dev
05:26 sanoj joined #gluster-dev
05:29 kdhananjay joined #gluster-dev
05:31 mchangir joined #gluster-dev
05:35 susant joined #gluster-dev
05:56 apandey joined #gluster-dev
05:57 kotreshhr joined #gluster-dev
06:05 nbalacha kdhananjay, ping
06:05 kdhananjay nbalacha: will get back to you in some time. in the middle of another chat..
06:05 nbalacha kdhananjay, ack
06:11 Girjesh joined #gluster-dev
06:22 kdhananjay nbalacha: tell me
06:22 nbalacha kdhananjay, can you take a look at https://bugzilla.redhat.com/show_bug.cgi?id=1471031#c5
06:22 glusterbot Bug 1471031: unspecified, unspecified, ---, nbalacha, POST , dht_(f)xattrop does not implement migration checks
06:22 kdhananjay nbalacha: checking
06:23 nbalacha and let me know if my understanding of xattrops in sharding is correct.
06:23 nbalacha kdhananjay, also, how does truncate work here?
06:23 kdhananjay nbalacha: truncate of a normal sharded file?
06:23 nbalacha kdhananjay, yes
06:24 kdhananjay nbalacha: so if the file is truncated to a larger size, then shard simply updates the aggregated file size xattr
06:24 nbalacha kdhananjay, what about if it is smaller?
06:25 kdhananjay nbalacha: if the file is truncated in the true sense of shrinking the size, shard does multiple things - identifies shards that need to be deleted, deletes them, if the smallest numbered shard here requires truncation, that is done too. then size is updated.
06:25 bennyturns joined #gluster-dev
06:25 nbalacha kdhananjay, would it be an ADD operation in that case?
06:26 kdhananjay nbalacha: xattrops are designed to be "addition of delta". if the delta is negative then it decrements the size by the amount specified
06:27 nbalacha kdhananjay, ok. got it.
06:27 kdhananjay nbalacha: but it is done in an atomic way
06:27 nbalacha kdhananjay, let me know what you think of the comment
06:28 kdhananjay nbalacha: reading of the xattr (getxattr) into memory, followed by adding/subtracting the delta to the in-memory number, followed by writing the resultant number back (setxattr) are atomic at the level of posix
06:28 nbalacha kdhananjay, right - I am thinking more about te race between the dht ops and the xattrops
06:29 kdhananjay nbalacha: ok let me go through your comment first in that case
06:30 kdhananjay nbalacha: give me 5 min
06:30 nbalacha kdhananjay, thanks
06:32 kdhananjay nbalacha: so t3 op (xattrop (ADD)) is by a mount process?
06:33 kdhananjay nbalacha: client i mean
06:33 nbalacha kdhananjay, yes
06:33 xavih joined #gluster-dev
06:34 kdhananjay nbalacha: and the same xattrop wont be executed on file', because some markers on the file are still not set to indicate the file is about to be migrated?
06:34 nbalacha correct
06:36 bennyturns joined #gluster-dev
06:37 kdhananjay nbalacha: where is t4 coming from?
06:38 nbalacha kdhananjay, all xattrops are clients - everything else is the rebalance file migration
06:38 kdhananjay nbalacha: ah ok..
06:38 kdhananjay nbalacha: so t4 is executed on both copies?
06:38 nbalacha kdhananjay, yes, with the changes in the patch
06:39 kdhananjay nbalacha: and x3' == x3 ?
06:40 nbalacha kdhananjay, nope. x3 is the value calculated on the source. x3' is on the target but the xattrop works on a diff xattr value
06:45 kdhananjay nbalacha: right. so x3' missed the x2 updagte
06:45 kdhananjay *update
06:45 kdhananjay nbalacha: at t3
06:46 nbalacha kdhananjay, yes, so the same value sent as part of the xattrop gives 2 different results on disk
06:46 kdhananjay nbalacha: got it
06:55 kdhananjay nbalacha: so is this a correct assumption - every single write (and hence xattrop) will be witnessed by the original src copy?
06:56 kdhananjay nbalacha: ah no. at t6 that assumption breaks
07:00 nbalacha kdhananjay, there are 3 phases (one before the phase1 bits are set, after the phase1 bits are set and after they are removed)
07:06 kdhananjay nbalacha_lunch: got it
07:08 pkalever joined #gluster-dev
07:08 rwheeler joined #gluster-dev
07:10 pkalever joined #gluster-dev
07:50 nbalacha joined #gluster-dev
08:15 Saravanakmr joined #gluster-dev
08:46 kdhananjay nbalacha: so whatever problem you've found in the bz seems relevant
08:46 nbalacha kdhananjay, ok.
08:46 kdhananjay nbalacha: where are reads/lookups/getxattrs etc served from during hte course of the migration?
08:46 kdhananjay nbalacha: same places? src first and in the end destination?
08:46 nbalacha kdhananjay, the src
08:47 gyadav joined #gluster-dev
08:47 nbalacha kdhananjay, reads are always served from the src
08:47 nbalacha kdhananjay, only writes are redirected to both
08:52 kdhananjay nbalacha: hmm then what about reads after t6?
08:58 itisravi joined #gluster-dev
08:58 nbalacha kdhananjay, depends
08:58 nbalacha if the inode ctx was updated to get the new hashed file, it will read from dst
08:59 nbalacha kdhananjay, but there is a potential race there
09:00 nbalacha with the current rebalance code
09:11 ndevos misc, nigelb: I got a forwarded email that show SPF error and delivery failures last week, is that something you have been aware of? http://termbin.com/np54
09:19 kdhananjay nbalacha: ok.. so if the reads are always guaranteed to be served from the good copy ( the copy that witnessed all writes and xattrops so far), then we might just sync all xattrs to dst before marking it as readable?
09:19 kdhananjay nbalacha: instead of changing xattrop fops in dht?
09:20 nbalacha kdhananjay, still leaves a window
09:20 nbalacha if a fop comes after the copy and before dst file has been converted to a data file
09:21 kdhananjay nbalacha: ah! cant those two ops be made atomic?
09:21 nbalacha kdhananjay, I don't see how at the moment
09:22 nbalacha unless you mean change posix_setattr itself itsel?
09:22 kdhananjay nbalacha: ah no. i meant at dht layer.
09:23 nbalacha kdhananjay, I don't see how
09:23 nbalacha because there are multiple dht clients
09:23 kdhananjay nbalacha: ah so then they will also need to start taking locks... understood
09:24 nbalacha kdhananjay, I am making some changes to the dht_migrate_file code to see if this can be handled. Take a look at the patch once it is up.  I will add you as a reviewer
09:25 kdhananjay nbalacha: sure!
09:27 nigelb ndevos: That one is for misc.
09:34 amarts joined #gluster-dev
09:41 misc ndevos: yes, and I have been bugging IT since more than 1 year
10:08 major joined #gluster-dev
10:08 nbalacha joined #gluster-dev
10:09 ndevos misc: ok, thanks, is there a bug where (lack of) progress is reported?
10:10 misc ndevos: a IT ticket, but not related to gluster
10:11 ndevos misc: ah, so the spf record for redhat.com is borked?
10:12 risjain joined #gluster-dev
10:14 devyani7 kshlm, can you please take a look at this: https://paste.opensuse.org/26326759
10:15 kshlm devyani7, Okay. That is strange.
10:15 devyani7 :|
10:15 kshlm I just did a `dep ensure` and it worked.
10:16 kshlm risjain, Is also facing a similar problem btw.
10:16 kshlm risjain, devyani7,  What version of dep do you have
10:16 kshlm `dep version`
10:17 misc ndevos: no, that's more subtle :)
10:17 pranithk1 joined #gluster-dev
10:17 devyani7 kshlm,  version     : v0.3.1
10:17 kshlm I'm on v0.3.1 built from git 83789e2
10:17 misc ndevos: IT do verify the SPF record of redhat.com on their gateway
10:17 risjain kshlm, My issue got resolved, I just started from scratch as you suggested
10:17 misc so if someone send a email from redhat.com to gluster.org which go back to redhat.com, because that's a alias
10:18 kshlm risjain, Cool.
10:18 devyani7 kshlm, yup, same git hash
10:18 misc it got kicked out, because gluster.org server is not in the SPF list of redhat.com
10:18 misc ndevos: so, you will say "why not add it", and that's a fair question
10:18 misc and we can't add it, because there is a limit on the size of record ...
10:18 misc we have been asking for whitelisting to IT since a while
10:19 kshlm devyani7, what commit are you on? Or are you on your own development branch?
10:19 misc and I kinda forgot the status on that
10:19 ndevos misc: hmm, I understand. in any case, its annoying for people getting those rejected mails...
10:19 devyani7 kshlm, its my development branch, but is updated against master.
10:19 misc ndevos: yep
10:20 misc ndevos: I fully agree
10:20 misc and I did escalate the issue, did sent them patch
10:20 misc the departement had time to reord 1 or 2 times since ...
10:21 kshlm devyani7, Could be that the Gopkg.lock and Gopkg.toml have some merge issues.
10:22 kshlm devyani7, Could you checkout the master versions of the files and try.
10:22 kshlm Take a backup of the files.
10:22 kshlm Then run `git checkout origin/master Gopkg.toml Gopkg.lock`
10:22 devyani7 ok
10:26 kshlm devyani7, To me it seems like you guys faced a temporary internet problem at the office.
10:27 devyani7 is it? mmh
10:28 kshlm Could be. Happened to me once.
10:28 kshlm I don't remember the exact faliure messages though.
10:29 kshlm And I think it was github that had borked.
10:31 Shu6h3ndu joined #gluster-dev
10:32 vishnu_kunda joined #gluster-dev
10:43 nbalacha joined #gluster-dev
10:44 nishanth joined #gluster-dev
10:47 pranithk1 left #gluster-dev
10:51 sanju joined #gluster-dev
11:06 sanoj joined #gluster-dev
11:13 itisravi__ joined #gluster-dev
11:28 itisravi__ joined #gluster-dev
11:40 shyam joined #gluster-dev
12:08 kotreshhr left #gluster-dev
12:13 mchangir joined #gluster-dev
12:29 rraja joined #gluster-dev
13:07 nbalacha joined #gluster-dev
13:27 susant left #gluster-dev
14:15 rwheeler joined #gluster-dev
14:17 rwheeler joined #gluster-dev
14:27 shyam joined #gluster-dev
14:33 sunny joined #gluster-dev
14:35 ndevos kkeithley: https://wiki.centos.org/Events/Dojo/Brussels2018
14:38 gyadav joined #gluster-dev
14:44 mchangir joined #gluster-dev
14:49 Manikandan joined #gluster-dev
14:54 msvbhat joined #gluster-dev
15:01 sanju joined #gluster-dev
15:08 bennyturns joined #gluster-dev
15:17 shyam left #gluster-dev
15:25 jiffin joined #gluster-dev
15:37 jiffin joined #gluster-dev
15:48 timotheus1_ joined #gluster-dev
16:01 gyadav joined #gluster-dev
16:17 Girjesh joined #gluster-dev
16:27 pranithk1 joined #gluster-dev
16:48 pranithk1 joined #gluster-dev
16:49 jiffin1 joined #gluster-dev
16:55 risjain joined #gluster-dev
16:59 kdhananjay joined #gluster-dev
18:21 bennyturns joined #gluster-dev
20:35 major joined #gluster-dev
20:47 timotheus1_ joined #gluster-dev
21:38 susant joined #gluster-dev
21:46 susant left #gluster-dev
22:31 bennyturns joined #gluster-dev

| Channels | #gluster-dev index | Today | | Search | Google Search | Plain-Text | summary