Perl 6 - the future is here, just unevenly distributed

IRC log for #gluster-dev, 2015-11-25

| Channels | #gluster-dev index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:45 shyam joined #gluster-dev
00:54 hagarth_ jarrpa: please drop a note on gluster-infra ML
01:05 EinstCrazy joined #gluster-dev
01:05 zhangjn joined #gluster-dev
01:52 zhangjn joined #gluster-dev
02:09 dlambrig_ joined #gluster-dev
02:10 jarrpa hagarth_: Will do!
02:53 Humble joined #gluster-dev
03:28 atinm joined #gluster-dev
03:34 kdhananjay joined #gluster-dev
03:37 kotreshhr joined #gluster-dev
03:38 shubhendu joined #gluster-dev
03:43 kotreshhr left #gluster-dev
03:50 sakshi joined #gluster-dev
03:56 overclk joined #gluster-dev
04:00 mchangir joined #gluster-dev
04:04 josferna joined #gluster-dev
04:04 nbalacha joined #gluster-dev
04:10 vimal joined #gluster-dev
04:24 bkunal joined #gluster-dev
04:26 atalur joined #gluster-dev
04:26 aravindavk joined #gluster-dev
04:28 gem joined #gluster-dev
04:44 nishanth joined #gluster-dev
04:48 aspandey joined #gluster-dev
04:49 kshlm joined #gluster-dev
04:55 kotreshhr joined #gluster-dev
04:58 hgowtham joined #gluster-dev
05:06 skoduri joined #gluster-dev
05:11 itisravi joined #gluster-dev
05:12 poornimag joined #gluster-dev
05:15 Manikandan joined #gluster-dev
05:18 hgowtham joined #gluster-dev
05:18 rraja joined #gluster-dev
05:20 surabhi joined #gluster-dev
05:20 shubhendu joined #gluster-dev
05:25 ndarshan joined #gluster-dev
05:25 surabhi joined #gluster-dev
05:28 zhangjn joined #gluster-dev
05:34 pppp joined #gluster-dev
05:39 ashiq joined #gluster-dev
05:41 poornimag joined #gluster-dev
05:45 rjoseph joined #gluster-dev
05:46 raghu joined #gluster-dev
05:48 rafi joined #gluster-dev
05:57 asengupt joined #gluster-dev
05:58 ggarg joined #gluster-dev
06:03 surabhi joined #gluster-dev
06:08 ppai joined #gluster-dev
06:09 shubhendu joined #gluster-dev
06:09 skoduri joined #gluster-dev
06:10 anekkunt joined #gluster-dev
06:12 Bhaskarakiran joined #gluster-dev
06:26 surabhi joined #gluster-dev
06:32 josferna joined #gluster-dev
06:33 kdhananjay joined #gluster-dev
06:33 ashiq Manikandan++, thanks :)
06:33 glusterbot ashiq: Manikandan's karma is now 37
06:33 primusinterpares joined #gluster-dev
06:35 Manikandan ashiq, np :)
06:42 Humble joined #gluster-dev
06:45 itisravi josferna: hey joseph
06:45 josferna itisravi, hello
06:45 itisravi josferna: /tests/basic/tier/record-metadata-heat.t always seems to be faliling test #16 on upstream master
06:46 itisravi My patch failed regression @ https://build.gluster.org/job/rackspace-r​egression-2GB-triggered/16147/consoleFull but when I tested, the test fails even without any changes on master.
06:46 atinm itisravi, its part of bad test
06:46 josferna itisravi : its been marked bad test
06:46 itisravi josferna: just merged?
06:47 itisravi otherwise why is it complaining?
06:47 itisravi josferna: oh wait my bad
06:47 itisravi josferna: it is ignoring the result.
06:48 josferna itisravi: http://review.gluster.org/12591
06:48 itisravi okay.
07:02 vmallika joined #gluster-dev
07:13 hgowtham joined #gluster-dev
07:19 alexey__ joined #gluster-dev
07:21 pranithk joined #gluster-dev
07:29 skoduri joined #gluster-dev
07:33 rjoseph joined #gluster-dev
07:36 poornimag joined #gluster-dev
07:39 ppai joined #gluster-dev
07:59 zhangjn joined #gluster-dev
08:01 zhangjn joined #gluster-dev
08:29 zhangjn joined #gluster-dev
08:30 pranithk xavih: Hey, I have some doubts about ec algorithm... could you ping me when you are online
08:30 xavih pranithk: hi
08:31 pranithk xavih: hey!, remember the mail you sent me describing the algo? How do we construct the matrix M?
08:31 pranithk xavih: I mean the matrix M is [x^3 x^2 x 1] right?
08:32 xavih pranithk: the only requisite for M is to have N linearly independent rows
08:32 xavih pranithk: there are many ways to do that, but one possibility is the vandermonde matrix, which is used in current implementation of ec
08:33 pranithk xavih: oh let me take a look at that matrix
08:34 xavih pranithk: a vandermonde matrix uses a set of N distinct numbers, and each row is constructed by exponentiating this number to 0..K-1
08:34 pranithk xavih: Do you have a helpful link which describes it?
08:36 xavih pranithk: https://en.wikipedia.org/wiki/Vandermonde_matrix this describes well enough how it's constructed and some properties
08:36 pranithk xavih: Oh you used transpose of the matrix?
08:36 xavih pranithk: I tweaked it a bit to be able to optimize the computation of each row :P
08:37 pranithk xavih: got it
08:37 xavih pranithk: but it's the same as long as the linear independency is kept
08:37 pranithk I see
08:37 xavih pranithk: it's not exactly the transpose. I only flipped the columns
08:37 pranithk xavih: Okay, I will go through these. I may bug you for 6 more hours today. I need to understand the algo today
08:37 pranithk xavih: I see
08:38 xavih pranithk: instead of using x^0, x^1, x^2, ..., x^(k-1), I used x^(k-1), ..., x^2, x^1, x^0
08:38 pranithk xavih: got it
08:39 xavih pranithk: when a multiplication is done by vector (a, b, c, ..., k) , you get: ax^(k-1) + bx^(k-2) + ... + kx^0
08:39 pranithk xavih: yeah
08:40 xavih pranithk: this can be rewritten as: (((ax + b) * x + c) * x + ...) + k, which is easier to compute
08:40 pranithk xavih: question... in the mail, why 3^3 became '5'? is there any 'mod' operation going on?
08:40 pranithk xavih: ah! got it
08:41 xavih pranithk: all operations are done in a galois field instead of the normal integers field
08:41 xavih pranithk: in a galois field, addition is implemented as xor (i.e. there are no carries)
08:41 pranithk xavih: GF(2^8) ?
08:42 xavih pranithk: multiplications are implemented exactly equal as the old school method, but changing normal addition by xors
08:42 xavih pranithk: yes, I use a galois field of 8 bits
08:42 xavih pranithk: for example 3 + 1 = 2 in a galois field
08:43 pranithk xavih: okay. I learnt galois filed just yesterday. Let me see how much I will understand now...
08:43 xavih pranithk: for GF(2^8), the modulus is 0x11D. This means that when you multiply 0x80 by 2, you get 0x100. You xor the modulus, getting 0x100 + 0x11D = 0x1D
08:44 xavih pranithk: so 0x80 * 2 = 0x1D (in a GF(2^8) mod 0x11D)
08:45 pranithk xavih: Isn't 0x11D 285?
08:45 xavih pranithk: you apply the modulus when a result has more bits that the field size
08:46 xavih pranithk: yes
08:46 xavih pranithk: but it's easier to think in hex
08:46 xavih pranithk: or even binary
08:46 pranithk xavih: it is not a prime right? oh but in non-prime field it is an algebraic expression that is the prime...
08:46 xavih pranithk: it's not a prime in the integer field, but it's a prime in the galois field
08:47 xavih pranithk: yes, it represents an irreductible polynomial
08:47 pranithk xavih: got it
08:51 shubhendu joined #gluster-dev
09:16 ppai joined #gluster-dev
09:39 zhuchkov joined #gluster-dev
09:42 kbyrne joined #gluster-dev
09:43 shubhendu joined #gluster-dev
09:47 pranithk xavih: What is the size of the co-efficient in ec? 512 bytes?
09:48 pranithk xavih: "Coefficients a, b, c and d are taken from the data to encode. One common  implementation is to use 1 byte for each coefficient (ec uses another  implementation to increase performance, but it's equivalent to this)"
09:50 kbyrne joined #gluster-dev
09:55 xavih pranithk: in ec each coefficient is only 8 bits. Another thing is how are they interpreted to take advantage of SIMD
09:57 pranithk xavih: oh, let me try to understand ec_method_encode with this
09:57 pranithk xavih: what is chunk size then?
09:58 xavih pranithk: instead of loading bytes into registers and do the math, each register is loaded with one bit of each of the 8 bits of the GF(2^8)
09:58 pranithk xavih: Oh that is the minimum size of the file...
09:58 xavih pranithk: 8 registers are used to load the 8 bits, and xors are done between the registers to compute the math
10:00 xavih pranithk: since one register has many bits, instead of loading one bit in each registers, multiple consecutive bits are considered the first bit of multiple GF(2^8) elements. The following sequence of bits is considered the second bit of elements, and so on
10:00 josferna joined #gluster-dev
10:00 Manikandan joined #gluster-dev
10:01 pranithk xavih: GF (2^8) has 256 elements right?
10:02 xavih pranithk: I decided to use sequences of 512 bits for each bit of the GF(2^8) elements
10:02 xavih pranithk: yes, there are 256 elements
10:02 xavih pranithk: 512 bits are 64 bytes. Since I need 8 of these sequences, a total of 512 bytes is needed as a minimum
10:03 pranithk xavih: I didn't understand " I decided to use sequences of 512 bits for each bit of the GF(2^8) elements" :-(.
10:03 pranithk xavih: Why do we need 512 bits for each bit?
10:03 xavih pranithk: to take advantage of SIMD
10:04 pranithk xavih: Oh, wait, I don't know much about SIMD, will ping you after I learn about it
10:04 xavih pranithk: it simply means Single Instruction Multiple Data
10:05 pranithk xavih: yeah, wikipedia says the same, but I am not sure what it means... so reading :-)
10:05 xavih i.e. when you have to do the same operation on multiple elements, if you can load more than one element in processor registers, you can do a single operation that affects more than one element
10:06 pranithk xavih: oh
10:06 xavih pranithk: an xor is an easy example. Suppose you want to do an xor of two memory regions. You can load data byte by byte, do the xor and store the byte
10:06 pranithk xavih: yes, this I understand :-)
10:07 pranithk xavih: what will happen in SIMD?
10:07 xavih pranithk: but you can also load multiple bytes in a single read, do a single xor and a single store and you will have done the same number of operations but computed more than one byte
10:08 xavih pranithk: in current ec implementation, each multiplication is made on 64 elements of the GF(2^8)  simultaneously
10:08 pranithk xavih: each element is 8 bits?
10:08 xavih pranithk: using sse and avx I can do the same with 128 and 256 elements simultaneously
10:08 xavih pranithk: yes
10:08 pranithk xavih: ah that makes sense, thats why 512?
10:09 xavih pranithk: 512 is to make room for a possible extension that is able to handle 512 bits in a single instruction
10:13 xavih pranithk: maybe this helps you understand what ec is doing: you should interpret the data to be encoded as a stream of bits
10:14 xavih pranithk: the first 512 bits are treated as the first bit of each of 512 elements of GF(2^8)
10:15 xavih pranithk: the following 512 bits are treated as the second bit of these 512 elements
10:15 xavih pranithk: and so on until eighth bit
10:16 pranithk xavih: ah! now I understand what is EC_METHOD_WIDTH
10:16 pranithk xavih: right?
10:16 xavih pranithk: a0, a1, a2, ..., a511 are the first bits. b0, b1, ..., b511 the second bit, etc
10:16 xavih pranithk: yes
10:17 pranithk xavih: okay. wait let me digest this for some time. I will ask you some more doubts
10:17 xavih pranithk: then the first element of the GF(2^8) is composed by a0, b0, c0, d0, e0, f0, g0 and h0
10:17 xavih pranithk: ok
10:17 pranithk xavih: no no go ahead
10:17 pranithk xavih: I will see the complete thing and then will check if I understood it
10:18 xavih pranithk: you can interpret each block of 512 bytes as a bit matrix composed as follows:
10:18 xavih pranithk: a0 a1 a2 ... a511
10:18 xavih pranithk: b0 b1 b2 ... b511
10:19 xavih pranithk: ...
10:19 pranithk xavih: got it
10:19 xavih pranithk: h0 h1 h2 ... h511
10:19 xavih if you read by rows, you get the contents of the file
10:19 xavih pranithk: if you read by columns, you get the elements of the GF(2^8)
10:20 pranithk xavih: wait, I think I am slowly understanding...
10:21 xavih pranithk: once you have the data organized in this way, a multiplication is simply the xor of some bits in some order
10:22 pranithk xavih: hmm...
10:23 xavih pranithk: if you want, I can show an small specific example step by step in GF(2^2)
10:24 pranithk xavih: that would be great I think.
10:25 xavih pranithk: ok
10:25 xavih pranithk: let's take 2 elements of GF(2^2): A and B
10:25 pranithk xavih: okay
10:26 xavih pranithk: both elements have 2 bits: A=(a1 a0) and B=(b1 b0) (a0 and b0 are the least significant bit)
10:26 pranithk xavih: okay
10:26 xavih pranithk: BTW, the modulus in GF(2^2) is 7
10:26 xavih pranithk: ok. First the most simple operation: addition
10:26 pranithk xavih: addition is just xor no?
10:26 xavih pranithk: addition in a galois field is the xor
10:26 xavih pranithk: yes
10:27 xavih pranithk: so A '+' B = A xor B
10:27 pranithk xavih: multiplication is multiplication then mod by 7?
10:27 xavih pranithk: wait, wait, let's go step by step
10:27 pranithk xavih: cool
10:28 xavih pranithk: if we try to do the addition bit by bit, we can see that we only have to xor bits with the same weight, right ?
10:28 xavih pranithk: i.e. if C = A xor B, then c0 = a0 xor b0 and c1 = a1 xor b1, right ?
10:28 pranithk xavih: got it, yes
10:29 xavih pranithk: ok, now the multiplication
10:29 pranithk xavih: yes..
10:29 xavih pranithk: first we'll do a multiplication with actual numbers instead of symbols...
10:29 xavih pranithk: let's take A=2 and B=3
10:29 pranithk xavih: okay
10:31 xavih pranithk: using the old school multiplication method, we have (not sure how this will appear...):
10:31 xavih pranith:      10
10:31 xavih pranith:    * 11
10:31 xavih pranith: -------
10:31 xavih pranith:      10
10:31 xavih pranith: +   10
10:31 glusterbot xavih: -----'s karma is now -1
10:31 xavih pranith: -------
10:31 glusterbot xavih: pranith's karma is now -1
10:31 glusterbot xavih: -----'s karma is now -2
10:31 xavih pranith:     110
10:31 glusterbot xavih: pranith's karma is now -2
10:32 pranithk xavih: cool, I got the same answer
10:32 xavih pranithk: sorry for this :-/
10:32 xavih pranithk++
10:32 glusterbot xavih: pranithk's karma is now 32
10:32 xavih pranithk++
10:32 glusterbot xavih: pranithk's karma is now 33
10:32 pranithk xavih: doesn't matter xavi, go ahead
10:33 xavih pranithk: ok, since the result is bigger than 2 bits, we need to apply the modulus
10:33 pranithk xavih: yes
10:33 xavih pranith:     110
10:33 xavih pranith:   + 111
10:33 xavih pranith: -------
10:33 xavih pranith:     001
10:33 glusterbot xavih: -----'s karma is now -3
10:33 glusterbot xavih: pranith's karma is now -3
10:34 xavih pranithk: so we have that 2 * 3 = 1 in GF(2^2) with mod=7
10:34 pranithk xavih: for modulus, we just add it is it?
10:34 xavih pranithk: yes, because we want to remove the highest bit and this is done by xoring (adding) the modulus
10:35 xavih pranithk: right ?
10:35 pranithk xavih: hmm... how do we compute modulus? (the elements in GF (2^2) are [0, 1, x, x+1] right?
10:35 xavih pranithk: yes
10:36 xavih pranithk: the modulus is computed by searching a polynomial that is not divisible by any other polynomial in the field
10:36 pranithk xavih: oh x^2 + x + 1?
10:37 xavih pranithk: yes, for GF(2^2),
10:38 pranithk xavih: okay cool.
10:38 xavih pranithk: being exact, not all irreducible polynomials are valid. You also need to make sure that there exists a generator polynomial that can generate all elements of the field using that polynomial
10:38 pranithk xavih: we completed till multiplication before I digressed about modulus
10:39 xavih pranithk: but this is a theoretical detail not needed for anything else
10:39 pranithk xavih: okay.
10:39 xavih pranithk: ok, now let's do the same with symbols
10:39 pranithk xavih: cool
10:41 pranithk xavih: will drink some water and come. You keep typing...
10:41 xavih pranith:           a1    a0
10:41 xavih pranith:         * b1    b0
10:41 xavih pranith: ------------------
10:41 xavih pranith:           a1    a0
10:41 xavih pranith: +  b1     b0
10:41 glusterbot xavih: ----------------'s karma is now -1
10:41 xavih pranith: ------------------
10:41 glusterbot xavih: pranith's karma is now -4
10:41 glusterbot xavih: ----------------'s karma is now -2
10:41 xavih pranith:    b1 (a1 + b0) a0
10:41 glusterbot xavih: pranith's karma is now -5
10:42 xavih pranithk: the key here is to decide when we need to apply the modulus
10:42 xavih pranithk: it seems easy: when b1 is 1, we need to apply the modulus, when it's 0, we don't need to apply it
10:43 xavih pranithk: if b1 is 1, then we'll need to xor bits 0 and 1 of the result with '1' (because the modulus has two '1' in those positions)
10:44 xavih pranithk: if b1 is 0, we don't need to do anything, but it can be seen as xoring the value '0'
10:44 xavih pranithk: so, if b1 is 1, we xor with '1', and if b1 is 0, we xor with '0'
10:44 xavih pranithk: this is the same than xoring bits 0 and 1 always with b1
10:45 xavih pranithk: so the result, bit by bit, of C = A * B is:
10:45 ppai joined #gluster-dev
10:45 xavih pranithk: ooops
10:45 xavih pranithk: I wrote incorrectly the equation... :-/
10:47 xavih pranith:                 a1         a0
10:47 xavih pranith:               * b1         b0
10:47 xavih pranith: ------------------------------
10:47 xavih pranith:               a1*b0      a0*b0
10:47 xavih pranith: +  a1*b1      a0*b1
10:47 xavih pranith: ------------------------------
10:47 glusterbot xavih: ----------------------------'s karma is now -1
10:47 glusterbot xavih: pranith's karma is now -6
10:47 glusterbot xavih: ----------------------------'s karma is now -2
10:47 xavih pranith:    a1*b1 (a1*b0 + a0*b1) a0*b0
10:47 glusterbot xavih: pranith's karma is now -7
10:47 xavih pranithk: now it's ok
10:48 xavih pranithk: in this case, we need to xor bits 0 and 1 with a1*b1
10:48 xavih pranithk: so we have c0 = a0*b0 + a1*b1, c1 = a1*b0 + a0*b1 + a1*b1
10:48 xavih pranithk: this is the result of the multiplication of two generic numbers in GF(2^2)
10:49 xavih pranithk: now we can optimize this by building equations for each possible number B
10:49 xavih pranithk: for example, for B=3 (b0 =1, b1 = 1), we can simplify the equations this way:
10:50 xavih pranithk: c0 = a0 + a1, c1 = a0
10:50 pranithk xavih: I understood till now :-)
10:50 xavih pranithk: we can do that for each possible number and get equations for each resulting bit that only depend on xors of input bits
10:51 xavih pranithk: this is what ec does
10:51 pranithk xavih: These optimizations are what are there in gf_mul_add_AB?
10:51 pranithk xavih: cool,
10:51 xavih pranithk: yes
10:52 pranithk xavih: so you hand computed all these things for writing code?
10:52 xavih pranithk: note that instead of using c0 = a0 + a1 for each bit, I can load 64 bits 0 of 64 numbers into one cpu register and 64 bits 1 of the 64 numbers into another register
10:52 xavih pranithk: then doing a simple xor I'll get the bit 0 of all the 64 results
10:53 xavih pranithk: no, I built a program to find all combinations :P
10:53 pranithk xavih: Devuda!! (means OMG)
10:53 xavih pranithk: it's too work to do so by hand in GF(2^8)
10:53 pranithk xavih: exactly
10:54 xavih pranithk: it's even better if we use SSE or AVX, where the registers are 128 and 256 bits long
10:54 pranithk xavih: but if we give the same data and same matrix we have for a different implementation we get the same values though?
10:54 xavih pranithk: the same exact operations work on 2 or 4 times more data
10:54 pranithk xavih: got it
10:54 xavih pranithk: not exactly
10:55 kshlm misc, are you here?
10:55 kshlm csim, ^
10:55 pranithk xavih: oh! why?
10:55 xavih pranithk: since I'm interpreting the data in a special way, you would need to "reconfigure" the input data before applying a more standard algorithm
10:55 csim kshlm: I am
10:55 xavih pranithk: once the transformation if done, the result is the sameç
10:55 csim kshlm: but I am not qualified for matrix operations
10:56 pranithk xavih: but the performance for doing transformation will take performance hit?
10:56 xavih pranithk: if you do the transformation, yes, but I don't really need to transform the data
10:56 xavih pranithk: I read the data in this "special" way and I write it to the bricks with the same encoding
10:57 xavih pranithk: so no transformation is needed really
10:57 csim ( despites people calling me the Keanu Reeves of Paris", I am not also qualified for anything related Matrix )
10:57 xavih pranithk: you only need to transform it if you want to apply other algorithms that expect data to be encoded byte by byte
10:57 kshlm csim, Don't worry. I won't ask you to multiply anything :)
10:59 pranithk xavih: Hmm...
10:59 pranithk xavih: Let me take a look at the code with this new understanding...
10:59 xavih pranithk: ok
11:00 pranithk xavih: Is this why we get better performance than jerasure?
11:01 xavih pranithk: I don't know exactly how jerasure implements multiplication, but probably this encoding is the reason
11:01 xavih pranithk: jerasure is worse for small blocks, but this method is really good with small blocks because it takes full advantage of processor caches
11:02 xavih pranithk: using optimum block sizes for jerasure and this method, this one runs 3 times faster than jerasure
11:03 pranithk xavih: got it
11:03 xavih pranithk: for small blocks, this method runs up to 10 times faster
11:03 zhuchkov joined #gluster-dev
11:06 ppai joined #gluster-dev
11:06 pranithk xavih: I think I got it. I will read the whole log once more.
11:06 pranithk xavih: Till when are you online?
11:07 xavih pranithk: about 3 hours
11:07 pranithk xavih: cool. Got it
11:13 kkeithley1 joined #gluster-dev
11:15 overclk joined #gluster-dev
11:27 ppai joined #gluster-dev
11:28 Manikandan joined #gluster-dev
11:40 csim restarting gerrit to take the change in account for replication
11:47 EinstCrazy joined #gluster-dev
11:48 vmallika joined #gluster-dev
11:48 zhangjn joined #gluster-dev
11:49 zhangjn joined #gluster-dev
11:50 EinstCrazy joined #gluster-dev
11:51 josferna joined #gluster-dev
11:53 kotreshhr joined #gluster-dev
12:03 jdarcy joined #gluster-dev
12:10 mchangir joined #gluster-dev
12:17 pranithk xavih: what is the reason to take 8 registers at a time?
12:18 xavih pranithk: using galois fields other than GF(2^1), GF(2^2), GF(2^4) or GF(2^8) makes bad alignments of data, causing more RMW cycles
12:19 xavih pranithk: GF(2^4) has only 15 valid values (from 1 to 15), and this could be not enough if you have more than 15 bricks in a single disperse set
12:19 xavih pranithk: so GF(2^8) is the smallest field with enough space for all supported configurations
12:21 pranithk xavih: what can be done to decrease the smallest fragment size of 512 bytes?
12:21 pranithk xavih: Can we do anything?
12:21 xavih pranithk: we can reduce it, but reducing it too much will make some processor extensions unusable
12:21 pranithk xavih: you mean reducing chunk-size?
12:22 xavih pranithk: for example, we could reduce it to 64 bytes (8 bytes per bit). This will work with 64 bits registers, but SSE and AVX won't be valid since they could not read an entire register
12:22 xavih pranithk: chunk size is the smallest fragment
12:23 pranithk xavih: yeah
12:24 xavih pranithk: we could reduce the fragment size to 256 bytes. This will work with all currently available extensions, since AVX uses registers of 256 bits
12:24 xavih pranithk: however intel will have 512 bits AVX extensions available soon
12:24 pranithk xavih: I want to understand your terminology... when you say "8 bytes per bit" what exactly does it mean? a0,a1,a2....a63? b0,b1,b2,...b63 h0,h1....h63?
12:25 pranithk xavih: got it about the reason for 512 bytes.
12:25 xavih pranithk: I mean that I interpret 8 bytes of data as one bit. Yes, they are a0,a1,...,a63
12:25 pranithk xavih: basically, this will also lay the smallest file size after which the ec will start giving you benefits...
12:26 pranithk xavih: if we have 4+2, ec will be beneficial only if the file size is > 2KB right?
12:27 pranithk xavih: I am wondering if we can offer more things for people who want to benefit from ec with small files as well
12:28 xavih pranithk: yes and no. In theory you are right, however note that 512 is, probably, the smallest block size that any file system will assign to a file, so even if we could reduce that to a single byte, the underlying file system will still use 512 bytes or more
12:30 pranithk xavih: yep, you are right!!
12:31 pranithk xavih: cool then. That answers most of my questions.
12:31 pranithk xavih: Do you have a program with which you generated the gf_mul_addAB functions?
12:33 xavih pranithk: yes
12:38 pranithk xavih: with this implementation we can have max of 256 bricks under EC right?
12:38 xavih pranithk: 255 for each disperse set (0 cannot be used)
12:40 pranithk xavih: I think it would be simpler to understand the program you wrote to generate these functions than these functions?
12:42 xavih pranithk: not sure. All I've explained is not related to the ec_mul_add functions. These functions are only the implementation of the theory for specific cases. The complexity to understand it remains the same I think
12:42 pranithk xavih: hehe
12:44 pranithk xavih: okay xavi, thanks a lot for the explanations so far. I will get back to you if I have any doubts....
12:44 xavih pranithk: no problem :)
12:57 vmallika joined #gluster-dev
13:00 surabhi joined #gluster-dev
13:06 ppai joined #gluster-dev
13:10 rjoseph joined #gluster-dev
13:16 kotreshhr left #gluster-dev
13:17 Manikandan joined #gluster-dev
13:30 kkeithley_ @learn backports as Request a bug fix to be backported to a particular branch at https://public.pad.fsfe.org/​p/gluster-backport-requests
13:30 glusterbot kkeithley_: The operation succeeded.
13:31 kkeithley_ do we have a document/checklist for the tasks a release manager has to perform?
13:34 anoopcs rastar, ^^
13:36 anoopcs kkeithley_, I remember this thread. http://www.gluster.org/pipermail/glu​ster-devel/2015-October/047024.html
13:37 anoopcs I don't know whether we have a generic document.
13:52 kkeithley_ thanks anoopcs++
13:52 glusterbot kkeithley_: anoopcs's karma is now 22
13:54 anoopcs kkeithley_, np.
14:12 rafi1 joined #gluster-dev
14:22 ndevos kkeithley_: http://gluster.readthedocs.org/en/latest/Co​ntributors-Guide/GlusterFS-Release-process/ ?
14:22 jiffin joined #gluster-dev
14:31 kkeithley_ thanks ndevos++
14:31 glusterbot kkeithley_: ndevos's karma is now 218
14:39 Humble joined #gluster-dev
14:41 hagarth_ joined #gluster-dev
14:45 shyam joined #gluster-dev
14:46 hagarth_ ndevos: be there in a minute
14:46 ndevos hagarth_: sure, np!
14:54 Humble joined #gluster-dev
15:00 nbalacha joined #gluster-dev
15:09 pranithk joined #gluster-dev
15:11 vmallika joined #gluster-dev
15:15 kotreshhr joined #gluster-dev
15:15 kotreshhr left #gluster-dev
15:45 kotreshhr joined #gluster-dev
15:54 rafi joined #gluster-dev
16:00 hagarth_ pranithk++
16:00 glusterbot hagarth_: pranithk's karma is now 34
16:07 dlambrig_ joined #gluster-dev
16:09 ggarg joined #gluster-dev
16:12 cholcombe joined #gluster-dev
16:19 vmallika joined #gluster-dev
16:45 kotreshhr joined #gluster-dev
16:55 mchangir joined #gluster-dev
17:14 EinstCrazy joined #gluster-dev
17:15 kotreshhr left #gluster-dev
17:43 skoduri joined #gluster-dev
18:11 dlambrig_ joined #gluster-dev
18:28 Chr1st1an joined #gluster-dev
18:43 shaunm joined #gluster-dev
20:46 ggarg joined #gluster-dev
21:11 hagarth_ joined #gluster-dev
21:31 bmikhael joined #gluster-dev
21:31 bmikhael joined #gluster-dev
21:32 bmikhael joined #gluster-dev
21:55 cristov joined #gluster-dev
23:47 EinstCrazy joined #gluster-dev

| Channels | #gluster-dev index | Today | | Search | Google Search | Plain-Text | summary