Perl 6 - the future is here, just unevenly distributed

IRC log for #perl6-lwp-gsoc, 2014-07-07

| Channels | #perl6-lwp-gsoc index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
02:03 FROGGS joined #perl6-lwp-gsoc
06:44 sergot morning o/
06:57 FROGGS joined #perl6-lwp-gsoc
07:10 dalek joined #perl6-lwp-gsoc
07:29 dalek openssl: 886cd23 | sergot++ | socket.p6:
07:29 dalek openssl: try to get fd of the connection
07:29 dalek openssl: review: https://github.com/sergot/openssl/commit/886cd23665
07:30 sergot FROGGS: why it doesnt work?
07:30 sergot it prints
07:30 sergot P6opaque: no such attribute '$!PIO'
07:30 FROGGS perhaps you have to decontainerize it
07:30 FROGGS (the $sock)
07:31 dalek openssl: fc9448d | sergot++ | socket.p6:
07:31 dalek openssl: decont added
07:31 dalek openssl: review: https://github.com/sergot/openssl/commit/fc9448d306
07:32 sergot still the same
07:32 FROGGS hmmm
07:32 FROGGS try IO::Socket::INET instead of IO::Socket
07:34 FROGGS yeah, that does it...
07:35 sergot yes, thanks!
07:35 FROGGS sadly one can't just print it
07:35 FROGGS but you can pass it around...
07:35 sergot true as well
07:35 FROGGS but, I would recommend you bind it to a Mu $fd
07:35 FROGGS otherwise you introduce a container again
07:36 dalek openssl: 6ccd25c | sergot++ | socket.p6:
07:36 dalek openssl: getattr fixed
07:36 dalek openssl: review: https://github.com/sergot/openssl/commit/6ccd25c00b
07:36 sergot like that?
07:36 sergot I'm not sure if it is the fd. It can be a class or something.
07:38 FROGGS yes, that looks good
07:38 FROGGS it is the handle of the connection, it totally can be an fd
07:40 sergot I assume I can pass it as an argument to SSL_set_fd?
07:42 FROGGS yeah
07:48 sergot $!PIO is MVMOSHandle
07:48 sergot This representation (MVMOSHandle) cannot unbox to a native int
07:48 moritz it probably has the FD as a struct member somewhere
07:49 sergot yes, it has:
07:49 sergot I found it yesterday
07:49 moritz we probably need an nqp op nqp::getfdfromfilehandle or so
07:50 FROGGS hmmm
07:50 sergot ->body->data->fd
07:50 sergot or something
07:52 sergot MVMOSHandle *h; MVMIOFileData *data = (MVMIOFileData *)h->body.data; int fd = data->fd; // it's here :)
07:52 FROGGS that is an interesting problem....
07:53 sergot indeed
07:53 sergot I like what moritz proposed
07:53 FROGGS ohh, I missed that
07:54 sergot I think I can write it for moar
07:55 sergot I shoudn't be hard
07:55 FROGGS what exactly?
07:55 sergot implement nqp::getfdfromfilehandle
07:55 FROGGS is that what moritz proposed?
07:55 sergot 09:49 < moritz> we probably need an nqp op nqp::getfdfromfilehandle or so
07:55 sergot Yes
07:55 FROGGS ahh
07:55 FROGGS ahh dang, and I'm searching in the log >.<
07:56 FROGGS moritz: get a nickname that has a different .chars than ours! :P
07:56 sergot haha :d
07:58 sergot FROGGS: what do you think about this then? :)
07:59 FROGGS it looks like as if it will solve our problem, yes
07:59 FROGGS my first impression was: ohh no, not another op
07:59 FROGGS but I think it is a sensible primitive
08:02 sergot Where should I place it. is io.c a good choice?
08:02 sergot I mean, core/io/io.c
08:02 FROGGS yes
08:03 FROGGS nqp::getfdfromfh might be still okay but is shorter
08:04 FROGGS btw, I'm merging nativecast today, except when jnthn has some concerns...
08:05 FROGGS (in case he does no review I'm merging anyway, because I'm certain it is okay)
08:05 sergot Ok :)
08:05 sergot great
08:12 sergot Isn't src/io/syncfile.c a better place?
08:12 sergot I need struct } MVMIOFileData;
08:12 sergot which is there
08:13 FROGGS sergot: this would also work
08:14 FROGGS hmm
08:14 FROGGS I'd put it in io.c I think because a IO::Socket::INET is not about a syncfile but maybe a syncstream
08:14 FROGGS and your op is more general
08:16 sergot ok
08:16 sergot but MVMIOFileData struct is only in syncfile.c
08:17 sergot so.. hm
08:17 sergot fd is also used only there
08:18 sergot not in syncstream and asyncfile
08:20 FROGGS that could also mean that we do not have a fd for that kind of connection
08:20 sergot that's sad :P
08:23 FROGGS we'll find out :o)
08:24 sergot ok |:)
08:34 sergot Cant I use nativecallcast for this somehow?
08:35 sergot 09:50 < FROGGS> I love Perl 6: https://gist.github.com/FROGGS/79fe3fcf799724fa6106
08:35 sergot like you?
08:35 sergot but still it would work for syncfile
08:35 FROGGS no, please dont
08:36 FROGGS because you do not want to rewrite moarvm in perl 6, and you'd have to do quite a bit just to get it a little bit stable
08:36 sergot ok :)
08:37 FROGGS first you'd need to find out where our connection is created
08:37 sergot ok, will try
08:37 FROGGS and then try to obtain the fd, and just print it in MoarVM's C code where the connection is made
09:09 sergot IO::Socket is a SyncStream, for usre
09:09 sergot sure
09:09 sergot https://github.com/MoarVM/MoarVM/blob/master/src/io/syncsocket.c#L76
09:09 sergot here we do the connection
09:10 sergot and it uses libuv to keep the fd
09:10 sergot and it uses libuv for everything :)
09:18 FROGGS sergot: where does it keep the fd?
09:19 sergot I'm trying to figure this out :)
09:24 FROGGS uv__handle_fd / us__stream_fd should give it to you as it seems
09:27 sergot Can we use it directly from io/syncstream.c?
09:28 sergot fd
09:32 sergot I've found those functions, but I couldn't use them
10:03 sergot FROGGS: see #moar
10:29 FROGGS hmmmm
10:58 dalek joined #perl6-lwp-gsoc
10:59 sergot according to our talk on #moarvm, what is to do for now?
11:01 FROGGS make the low level bindings to openssl
11:02 FROGGS we need that in either case, and then we can talk about how to factor it on the perl 6 level
11:02 FROGGS because then we can just try one way or the other
11:03 sergot ok :)
11:04 FROGGS it might make sense to only bind the things we need for an echo server like this: http://nodejs.org/api/tls.html
11:04 FROGGS since we can use openssl's client tool for testing
11:05 FROGGS we could also flip it around and implement a minimalistic client, in case we have a test server somewhere
11:13 sergot I'll start with an echo server.
11:15 sergot FROGGS: I assume I should use C's sockets, right?
11:16 sergot just for now
11:19 FROGGS that is the only way openssl could 'own' the connection, no?
11:20 sergot true
11:49 dalek openssl: 3a924f0 | sergot++ | / (2 files):
11:49 dalek openssl: client connection functions added
11:49 dalek openssl: review: https://github.com/sergot/openssl/commit/3a924f0062
11:49 sergot it's something...
11:50 * FROGGS pulls
11:51 dalek openssl: 341cad9 | sergot++ | client.c:
11:51 dalek openssl: if connect() fixed
11:51 dalek openssl: review: https://github.com/sergot/openssl/commit/341cad982f
11:52 dalek openssl: 8eea424 | sergot++ | test.p6:
11:52 dalek openssl: say $fd added
11:52 dalek openssl: review: https://github.com/sergot/openssl/commit/8eea424964
11:52 sergot I was trying to connect to filip.sergot.pl:80
11:52 sergot (which is not ssl)
11:53 sergot And I'm thinking if this can cause this read/write problem
11:53 FROGGS as I understand that should work too, no?
11:53 FROGGS ohh
11:53 sergot I thought it should work with non-ssl server too
11:53 sergot write: -1
11:53 sergot which probably means thats something wrong with BIO
11:54 FROGGS perl6-m -Ilib -I../../zavolaj/lib test.p6
11:54 FROGGS SSL_CTX.new(method => SSL_METHOD.new(version => 771))
11:54 FROGGS SSL.new(version => 771, type => 0, method => SSL_METHOD.new(version => 771), rbio => BIO, wbio => BIO, bbio => BIO, rwstate => 1, in_handshake => 0, handshake_func => OpaquePointer, server => 0, new_session => 0, quiet_shutdown => 0, shutdown => 0, state => 20480, rstate => 240)
11:54 FROGGS Cannot locate native library './libclient.so'
11:54 FROGGS :o(
11:54 sergot run: gcc -shared -o libclient.so -fPIC client.c
11:55 FROGGS ahh
11:55 FROGGS FD: 10
11:55 FROGGS write: -1
11:55 FROGGS read == -1 [1]: 0 0 0 0 0
11:56 sergot so, it has created the connection, but it can't write to it
11:57 sergot https://www.openssl.org/docs/ssl/SSL_get_error.html
11:57 sergot "If ret == -1, the underlying BIO reported an I/O error (for socket I/O on Unix systems, consult errno for details)."
11:58 sergot hmm, but SSL_get_error (the value between []) is 1
12:01 FROGGS "
12:01 FROGGS For the transparent negotiation to succeed, the ssl must have been initialized to client or server mode. This is being done by calling SSL_set_connect_state(3) or SSL_set_accept_state() before the first call to an SSL_read(3) or SSL_write() function. "
12:06 dalek openssl: af52a93 | sergot++ | test.p6:
12:06 dalek openssl: add SSL_set_{connect,accept}_state
12:06 dalek openssl: review: https://github.com/sergot/openssl/commit/af52a93c39
12:06 sergot still the same
12:06 FROGGS yeah, same for me, but that seems still needed
12:07 sergot true, thanks I didn't notice that
12:15 FROGGS sergot: it segfaults when I try to my locally running apache (port 443)
12:16 FROGGS connect to*
12:17 sergot try filip.sergot.pl:443
12:17 sergot it works
12:17 sergot I got:
12:17 sergot FD: 11
12:17 sergot write: 9
12:17 sergot read == 5 [0]: 0 0 0 0 0
12:17 sergot it's not -1 anymore
12:17 FROGGS write: 9
12:17 FROGGS read == 5 [0]: 0 0 0 0 0
12:17 FROGGS exactly :o)
12:17 sergot So we know it wont work with 80 :))
12:18 sergot anyway... why do we have zeroes there.
12:18 FROGGS at least not with that connect method
12:21 sergot true
12:24 sergot hmmm
12:27 dalek openssl: 30033b0 | sergot++ | test.p6:
12:27 dalek openssl: connect to port 443
12:27 dalek openssl: review: https://github.com/sergot/openssl/commit/30033b04fe
12:27 dalek openssl: efa3f39 | sergot++ | test.p6:
12:27 dalek openssl: add commented SSL_set_accept_state
12:27 dalek openssl: review: https://github.com/sergot/openssl/commit/efa3f39e33
12:27 dalek openssl: a412e17 | sergot++ | test.p6:
12:27 dalek openssl: say $ssl just before read/write
12:27 dalek openssl: review: https://github.com/sergot/openssl/commit/a412e17f1b
12:28 sergot $ssl looks promising :)
12:28 sergot BIO seems to work
12:29 FROGGS yeah, this seems to work out :o)
12:32 sergot SSL_read says, that it read 5 bytes. Maybe there is something wrong with the CArray[uint8]?
12:34 dalek openssl: 7bc0702 | sergot++ | test.p6:
12:34 dalek openssl: add printing SSL_shutdown result
12:34 dalek openssl: review: https://github.com/sergot/openssl/commit/7bc07028a4
12:35 sergot 0 means: The shutdown is not yet finished.
12:37 dalek openssl: e0699e0 | sergot++ | test.p6:
12:37 dalek openssl: run SSL_shutdown in a loop
12:37 dalek openssl: review: https://github.com/sergot/openssl/commit/e0699e0c9e
12:37 sergot now it works :)
12:37 sergot I mean, just shutdowning
12:39 FROGGS the problem is that SSL_read writes to the thing str-to-carray returns and then you throw that away
12:39 FROGGS you would have to malloc something in C land that does not move due to our gc, and then read that
12:40 sergot my $c = CArray[uint8].new;
12:40 sergot my $read = SSL_read($ssl, $c, 5);
12:40 FROGGS so perhaps wrap that C function to malloc something before calling it
12:40 FROGGS sergot: no, $c can move in memory
12:41 sergot hm, ok
12:42 sergot anyway, SSL_read doesnt use str-to-carray
12:42 sergot SSL_write does
12:42 sergot and it reads from the result of str-to-carray
12:42 sergot this is probably what you meant
12:42 sergot :)
12:42 FROGGS true
12:43 FROGGS no, SSL_read reads into that buffer, no?
12:44 sergot I send here actual code:
12:44 sergot 14:40 < sergot> my $c = CArray[uint8].new;
12:44 sergot 14:40 < sergot> my $read = SSL_read($ssl, $c, 5);
12:44 sergot so it reads into $xc
12:44 sergot $c
12:45 FROGGS that's what I say, it reads into that thing, not reading from it
12:45 FROGGS and $c can move, so this is unsafe
12:45 sergot Ok :)
12:46 FROGGS you have to read into a buffer which is managed by C
12:46 FROGGS I guess NativeCall's explicitly-manage() is meant to help here, but it did not work for me
12:47 sergot Can I just write a function char *get_me_a_buffer_now ?
12:47 sergot in client.c
12:48 FROGGS if it mallocs, sure
12:48 FROGGS I guess we will have OpaquePointer.alloc and OpaquePointer.free some day, but not today :o)
13:08 sergot ok :)
13:47 dalek openssl: fd7ae6a | sergot++ | / (2 files):
13:47 dalek openssl: get_buff added
13:47 dalek openssl: review: https://github.com/sergot/openssl/commit/fd7ae6ae1d
13:47 sergot FROGGS: it works! :)
13:47 sergot write: 9
13:47 sergot read == 5 [0]: < h t m l
13:47 sergot yaay
13:49 sergot pure happiness :)
13:50 FROGGS yay!
13:50 FROGGS btw, if I say it should read 100 bytes, I get back 44... is that the entire message?
13:51 sergot hm, no, it's not the entire message :(
13:53 FROGGS well, that's no problem, we have to read again then :o)
13:55 sergot I'm afraid because of something else.
13:55 sergot We dont have any headers
13:56 sergot just html text
14:09 FROGGS that also does not show headers: openssl s_client -connect filip.sergot.pl:443
14:09 FROGGS then type: GET /
14:10 FROGGS btw, I do that in the test script: say "read == $read [{SSL_get_error($ssl, $read)}]: {$c[0..$read]>>.chr.join('')}";
14:13 sergot we need headers I think
14:14 sergot to make it works with our HTTP::* stuff
14:21 FROGGS sergot: make a proper http request... this seems to do for example: my $s = "GET /foo HTTP/1.1\r\nHost: filip.sergot.pl\r\n\r\n";
14:21 FROGGS so if you send a lax request, you get a lax answer, I think that's fair :o)
14:23 FROGGS this also works: my $s = "GET /foo HTTP/1.0\r\n\r\n";
15:42 FROGGS joined #perl6-lwp-gsoc
16:44 sergot FROGGS++ fair enough :)
16:45 sergot FROGGS: could you close my pull requests?
16:45 sergot the nativecast ones
16:45 FROGGS will do in a bit
17:00 sergot Ok, have a question
17:02 sergot hmm, nvm for now
17:02 sergot We can talk about this later
17:02 sergot just wanted to know, how to connect IO::Socket::SSL with what I did today
17:12 dalek openssl: 9769f7c | sergot++ | test.p6:
17:12 dalek openssl: optimized test
17:12 dalek openssl: review: https://github.com/sergot/openssl/commit/9769f7c10b
17:16 dalek openssl: 2c495ba | sergot++ | test.p6:
17:16 dalek openssl: empty IO::Socket::SSL added
17:16 dalek openssl: review: https://github.com/sergot/openssl/commit/2c495ba299
17:21 sergot It doesnt make sense to implement stuff twice, I want to use HTTP::* with IO::Socket::SSl, FROGGS: do you see it possible in the nearest future ? :)
17:57 sergot Another question, should we write OpenSSL bindings as they are, like, SSL_connect($ssl, ...) or provide more p6 class?
17:59 sergot aaandddd.. how can I write something like this: module SSL::BIO { class BIO {}; class BIO_METHOD {}} and dont have to write SSL::BIO::BIO to get this class?
17:59 sergot I can write class SSL::BIO is repr(...) but what about SSL::BIO::BIO_METHOD then?
18:05 dalek openssl: f773a4c | sergot++ | lib/SSL.pm6:
18:05 dalek openssl: lib/SSL.pm6 added
18:05 dalek openssl: review: https://github.com/sergot/openssl/commit/f773a4cdcc
18:07 sergot I started some cleaning
18:14 FROGGS 1) first you should create the very low level bindings, 2) then you can wrap these in helper subs and 3) create an IO::Socket:SSL that has the same API as IO::Socket::INET
18:15 FROGGS module SSL::BIO { class BIO {}; class BIO_METHOD {}} <-- what should that be?
18:15 FROGGS you don't need the SDL::BIO as a module at all
18:16 FROGGS can't you put a class SSL::BIO and class SSL::BIO_METHOD in lib/SSL/BIO.pm?

| Channels | #perl6-lwp-gsoc index | Today | | Search | Google Search | Plain-Text | summary