Perl 6 - the future is here, just unevenly distributed

IRC log for #rosettacode, 2012-01-28

| Channels | #rosettacode index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:29 mwn3d_phone joined #rosettacode
01:31 BenBE joined #rosettacode
01:32 BenBE joined #rosettacode
01:33 BenBE Hi folks!
01:36 sorear Hello!
01:41 mikemol Hey
01:41 mikemol Long time no see!
01:42 mikemol BenBE: btw: https://github.com/mikemol/mwgeshiglue
01:44 mikemol BenBE: I've identified that GeSHi is the biggest user of CPU on my server, so I need to wrap it with some explicit caching.
01:44 mwn3d_phone joined #rosettacode
01:46 BenBE mikemol: I guessed it might take quite some resources.
01:46 mikemol I figure using ($pfx . sha1($code) . md5sum($code)) as a memcache key is probably appropriate.
01:46 mikemol BenBE: It's most visible when I look at the google analytics data for which pages take the longest time to load.
01:46 mikemol Several of the largest pages *average* a few minutes.
01:46 BenBE mikemol: That's what I also used for the snippet fetching to store the samples.
01:46 sorear I wonder if it would be possible to make GeSHi run on *save*
01:47 BenBE sorear: It is ...
01:47 mikemol BenBE: No, GeSHi runs every time the page renders.
01:47 BenBE Well, not quite, but at the moment you open the page you just saved.
01:47 mikemol sorear's wondering if the GeSHi rendering can be done once, on edit, and never be recalculated after that.
01:48 BenBE mikemol: Yeah, but should be cached once the page has been rendered ...
01:48 BenBE mikemol: Well, you could, but usually don't really want this for the case of parser updates.
01:48 mikemol Kinda sorta. Things are only cached for a max of 15 minutes, and different things in HTTP headers can cause cache misses.
01:49 mikemol That's why I'm not running deflate compression on the server output, for example.
01:49 * sorear wonders if deflate can be handled by squid
01:50 mikemol Actually, now that I think about it, I'm not sure that makes a difference for fixing the cache misses. The client will send different headers, triggering squid to cache miss, regardless of what the server returns.
01:50 mikemol sorear: It's on their wishlist. Apparently, it's been on their wishlist for quite some time, and never got finished.
01:57 BenBE mikemol: Make the caching for code samples already highlighted be around one or more days ...
01:58 mikemol BenBE: Caching is currently front-end, by Squid, and is whole-page. That's a little more coarse-grained than I'd like.
02:00 mikemol If I take each highlighted code sample and cache that in a separate memcached, that lets me preserve caching for code examples which don't change when a large page is changed, and even pages which are otherwise rendering at the same time borrow from each others' work. (When one page's rendering gets far enough ahead of the other's, at least, which would be generally likely in a rush on a large page.)
02:01 mikemol So if you've got two people who hit a page which takes 60s to render, and one initiates a request 10s after the other initated the request, the first request will get 10s worth of work done which will be usable by the first.
02:01 sorear it might even make sense to use a persistant DB for this cache, since it's storing a small amount of data that tends to stay valid for weeks at a time
02:01 * sorear makes clueless comment
02:01 mikemol Er. The first request will get 10s worh of work done which will be useable by the second.
02:01 mikemol sorear: That's a fair point, and something I was pondering earlier.
02:02 mikemol Two observations, though.
02:02 mikemol 1) I'd need to provide a script for flushing that table when a GeSHi engine update is done.
02:02 sorear is the GeSHi engine updated automatically?
02:03 sorear or you could add GeSHi version numbers / git revision hashes / whatever to the cache key
02:03 mikemol 2) That would put much more load on the database, which I've currently allowed myself to become comfortable with mostly sitting on disk.
02:03 sorear (hopefully the cache is size-bounded and has an eviction policy)
02:03 mikemol GeSHi is kinda-sorta updated automatically, in that BenBE ssh's in and updates it.
02:03 mikemol Saves me from having that particular headache. :)
02:03 BenBE sorear: Nope, I've to push it manually via SSH.
02:04 BenBE sorear: The code cache would need about 30k entries which is usually saturated in about 1-2 days.
02:05 mikemol Also, there's the problem of expiring old entries.
02:06 mikemol I suspect the best option is to just use a large memcached, and allow the fresh and frequently-used cached objects to bump out old ones.
02:07 sorear weekly cron job, delete from codecache where date < now - 7*86400
02:07 sorear (I don't know much SQL)
02:07 mikemol Meh. Close enough; it's understanding data relationships that counts. :)
02:08 BenBE http://rosettacode.org/wiki/Rosetta_Code:Village_Pump/Syntax_Highlighting --> The IP guy at the end ...
02:08 fedaykin "Rosetta Code:Village Pump/Syntax Highlighting - Rosetta Code" http://rldn.net/n3US
02:08 mikemol The other problem with putting it in mysql is that it'd cause a great deal of writes to disk, which would block the page render until each is complete.
02:08 mikemol BenBE: Yeah, I saw that. Ugh.
02:09 BenBE I expected my short answer to be precise enough ... but well ...
02:10 mikemol His comment makes him look extraordinarily frustrated. I doubt he'll take any answer well.
02:10 mikemol Also not sure he'll see it unless it happens soon, though. Without an account, there's no way for him to have the wiki notify him when the page changes.
02:11 BenBE Well, I was kinda straight forward in my answers, though. Can't actually blame him ;-) But GeSHi is OSS so I've to expect people to contribute if they want a project to improve.
02:12 BenBE He already replied to my comment with a quite longish one ... not improving on my impression about him ...
02:15 mikemol BenBE: I'd be OK with RC moving to the 1.1.x branch, if we can get existing languages' support ported forward to it.
02:16 mikemol Even if that means using some automated mechanism to translate 10x GeSHi langfiles to an 11x format.
02:17 mikemol Also, there's the github repo for the mw/geshi glue code. Send me a pull request for using 1.1.x, and I'll merge it.
02:19 BenBE mikemol: The 1.1.x uses a completely rewritten parser which is API-incompatible with the 1.0.x one.
02:19 mikemol I know.
02:20 mikemol I don't know the specifics, but I know that much.
02:20 BenBE That's what I mostly ment in the wiki.
02:20 BenBE Also automatic conversion is also hard as the language structure of 11x is mostly context-based whereas 1.0.x is lexing-oriented.
02:21 BenBE The Rule of 1.0.x is: It's comment or string; if not, it's something else. And if it's something else it probably contains keywords, numbers or symbols.
02:22 BenBE In 11x you have to include symbols into every syntax context that can be spawned and where those symbols can appear. That's not always straight forward.
02:24 mikemol Why can't 1.0.x and 1.1.x be included in the same script?
02:25 Coderjoe what was the SMW hidden wikilink markup format again?
02:26 BenBE mikemol: Name conflict with the main GeSHi class. The easy way would be renaming GeSHi 1.0.x internally to GeSHi10, but this would cause update problems ...
02:27 BenBE http://rosettacode.org/wiki/Rosetta_Code:Village_Pump/Syntax_Highlighting <-- Just added a new update ;-)
02:27 fedaykin "Rosetta Code:Village Pump/Syntax Highlighting - Rosetta Code" http://rldn.net/n3US
02:32 Coderjoe is it [[property::value| ]]  ?
02:32 fedaykin http://rosettacode.org/wiki/property::value|_
02:32 mikemol That's one way.
02:32 mikemol The other is to use {{#set (or is it {{{#set ?)
02:33 Coderjoe there is {{#set and {{#declare
02:33 mikemol Wasn't sure exactly what you were saying there for a minute.
02:33 Coderjoe and look what {{#declare is doing: http://rosettacode.org/wiki/Special:Browse/User:Coderjoe-2FSandbox
02:33 fedaykin "User:Coderjoe/Sandbox" http://rldn.net/DfG
02:33 Coderjoe I thought it was working before, but I guess I was mistaken
02:34 mikemol IIRC, that was the quirk you ran into before.
02:35 Coderjoe I'm tempted to change [[Template:Example]] to use the wikilink syntax. iirc that worked last time I poked it. (I remember saying something along the lines of "I'll be damned")
02:35 fedaykin http://rosettacode.org/wiki/Template:Example
02:35 Coderjoe erg. gotta run for a few minutes
02:36 mikemol joule. Handling three conversations at once.
02:39 BenBE Well, I'm off ... cul8r
02:39 mikemol BenBE: If we can find some way to get 1.0.x and 1.1.x to coexist, I'd be happy to set up a way to call into 1.1.x.
02:40 mikemol At the very least, that'll be somewhat necessary for a transition.
02:41 mikemol Also, if you can put together a list of languages in 1.0.x which aren't in 1.1.x, and post it in a blog or some other place I can link to, I could start pushing people in that direction.
02:43 BenBE mikemol: The problem is that you can do things like cpp/11 vs. cpp/9x, or cpp/qt ...
02:44 mikemol BenBE: What I'd do for a transition is add a new tag. <lang2> or <lang11>. Something like that.
02:45 mikemol What I mean by 'pushing people' is watching for people with interest and energy, and directing them over toward getting that problem solved.
02:45 mikemol Well, the problem of missing langauges.
02:45 BenBE Mostly there's the real frontend API missing a re-implementation for 1.1.x
02:46 BenBE You can get highlighted code but that's still quite unflexible.
02:48 BenBE Also the 1.1.x releases are much slower than their 1.0.x counterparts.
02:48 BenBE So if you already now complain about too much load you'll even more so with 1.1.x.
02:48 mikemol Well, I intend to resolve that with caching. ^^
02:49 BenBE :P
02:50 Coderjoe would there possibly be a way to have the mw glue code detect if the language is defined in 1.1.x and if not auto-fallback to 1.0.x?
02:51 mikemol Coderjoe: Sure. The glue code already detects if the language is supported in 1.0.x, by crawling GeSHi's langfile list. Some appropriate means could be devised to see what languages and dialects were supported by 1.1.x.
02:51 Coderjoe though I suppose there is still the issue of using the same class name for both versions
02:51 BenBE Coderjoe: BAsically yes, but as mentioned: The GeSHi classes both are called GeSHi with 1.0.x and 1.1.x and I'm not intending to rename any of those two to avoid the conflict.
02:52 Coderjoe I can't recall, does php have some way to do something similar to c++'s namespaces?
02:53 BenBE Coderjoe: That's the main issue here.
02:53 BenBE Not in the versions of PHP that GeSHi is targeted at.
02:53 BenBE 1.0.x still allows for PHP 4.3.0; while GeSHi 1.1.x allows PHP 5.2.1 and above.
02:54 BenBE Both don't support namespaces.
02:54 BenBE I'm working (low prio) on a GeSHi-REST-API for highlighting which will support both strands, but this is not fully done yet.
02:56 Coderjoe does php have a way to rename classes dynamically (kinda like class_alias, but actually renaming rather than aliasing)?
02:56 mikemol I don't think so.
02:56 Coderjoe ...
02:57 Coderjoe incomplete php docs suck
02:57 Coderjoe what the ehll does class_alias do with no parameters? what about only one?
02:57 BenBE Coderjoe: Not by default.
02:57 Coderjoe bool class_alias ([ string $original [, string $alias ]] )
02:58 mikemol BenBE: Sorry to say, I won't be pointing RC's backend at a separate centralized server. The bandwidth consumed would be a killer.
02:58 BenBE mikemol: Never expected you too. And even if: The API was more targeted towards "occasional" highlighting of snippets to compare different versions.
02:58 mikemol Six months to a year ago, it'd have been a possibility. Now, I'm looking at blowing my bandwidth quota by the end of the year, at my current growth rate.
02:59 BenBE BUT: I've a i7 that want's to be used :P
02:59 mikemol Provider promises that he'll be increasing network quotas soon, though.
02:59 BenBE What's your current quota?
03:00 mikemol 320GiB/mo.
03:00 BenBE 10TB until they limit bandwidth ... for me ATM ...
03:01 BenBE 1GbE before, 100Mbps after ...
03:01 BenBE But now I'm really off ;-)
03:19 Coderjoe argh.
03:20 Coderjoe ottawa county website, y u no work? https://www.miottawa.org/BusinessNames/search.do?type=Business&amp;lastName=rosettacode.org&amp;firstName=
03:20 mikemol Yeah, that's been broken for a while.
03:21 Coderjoe well, i fixed the link you had, and it seemed to work then, even without cookies or the like
03:23 Coderjoe I suspect they did something that forced it to only work with post requests
03:23 Coderjoe the form was post before, but still accepted get requests
03:24 Coderjoe hmm
03:24 Coderjoe get works, as long as my JSESSIONID cookie is there
04:58 r4m joined #rosettacode
10:17 GlitchMr joined #rosettacode
10:34 mwn3d_phone joined #rosettacode
13:30 mwn3d_phone1 joined #rosettacode
13:30 ttmrichter joined #rosettacode
15:45 p6eval joined #rosettacode
16:31 BenBE joined #rosettacode
17:01 kpreid joined #rosettacode
17:10 mwn3d_phone1 joined #rosettacode
17:50 mwn3d_phone joined #rosettacode
18:47 mwn3d_phone1 joined #rosettacode
19:08 mwn3d_phone joined #rosettacode
19:34 mwn3d_phone1 joined #rosettacode
19:34 BenBE joined #rosettacode
20:26 mwn3d_phone joined #rosettacode
20:46 mwn3d_phone1 joined #rosettacode
21:03 mwn3d_phone joined #rosettacode
21:03 mwn3d_phone1 joined #rosettacode
21:15 mikemol *rage*
21:16 mikemol http://rosettacode.org/mw/index.php?title=Web_scraping&amp;diff=prev&amp;oldid=131324
21:16 fedaykin "Web scraping - Rosetta Code" http://rldn.net/8sZ
21:19 sorear ?
21:19 mikemol He replaced an instance of <lang>some source code</lang> with raw HTML+classless CSS.
21:20 sorear so, uh, revert the edit and leave a notice on eir talk page?
21:20 mikemol Try telling at a glance what (if any) semantic difference a the diff might have had. That'd be a major, recurring problem in edit reviewing and monitoring.
21:21 mikemol I did revert it. And he's got a one-day block.
21:21 sorear it's not hard to tell that there's a semantic change
21:21 sorear ey deleted more paragraphs than ey added
21:21 mikemol Well, there was that. Though at least one of those paragraphs wasn't highlighted to begin with.
21:22 mwn3d_phone joined #rosettacode
21:25 sorear I don't see anything on the the talk page
21:26 mikemol I didn't put anything there. I did leave the reason in the block log. Also, it was the same IP as the guy thrashing over GeSHi yesterday. He asked if RC would use his technique, and I told him I would be very unlikely to accept it, given his up-to-then behavior and attitude.
21:27 mikemol I will leave a note in his talk page, though.
21:28 sorear Thrashing?
21:29 mikemol Next week, I'll probably disable anon editing, too. I've noticed a pattern in that the frequent anonymous editors have been coming in with a disregard for communication convention, up to and including intentional flouting of basic behaviors that make communication easier in MW talk pages. I don't want to overreact [more] based this one guy.
21:29 mikemol http://rosettacode.org/wiki/Rosetta_Code:Village_Pump/Syntax_Highlighting#GeShi_power_appears_to_be_very_limited
21:29 fedaykin "Rosetta Code:Village Pump/Syntax Highlighting - Rosetta Code" http://rldn.net/n3US
21:30 mikemol Well, he's one of plausibly two anon editors in that conversation. Or he may have been coming in through two different IPs (such as via a tor exit point).
21:31 mikemol Don't know. But one IP proposed coming up with a new system, and the other provided a proof of concept of a new system.
21:32 mikemol I'd guess that 24.x.x.x is his home IP, and 192.x.x.x is his work IP.
21:33 mikemol Dunno. Both IPs belong to communications companies. I know Shaw Communications sells residential cable access. Dunno about Morewave.
21:37 Coderjoe Services
21:37 Coderjoe » Broadband Wireless Internet Access
21:37 Coderjoe » Business Internet Dial Up Access
21:37 Coderjoe » Morewave Fiber Access
21:38 Coderjoe they also have hosting services
21:38 mikemol Probably hear from him tomorrow on his home IP, Monday on his work IP, or not for a very long time.
21:38 Coderjoe Wireless Coverage
21:38 Coderjoe Burnaby, Coquitlam, Delta, New Westminster, Richmond, Surrey, Vancouver, North Vancouver, South Vancouver
21:39 mikemol Heh.
21:39 Coderjoe ... and in case you missed it: Vancouver
21:39 mikemol Actually, thyis guy is the same as the one I was thinking of wrt flouting communications convention.
21:39 mwn3d_phone Vancouver canada or washington?
21:40 Coderjoe i think CA
21:42 mikemol Hm. Considering it's one guy (across two IPs) who's caused most of my irritation with anon edits, I should probably only block anon edits from his IPs.
21:49 mikemol Done. He could even create an account right this minute and resume editing...but then I'd block him again if he vandalized more pages.
21:57 Util mikemol: In case you missed it back on Jan 26th:
21:57 Util https://github.com/Util/RosettaCode_utilities
21:57 mikemol Cool
21:57 mwn3d_phone joined #rosettacode
22:21 mwn3d_phone joined #rosettacode

| Channels | #rosettacode index | Today | | Search | Google Search | Plain-Text | summary