mine's clever because it does 10 requests/10 seconds.
2012-04-30 12143, 2012
kepstin-laptop
in such a way that if someone sends a continuous stream of requests, it'll end up doing 1/1.
2012-04-30 12124, 2012
kepstin-laptop
(well, I actually did 10 requests in 11 seconds because rounding :)
2012-04-30 12107, 2012
alastairp
hmm
2012-04-30 12111, 2012
alastairp
kepstin-laptop: we should do that too?
2012-04-30 12110, 2012
ianmcorvidae
it's somewhat of a hack based on how we *happen* to know the MB webservice is actually ratelimited
2012-04-30 12122, 2012
ianmcorvidae
questionable whether it should be in an official/semi-official library
2012-04-30 12153, 2012
alastairp
:-P
2012-04-30 12133, 2012
pronik` joined the channel
2012-04-30 12127, 2012
alastairp
the idea is, you can have lookups inbetween other actions such as fingerprinting
2012-04-30 12149, 2012
alastairp
may as well not have spurious sleeps that do nothing but slow down an app
2012-04-30 12158, 2012
kepstin-laptop notes that his code wouldn't help the doing other stuff while fingerprinting case
2012-04-30 12126, 2012
kepstin-laptop
since if there's a continuous series of requests, it'll queue them up and issue them once per second :)
2012-04-30 12157, 2012
ianmcorvidae
I guess yours preferences WS over everything else, yeah
2012-04-30 12111, 2012
kepstin-laptop
I suppose I could make it some sort of priority queue, so that interactive queries bump to the head of the line, that would reduce the latency to max 1s for user requests.
2012-04-30 12120, 2012
ianmcorvidae
I'm not sure that's bad; you still let the program do other stuff in-between, since it's using js timeouts, yeah?
2012-04-30 12143, 2012
kepstin-laptop
ianmcorvidae: yeah, it's all asynchronous.
2012-04-30 12108, 2012
kepstin-laptop is planning to include similar asynchronous ratelimiting code in riker, now that he has something that works.
the neat thing about the algorithm used is that the 11th request will always be at least 10s after the 1st request, and similar for the 2nd/12th, and so on.
2012-04-30 12116, 2012
kepstin-laptop
which keeps it under the ratelimit checks.
2012-04-30 12136, 2012
MiX-MaN joined the channel
2012-04-30 12109, 2012
kepstin-laptop got roped into doing some RL stuff this weekend, so he didn't have time to work on Riker like he hoped :(
2012-04-30 12124, 2012
kepstin-laptop
I got it compiling again and updated the readme tho.
2012-04-30 12145, 2012
MiX-MaN
is is possible to run a personal mb server on amazon and get away with just the free account?
2012-04-30 12126, 2012
MiX-MaN
or does anyone have a guide on installing everything on freebsd.. i've tried using the main github one.. but fbsd is just a bit to different
2012-04-30 12139, 2012
kepstin-laptop
MiX-MaN: I think so - I believe that nikki is doing that, actually.
2012-04-30 12115, 2012
MiX-MaN
i think im gonna give it a shot..
2012-04-30 12125, 2012
MiX-MaN
its been sitting there collecting dust for quite awhile
2012-04-30 12131, 2012
kepstin-laptop
There are sometimes issues during schema change upgrades where the mini instance doesn't have enough ram to properly handle it, though
2012-04-30 12149, 2012
kepstin-laptop
it usually works, but it can take a long time. That's pretty rare, tho :)
2012-04-30 12158, 2012
MiX-MaN
ah
2012-04-30 12107, 2012
MiX-MaN
heh.. anything could beat what im doing right now :P
2012-04-30 12122, 2012
MiX-MaN
fbsd->vbox->winserver->vbox->ngs image
2012-04-30 12102, 2012
MiX-MaN
18k ms page times
2012-04-30 12124, 2012
kepstin-laptop runs his mb server on an atom box in his basement that doubles as a fileserver.
2012-04-30 12105, 2012
MiX-MaN
hmmm
2012-04-30 12127, 2012
MiX-MaN
i could dig out my old laptop
2012-04-30 12135, 2012
kepstin-laptop
I'm using an 'lxc' paravirtualized linux container to keep it isolated from the main system
2012-04-30 12158, 2012
kepstin-laptop
so very little overhead, but it gets its own ip address and a separate linux filesystem.
2012-04-30 12124, 2012
kepstin-laptop
anyways, it's midnight and I'm working tomorrow :)
2012-04-30 12134, 2012
MiX-MaN
hehe.. thanks for the info :)
2012-04-30 12136, 2012
MiX-MaN
laterz
2012-04-30 12118, 2012
ianmcorvidae
ianmcorvidae has changed the topic to: Next meeting: Reviews, MBS-3646 back compat (ijabz, ocharles), i18n (mailing list, language liaisons) (ian)
2012-04-30 12140, 2012
MiX-MaN
nikki: if you run mb server on amazon ec2.. you have any tips on keeping everything running under the free teir?
2012-04-30 12117, 2012
ianmcorvidae
MiX-MaN: looks like nikki uses linode, from a traceroute; the tiers are different for that
2012-04-30 12143, 2012
ianmcorvidae
but I could be wrong :)
2012-04-30 12149, 2012
ianmcorvidae
MiX-MaN: I know plaintext was setting up ec2 things though, at some point, if you can catch him
2012-04-30 12157, 2012
MiX-MaN
bah :P
2012-04-30 12120, 2012
MiX-MaN
im trying ec2 right now.. its running slow as hell though
2012-04-30 12132, 2012
MiX-MaN
made it to the deployment step
2012-04-30 12103, 2012
navap
MiX-MaN: I've fun MB on amazon's free tier. I didn't import the entire database, but I used it for dev work so that was okay
2012-04-30 12130, 2012
MiX-MaN
i was just looking at that
2012-04-30 12144, 2012
MiX-MaN
they give 8gb ... dont think thats going to be enough space
2012-04-30 12146, 2012
navap
It ran very well before, but now that we've switched to the carton method it's basically unusable
2012-04-30 12108, 2012
ianmcorvidae
because of the carton install step, or?
2012-04-30 12109, 2012
navap
I had started with a clean db and just imported the test data
2012-04-30 12138, 2012
navap
ianmcorvidae: I don't know exactly, but I suspect something with the new system requires much more processing power
2012-04-30 12146, 2012
MiX-MaN
im thinking about running a nfs share over to my other server and putting the db on there
2012-04-30 12153, 2012
ianmcorvidae
well, I mean more *when* does it stop being usable
2012-04-30 12109, 2012
ianmcorvidae
i.e. is it a matter of when you're installing, or at runtime?
2012-04-30 12116, 2012
navap
Runtime
2012-04-30 12119, 2012
MiX-MaN
ah
2012-04-30 12124, 2012
ianmcorvidae
ah, probably plack then
2012-04-30 12141, 2012
MiX-MaN
i've never used amazons stuff.. first time..
2012-04-30 12102, 2012
navap
Restarting takes 1-5 minutes, and even just regular browsing is painfully slow at times
2012-04-30 12117, 2012
navap
The free tier is severaly CPU limited
2012-04-30 12113, 2012
MiX-MaN
any tips on getting it going in freebsd
2012-04-30 12143, 2012
ianmcorvidae
dunno that I know anyone to have tried
2012-04-30 12149, 2012
navap
I used an ec2 ubuntu image and just followed the INSTALL
2012-04-30 12156, 2012
ianmcorvidae
but theoretically the setup shouldn't be any different, I guess
2012-04-30 12113, 2012
MiX-MaN
navap: thats exactly what im doing now
2012-04-30 12151, 2012
MiX-MaN
i was getting all kinds of compilation errors on fbsd when trying to run the cpan stuff
2012-04-30 12144, 2012
MiX-MaN
woo deployment is done
2012-04-30 12100, 2012
ianmcorvidae reimporting new data; I always forget to change the tmp directory, at least it failed quickly this time
2012-04-30 12123, 2012
ianmcorvidae
I guess I haven't done an import since I doubled the amount of RAM in this machine
2012-04-30 12144, 2012
ruaok
ianmcorvidae: did you tweak the PG settings
2012-04-30 12145, 2012
ruaok
?
2012-04-30 12104, 2012
ruaok
if you give it more shared_segments, it could haul even more ass.
2012-04-30 12110, 2012
ianmcorvidae
ooh, might do that then
2012-04-30 12122, 2012
ianmcorvidae
this machine has 8GB rather than 4GB, it's a lot nicer :)
2012-04-30 12131, 2012
ruaok
for production systems 1/4 - 13/ of ram is suggested.
2012-04-30 12133, 2012
ianmcorvidae
it was actually just the tar --bzip2 -xvf that was going faster before
2012-04-30 12148, 2012
ianmcorvidae
shared_buffers, you mean, or?
2012-04-30 12106, 2012
ianmcorvidae
heh, currently set to 24MB
2012-04-30 12115, 2012
ianmcorvidae
should probably up that
2012-04-30 12117, 2012
ruaok
yeah.
2012-04-30 12134, 2012
ruaok
give it a gig.
2012-04-30 12150, 2012
ruaok
you'll need up up shmmax as well
2012-04-30 12139, 2012
ruaok
start postgres. it will fail.
2012-04-30 12150, 2012
ruaok
it will tell you how much shmmax it wants.
2012-04-30 12155, 2012
ianmcorvidae
hah
2012-04-30 12103, 2012
ruaok
add this to /etc/sysctl.conf
2012-04-30 12104, 2012
ruaok
kernel.shmmax=1102553088
2012-04-30 12111, 2012
ruaok
give it your right number of course.
2012-04-30 12121, 2012
ruaok
that persists for future reboots
2012-04-30 12138, 2012
ruaok
for immediatie action: sysctl -w kernel.shmmax=1102553088
2012-04-30 12144, 2012
ruaok
then restart PG
2012-04-30 12146, 2012
ianmcorvidae
yeah, indeed
2012-04-30 12156, 2012
ianmcorvidae
I did this all on rika, just never did on my own system, heh
2012-04-30 12105, 2012
ianmcorvidae
always more to learn about postgres :)
2012-04-30 12140, 2012
ianmcorvidae
maybe this summer I'll pick up a copy of that "PostgreSQL 9 High Performance" book, that could be a fun read :)
2012-04-30 12105, 2012
ianmcorvidae
super-optimize my single-user development install of postgres ;)
2012-04-30 12137, 2012
ianmcorvidae
heh, release table in 20 seconds, not bad
2012-04-30 12109, 2012
ianmcorvidae
(this is unoptimized, I'd already started the import without having done the SHMMAX/shared_buffers tweaks)
2012-04-30 12148, 2012
Mineo
alastairp: to rebuild, just use "make html". to get a docs/_build/html folder that actually tracks the gh-pages branch, use the command from the commit message of 4bba9c19a04d9c9ca68d754c4a801938b7ac1f27
2012-04-30 12107, 2012
ianmcorvidae
Mineo: I got a _sources directory when I did the build -- that just something I should ignore, or?
