#metabrainz

nn freso

musicbrainz and caa updated.

chirlu: rika is up.

you still up?

Yes, I logged in already.

But Postgres doesn’t seem to be running.

great.

you have sudo, so you can start PG.

do what you can. I also have a 2TB portable drive that I am going to backup stuff to.

if you want to toss things for me to copy, mail me.

ruaok: sudo wants my password, which I don’t know (I set it to some random value after setting up pubkey auth years ago).

what's rika's IP?

72.29.167.152

can you sudo now?

Yes, thanks.

Pg running.

bitmap: zas: Gentlecat: we've been OKed to use as much bandwidth as we need.

so, no need for downtime.

I'll update the doc

sweet

done.

ok, we need to do no limiting on our end. just let'er rip. :)

Reportstool data saved.

If anyone else also wants it, you can fetch ~chirlu/reportstool.tar.xz (note that the configuration file contains the OAuth secret value that is probably still valid).

staticbrainz is part of the gateway configuration now

Did MB get selected for GCI?

Apparently yes.

So was Wikimedia.

ruaok: I think I’m finished with rika.

Lol.

Is there anything else I should copy off rika?

And thanks, for your help here...

Well, I’m not aware of anything important. Of course, the safest way is always to keep a full backup. :-)

Nothing of import in my stuff.

I'm copying some historical artifacts

chirlu: Yeah, I knew about Wikimedia :) and sweet!

The list is at https://opensource.googleblog.com/2016/11/annou...

ah, thanks

moin

moin

wow! MB gets selected again.

Freso: if there is anything that i can do then please inform me.

[metabrainz.org] gentlecat closed pull request #229: Docker (master...docker) https://git.io/vXECe

1300 req/s on ws for 200 concurrent connections, bottleneck is number of ws containers, not db (it depends on requests ofc)

and 3500 req/s on static files

^^ this is to make ruaok happy even before coffee ;)

zas: remind me how to check if multiple backends are being used by nginx

I just added another meb container, want to see if it shows up

Hey

no easy way yet, but log in kiki and do

grep -A20 -F '# metabrainz' /home/zas/openresty-gateways/files/nginx/conf.d/upstreams-auto-generated.conf

replace metabrainz by your service name

yep, great https://www.irccloud.com/pastebin/oy68koQ4/

important thing

we discovered an issue related to consul http checks

what's the issue?

under heavy load, if your service doesn't return 429 responses (too many requests), consul may consider the upstream as dead

and therefore remove it from this list, balancing the load to the remaining upstreams, eventually cascading to death

if under too many requests your service returns 5xx (or anything != 429), better use tcp health checks

we did that dor mb services (because they suck, using 5xx instead of 429)

"The status of the service depends on the HTTP response code: any 2xx code is considered passing, a 429 Too Many Requests is a warning, and anything else is a failure."

and if it is a failure... upstream is removed after few secs

i discovered it the hard way while doing load tests

almost all of the containers I've made are using TCP check because they are based on uWSGI

the only two exceptions are mbspotify nginx container (which is a bit broken now anyway) and sentry

bitmap: good morning

mornin'

armalcolite: I'm sure the BB team would appreciate another mentor on the team. :)

ruaok: ping, wake up, coffee, this is The Day

:o

Gentlecat: why https://github.com/metabrainz/docker-server-con... ?

if you add anything to kiki, it is expected to run on herb too. And backend servers should not run on kiki or herb at all

or do i miss something ?

ah, OK

I missed that

will move it somewhere else

gateways are totally redundant, we may switch from one to the other at any moment

zas: in /home/ftp-data/.ssh/authorized_keys on sakura we use some fancy rrsync commands, do you know if that's no longer necessary?

bitmap: make a copy for history (that's the ssh command trick right ?)

basically it runs a command based on key used

[22:47] <Leftmost> Tagging isn't a big application for me, but library management is, with tagging as a sidebar.

library managmeent. ie what I said earlier of the "digital shelf" :)

command="/home/ftp-data/bin/rrsync /media/ftp/root/pub/musicbrainz/data/fullexport/"

right

command="rm /media/ftp/root/pub/musicbrainz/data/fullexport/latest-is* ; /home/ftp-data/bin/rrsync /media/ftp/root/pub/musicbrainz/data/fullexport/"

I'm a bit confused what the rm is for

i guess it deletes ;)

but why :)

rrsync is the useful part

but we should have it somewhere