#metabrainz

Leftmost: who isn't?

Freso likes to act like he doesn't.

Leftmost: people make it easy to stalk you. :)

Freso chirlu` at my company, websites in HTTPS are sluggish like hell… the proxy acts like a man in the middle and redirects 2 or 3 times and invalidates all cache… then a page in HTTPS is likely to completely fail because of the borked certificate also

remind me to never fly AF again.

oh wait, I already don't. :)

LordSputnik: Should I make get requests like this : get('/creator/{bbid}?user_id=3') ?

stanislas: yup

Freso: reosarevok: mind if I put myself as mentor on the CB reviews tasks to help clear the backlog?

Feel free if you want

reosarevok: I've got some time, my desktop is doing a chkdsk on a 1TB drive :P

ruaok: grrr ;)

sorry. had to. :)

it’s not AF the problem, the problem is that someone was convinced to buy something called Zscaler, which is an awful POS :/

LordSputnik: Should I make my own tests (i have done some already) or there is a way to test it some other way ?

stanislas: no need to make tests, but I'd be very happy if you did :)

\o LordSputnik

If I were writing it, I'd try to test it if I could easily

Leo_Verto: hey :)

LordSputnik: Have you noticed that tests are becoming a serious part of bookbrainz-ws :) ?

LordSputnik: I will add tests about display_alias to my actual tests after commiting this - this way it would make more sense.

stanislas: I thought to myself the other day - there's probably more lines of tests than there are of code :P

but who's testing the tests? :D haha

search is dead. on it.

LordSputnik: Yeah, I also thought of this. But then I've asked myself - what is wrong with me ? :)

(about testing the tests)

JESUS2099: What do I do about Facebook's react class naming since it is very dynamic?

zas: which machine were you thinking of using for website requests?

search fixed.

bitmap: pingu now has and extra 8GB ram. total 24GB.

its total number of clients it can handle should be increased accordingly.

hm, I thought zas had already put it in rotation. it's commented on ernie

that was from post RAID fix.

I believe earlier he did some updates, but was waiting for you to put it into rotation

LordSputnik: What do you think https://github.com/bookbrainz/bookbrainz-ws/com... ?

nm, it's uncommented on bert and appears to be handling requests

stanislas: hmm, why did you use -1 as a default value?

because I didn't know if I can use 0

And ... why not ?

so it's currently at weight=20 on bert, same as asterix and astro, and has been handling requests for about 1.5 hours

stanislas: Well, generally it's best to avoid "magic numbers" if the language has something to represent "nothing" - in this case, Python has None, so I'd prefer that

bitmap: yes

And then directly check if the user_id is None, and avoid making the query if it is :)

it is in the same class of astro/asterix now

i put it in rotation to ensure it is working as it should, and it looks it is

LordSputnik: ok, I will change it to None, and there is bug in get_entity_language_ids

i think we could slow machines for websites, either lolo or dagon

agreed

stanislas: another thing is I think get_display_alias should return None if it can't find one, then the rest of the code should deal with that specially, rather than constructing a placeholder untitled alias

insert "use" ^ ..

stanislas: another way you could optimise it is by changing your find_alias_if function to run the function in the query rather than in Python, after the results have been fetched

LordSputnik: But 'If there are no aliases, use "(untitled)" or "(unnamed)" ' :)

Hmm, you can still do that without constructing an Alias ;)

Just deal with it in the JSON function rather than the get_display_alias function

LordSputnik: Ok, so I will return None

how do we proceed ? i propose to change ernie/bert nginx config to route requests; matching ^/ws/[12]/ and using two upstream sets

so we keep exact same setup on all machines, but just exclude/include requests from one set or the other

stanislas: That way, if someone wants to use get_display_alias somewhere else, they can tell if they have a valid Alias or nothing - at the moment, there's no way to tell if the stored alias is actually "(untitled)"

zas: I was thinking the same thing

if you can do that, I can remove the musicbrainz-ws services

LordSputnik: So I should just make {'display_alias':None} in return JSON ?

* returned JSON

whatever happened to the split where some instances do ws traffic and some do web page traffic? do we still do that?

Freso: JESUS2099 ping regarding reactjs being ehhh

I'm not sure we ever did that, at least while I was here

what we have now is separate unix sockets on each instance, one for ws traffic and another for website traffic

stanislas: yeah I think that would be better actually

yes, that.

remind me what the goal for that split was. I thought it was to ensure that web traffic always got better loading speed.

LordSputnik: How do you imagine doing it outside python ? What do you mean by that ?

LordSputnik: I am not doing any SQL queries here.

(on aliases

stanislas: entity_data.aliases is technically doing a query

Ignore what I said though, it'd be adding a load of complexity for not much benerfit - premature optimisation ;)

DB sockets or overall sockets?

bitmap: was it ? i thought it was to use a server with less load to serve website only requests

we currently have no exhaustion of sockets

ruaok: whatever the connection limit for for unix sockets is

odd. something there doesn't make sense to me.

but we are not favoring web traffic, so we should start doing that.

i propose to test it: https://github.com/metabrainz/nginx/blob/master...

have you two agreed on a plan yet? if so, I'd love to hear it.

zas: that's the current goal

i propose to create a new upstream named mbwebsite or smt

and use ngs upstream only for locations matching ^/ws/[12]/

which machines belong to which of the two pools?

website=dagon with lolo as spare

ngs=asterix astro pingu lolo (when not used as website)

though the nginx config may need a bit of cleanup

https://github.com/metabrainz/nginx/blob/master... ??

smells like JSB

JSB?

;)

johann sebastian bach

and mozard.

*t

Yes i got it ;)

Nice to have him in the team :))

basically since the load comes mostly from ws requests, website requests (including static) will be served by a much-less loaded machine

sorry, I was slightly misremembering things: https://github.com/metabrainz/chef-cookbooks/pu...

the goal was to prevent worker process exhaustion, not socket connections

ok, that makes more sense

so there are always a number of worker process that can't be hogged by the ws

but those workers are still on the same machine than ws workers

so we could set 100% workers on machines dedicated to a role

if we split on load balancer instead

right, once the machines are partitioned we can revert that PR

it might make sense on pino, still, but beta doesn't get a whole lot of traffic anyway

i would not revert it, just change the ratio where it applies

but instead of configuring from cookbooks we can use the envdir trick

hm? if the traffic is partitioned by machine, then only one of the services will be used

wait, what do you call services in this case ?

starman, I guess. we run two different starman services (musicbrainz-server, musicbrainz-ws) which each have their own worker procs

then the traffic is partitioned in nginx on the app server, to separate sockets: https://github.com/metabrainz/chef-cookbooks/bl...

Ok, this is what i understood. So on each server atm, we have N workers for musicbrainz-server and M workers for musicbrainz-ws

And this is defined by cookbook attributes

If we have a machine doing only ws, we only need musicbrainz-ws ?

N=0 M=max

yeah, the musicbrainz-server one wouldn't get used, based on the current config

LordSputnik: Sumbitted the task for review, I will add tests for it in my the master.

to me it makes sense to have only one service, if the machines are only serving one type of traffic anyway

stanislas: OK, I'll review it now

I think we could change daemontools run scripts a bit, so nprocs value is read from env dir instead (and attributes used to set this default value)

LordSputnik: One more thing

LordSputnik: How can I abort if user_id is not valid ?

then run script could check if the value is 0, and just sleep, rechecking for changed value on regular basis.

just abort(404) ?

stanislas: should just be able to call abort(<error_code>)

It would be more dynamic, and would not require to re-run cookbooks for temporary changes

ok, and then we could keep the current split on pino. that makes sense

stanislas: Hmm, although I think it would be best to just ignore the user_id in that case

stanislas: don't forget to change the default user_id from -1 to None

Ok that's late for me, i can't think much more, but i'll have a look at changes needed tomorrow morning.

dwni said they are available for the baron downgrade soon.

no early bed for me

If a service is totally disabled, no request should arrive to the server in question

i mean no request it isn't able to serve

LordSputnik: Should be ok now

ruaok: i'm not going to bed yet either, but i will not modify current nginx confs, it is more likely to break ;))

wise

and to be frank there's a Bowie concert on Arte starting now, i would like to see it (it was the last from Ziggy period, in 1973)

even wiser.

zag: enjoy it, i have that dvd, it’s great. :)

opatel99: sorry i don’t know neither facebook nor react