#metabrainz

I think that could improve matching figures a fair bit, at the detriment of precision

this is a similar issue to my JJ Johnson & Kai Winding example from before

especially if MB doesn't have all of the credits set up

I *think* my system is setup carefully enough for that to not end in disaster.

not sure what album monkey is listening to, but there definitely isn't an AC here: https://musicbrainz.org/release/0754aafd-ee25-4...

which is why it matched.

(with the detuning)

That listen comes from the album Bacchanal. Recording here: https://musicbrainz.org/recording/cdbb76d9-549b...

however, the *artist rels* of most of those recordings do include those other artists

Oh, and an interesting thing I just noticed :

"Gabor Szabo Jim Stewart Hal Gordon Jimmy Keltner Louis Kabok" with or without commas doesn't match.

yep, https://labs.api.listenbrainz.org/explain-mbid-...

I guess that's because ruaok deletes 'feat.' and anything following it?

but doesn't do that for ,

yes, but I propose to try exactly that.

all the cases I've tried seem to work ok.

see link above

two tracks remaining on that Sound of Contact album.

nice

spotify: "I am dimensionaut", MB: "I Am (Dimensionaut)"

that should end up matching, no?

yeah, https://labs.api.listenbrainz.org/explain-mbid-... shows a match

but not on LB

still not finished with Nov 17

great

I keep restarting it.

[listenbrainz-server] 14MonkeyDo merged pull request #1739 (03master…monkey-window-title-trackname): LB-747: Update browser window title with currently playing track's name https://github.com/metabrainz/listenbrainz-serv...

do we have gateways meeting now or in an hour?

in an hour

👍

monkey: https://listenbrainz.org/user/alastairp/?min_ts...

I have this page open and new items keep getting added to the TOP of the page.

Yeah, there's already a ticket for that.

that's LB-834

LB-834: New listens are added to top of wrong Recent Listens page https://tickets.metabrainz.org/browse/LB-834

k, thanks!

twice today I was going to tell monkey

and then I decided that he has too much on his plate already :)

LB-834

Ah, soz, didn't see the past above

ruaok: the solution is to tell _me_ to stop listening to music :)

au contraire, I just found out you ARE!

We're soon going to remove BrainzPlayer from the right hand side, maybe that'll be a good place for "sicne you opened this page…" sort of list

sure

I remember we had this long discussion about how we should deal with paging if you're using the app as new listens are coming in

and I don't think I agreed with all of your suggestions

Yeah, none of them seem great

We have no concept of "page 1" so it makes it pretty tricky

monkey: just listened to Bacchanal

it was good

Great album

no.

I *want* brainzplayer to (optionally) be on the right

CatQuest: I can isolate the CSS the we currently have. If you have a browser script that can load CSS for a specific page that'll work

lucifer: Thoughts on #1586 (CB review modal) ? Maybe something we want to hide while we finish the rest of the feature, or is it ready?

In any case, I think that's all the PRs on my side for today.

And ruaok were you able to have a look at #1718 yesterday?

lemme look now

monkey: i havent been able to look at the other PRs yet so not sure how we should proceed. i'll get back to you on this once i have looked at those.

OK, couldn't remember either where we are at

[musicbrainz-server] 14kawanoseiya opened pull request #2344 (03master…patch-1): Update INSTALL.md https://github.com/metabrainz/musicbrainz-serve...

Is there another "send random PRs and win prizes" competition? ^

meeting time! :D

hey

hi!

ok, should I start? alastairp ?

hi

so, I did a list about what's working for us regarding gateways, and what's not

first what's working:

autossl (auto-generation of letsencrypt certs)

zas: you are leading this meeting, yes?

well, I prefer to start, not really "leading" ;)

autoconfiguration of frontends & backends from consul (docker-server-configs + gitzconsul + serviceregistrator + consul + consul-template)

load balancing with weights

redundancy: being able to switch between gateways, even though it's manual, that's something we want to keep somehow for maintenance tasks

what doesn't work well:

ssl puts a serious pressure on cpu, limiting number of requests we can handle (aka the CAA redirect mess)

sorry, irccloud is sad. here now

too much "pre-processing" on gateways: some stuff we do there should be done on backends instead, but that's like this for multiple reasons

i missed some messages too. looking at chatlogs now

grrrr

Bad timing for a split

yep :(

lack of redundancy: no auto switch in case of failure

lack of horizontal scalability

only level 7 load balancing

so, I started to think about a redesign

I did a lot of research, about tools we could use, that are open source, and free (as in beer)

currently requests for all MeB services go to 1 gateway? and it's routed by nginx/openresty?

I never got a response from the haproxy people about non profit licensing.

yes

alastairp: in fact, everything is going through gateways, but few services

btw, I think it would be preferable to not have "holes" (aka services not going through gateways), it makes few things harder to control (like IP blocking etc)

so, to scale, we have to upgrade the hardware, that's not great

so the goal is to get traffic off of the gateways sooner? (not sure if we're talking about goals yet)

nope, the goal is to have better gateways, we still have to load balance over backends, do caching, do filtering , handle ssl, etc

but we need a solution that is more robust & scalable

any questions til now?

ok, I continue: hetzner failover IPs

those are great, basically we can switch one or more IPs between machines (kiki & herb in this case)

with the caveat that the automatic detecting that something needs failing over is VERY SLOW.

but this is a slow process, because we have to use hetzner API, which is slow to answer

how slow? 1m, 5m, 10m?

(still faster than updating dns tho, probably)

but if we detect the failure ourselves and tell hetzer to switch over it is fast, yes?

the switch itself takes 1 second or less, but the API takes 1min or more

is hetzner API the only way of doing the failover? (I ask because I have done the same with keepalived, and you just bring up the IP on the host that needs it)

it is never fast: detect the failure at T, connect to API at T+1 second, wait til T+60s, switch at T+61 seconds

alastairp: yes, it must be done via hetzner otherwise their routers can't route traffice.

right

oh, it is NEVER fast?

that's pretty crap.

yes :( it was a bit faster in the past, but now never under a minute

oy

devil's advocate: 60 seconds may be faster than us getting a telegram, working out what's wrong, sshing in to gateway and triggering swtichover

how do we do the failover currently?

alastairp: yes

lucifer: manually

right, i mean what's the process once we decide to do it? still need to go through the api?

why? because I never manage to have a stable setup with failures detection

we run a script, which asks API to switch IP

what is the usecase for switching over? We currently do it when zas upgrades gateways. We did it a few weeks ago when the server was overloaded. what other cases are there? server disappears unexpectedly?

unless you can convince them to reconfigure their switches to allow using something like ARP triggered from your machines to do the failover, there probably isn't a faster option.

kepstin: we can't convince them to do that, this is why we have to use the API

so the only difference in the automatic failover and the script we have now is the time to detect the failure after that both processes are same, right?

aka we can't keepalived those IPs as we would on our own network

lucifer: and as zas points out, getting a stable setup that only switches over on real errors

i see, makes sense

interesting, so it probably involves a routing reconfiguration, since they support this across the datacenter, not just for machines on the same switch. would explain why it's slow to set up :/

kepstin: yes

uh irccloud acting up again

> Switching a failover IP/subnet takes between 40 and 60 seconds.

yes, in practice, 60 seconds rather than 40s

but, it is important, the actual switch is fast: < 1 second, we lose connections though when it happens