#metabrainz

sigh. trying to restart my modem did shit all. fakk

sorry for the flakey connection guys

what's this I read about staticbrainz urls changing? :/

reosarevok

mayhem: pinch me if I'm dreaming, but we didn't do a blog post re. datasets do we? It's just drafted in a Google doc? Let me know if you want me to post it

[design-system] 14Aerozol opened pull request #109 (03master…master): New Weblate icon https://github.com/metabrainz/design-system/pul...

[design-system] 14Aerozol closed pull request #109 (03master…master): New Weblate icon https://github.com/metabrainz/design-system/pul...

[design-system] 14Aerozol reopened pull request #109 (03master…master): New Weblate icon https://github.com/metabrainz/design-system/pul...

[design-system] 14Aerozol merged pull request #109 (03master…master): New Weblate icon https://github.com/metabrainz/design-system/pul...

Oops :D

uh why like 4

weird :D

aerozol: yep, we forgot it. yep, we should post it.

and next week post the second installment.

and then do a HN post -- I think the roll your own tagger bit might catch some attention

i feel like this change away from staticbrainz is a lot of work in order to mitigate a small risk

atj: I agree.

also, not a good use of MeB $$$

just buy a cert

instead of devs spending hours making code changes

and staticbrainz is just cool. period. :)

if there is a concern about auto-renew of LE certificates we should resolve the root cause, seems like an infrastructure problem rather than something LE specific

It'd be a fairly trivial change code-wise I think, but if you don't feel it's worth it, I certainly don't mind either way :)

engage the power of slackgin!

Just let us know about the decision

slack-gin? I should try and make some.

Is that like normal gin, but with less of the "I'm drinking wood" taste? If so, then it can only be an improvement

slaking <-- that pokemon

(admittedly, teenage me only drank very cheap gin, which is probably the problem)

i'd say it's all gin personally

it does taste woody, yea

!recall oh no.

most gin seems to go to great lengths to disguise the taste of gin

my gin oaxaca is trying to hide its hoja santa taste. rather an odd gin.

mayhem: to be clear, this never is actually never in the past X days. https://github.com/metabrainz/troi-recommendati...

Oh.

oh nvm

i forget we attach timestamps to recs as well

Uh oh?

false alarm. all good.

Yes, that was the key that made it all easy.

Thanks for adding those. :-)

atj, mayhem: hardcoded certs have also a cost in term of maintenance, renewal, deployment. LE certs are more convenient from this point of view. The argument staticbrainz is cool ... well, yes, but that's not a website domain (it was meant at the start to store our static files, not to be accessed by humans).

i have no opinion on the coolness of the domain ;)

Fixing the root cause is easier to say than to do, because LE certs creates a dependency on external services and networks not under our control. We have a software reliability issue atm, but even if we strengthen things to maximum, you'll still be dependent on external stuff.

we'll always have a dependency on external factors

to me LE certs are a good trade-off for non-critical websites, easy to create, deploy, update

IMO there is a far greater risk of a Hetzner failure than LE

and LE failure needs to persists for 3+ weeks to impact us anyway

(should be renewing at ~30 days before expiry)

I mention Hetzner in the context of external dependencies (which we can't avoid)

the problem with LE certs is related to software needed to manage those... experience shows those are failing at some point, go unmaintained, contain bugs, not updated to keep up with new protocols, etc... a hardcoded cert is much simpler, so more reliable.

(but costs more)

but I think mayhem just decided the right solution, keep the cool staticbrainz.org and pay for a cert, until we made LE stuff robust enough.

we'll see if we can improve things next week with the work on openresty

zas: honestly, if we can't make LE robust we should quit our jobs

huge companies use LE on their critical apps

and they have huge failures...

*citation needed

and we're not a huge company

re: related software, isn't that an issue with all software you use?

I see our base docker image is based on bionic, that's 5 years old, right?

yes

Is there a plan to upgrade that in the near term?

(fixing wikidata-bot which expected *xenial* which is even worse, lol)

that's the problem with docker

base images are expected to be upgraded over time, the fact is we don't keep up fast enough. That's something we need to improve for sure.

bitmap: please merge #2945 if you approve it.

reosarevok: if we upgrade base images we need to ensure the stuff using them still works, usually it requires few changes (because dependencies can break)

Sure, I get it's not just changing the name :) hence asking if it's in the plans, not if we should do it today :D

I know you just finished a lot of moving part changes

yes we should

yes, definitively

musicbrainz-docker is using focal base image atm

you can use it to test even more recent base image for musicbrainz-web* containers at least.

we have plenty of stuff using phusion base image

yes, this is just an example, other prod containers will require even more attention.

[listenbrainz-android] 14kingbass01 opened pull request #176 (03main…fix157): Fix theme and colors for About and NewsBrainz Activities in Compose https://github.com/metabrainz/listenbrainz-andr...

We need better tests too, because changing base image may create issues we're not able to identify through automated testing atm

aren't you using versioned base images though?

akshaaatt: please review https://github.com/metabrainz/listenbrainz-andr...

so adding a new one won't break anything until you change the version

atj: it doesn't change the core problem, if you change underlying software (here a base image), you still need to ensure what you built over it works as expected, and atm we don't have much tools to do that. Just an example: if a command got its options or outputs changed, and you use it in a shell script, the shell script might break, and without proper testing that's hard to detect.

zas: yeah I get that, what I meant was that just creating a newer base image doesn't actively break anything

the images that use it as a base still need to opt in

nope it doesn't, each image can easily switch to another base image

I don't like how Docker seems to encourage ossification

it does, docker is a tradeoff, it eases devel & deployment on the plus side

about base images we could go through projects and list what they use as base images, and see which ones we can upgrade without too much risks

i'm sure everyone is lining up to volunteer for that job ;)

This is just needed to keep stuff working.

lucifer: which branch is running on metabrainz-prod right now?

morena :D

Well, I can do the listing

when we moved to docker we were quite new to it (docker itself was quite new), but overall it was a huge acceleration for our projects these last years, so that's rather a positive experience, but it has drawbacks we started to identify over the time. It also helped a lot in the transition from DWNI to Hetzner.

I’m not sure what stopped the focal update 2 years ago, but we can probably skip it and jump to jellyfish

No clue about the seeing which ones have few risks

atj was mentioning btw that apt-key is apparently going away soonish, but it seems it still works with jammy

the custom compilation of python in the docker-python images is a bit gnarly

Most of our projects seem to still use apt-key, anyway

It's been mentioned for a long time.

it's easy to migrate away from apt-key to be fair

probably a 3 line patch

mayhem: should be master

That seems a lot more complicated, fwiw :D

you don't need to do the gpg dance, at least on 20.04

if the key is armored, use .asc if not use .gpg

apt works it out

just giving dates at least

It's been deprecated because of a potential security issue, not for simplification.

lucifer: I am trying to debug sending invoices.

when I add debug statements and deploy to prod for sorting out what is happening, I get this:

mayhem: oh!

:)

did you make a new deploy?

yes

I have to debug invoice sending in prod, unfortunately.

need to apply this script

ok.

isn't it possible to setup a couple of dummy supporters on test.meb and test with it?

I need to send stuff that is in the production DB.

applied the script. try now

mayhem: what's the new release's tag?

works now, thanks!

invoice-debug

cool

don't revert to 2023-05-05.1 as that won't work now.