sigh. trying to restart my modem did shit all. fakk
sorry for the flakey connection guys
what's this I read about staticbrainz urls changing? :/
reosarevok
aerozol
mayhem: pinch me if I'm dreaming, but we didn't do a blog post re. datasets do we? It's just drafted in a Google doc? Let me know if you want me to post it
aerozol: yep, we forgot it. yep, we should post it.
and next week post the second installment.
and then do a HN post -- I think the roll your own tagger bit might catch some attention
atj
i feel like this change away from staticbrainz is a lot of work in order to mitigate a small risk
mayhem
atj: I agree.
atj
also, not a good use of MeB $$$
just buy a cert
instead of devs spending hours making code changes
mayhem
and staticbrainz is just cool. period. :)
atj
if there is a concern about auto-renew of LE certificates we should resolve the root cause, seems like an infrastructure problem rather than something LE specific
reosarevok
It'd be a fairly trivial change code-wise I think, but if you don't feel it's worth it, I certainly don't mind either way :)
mayhem
engage the power of slackgin!
reosarevok
Just let us know about the decision
mayhem
slack-gin? I should try and make some.
reosarevok
Is that like normal gin, but with less of the "I'm drinking wood" taste? If so, then it can only be an improvement
CatQuest
slaking <-- that pokemon
reosarevok
(admittedly, teenage me only drank very cheap gin, which is probably the problem)
atj, mayhem: hardcoded certs have also a cost in term of maintenance, renewal, deployment. LE certs are more convenient from this point of view. The argument staticbrainz is cool ... well, yes, but that's not a website domain (it was meant at the start to store our static files, not to be accessed by humans).
atj
i have no opinion on the coolness of the domain ;)
zas
Fixing the root cause is easier to say than to do, because LE certs creates a dependency on external services and networks not under our control. We have a software reliability issue atm, but even if we strengthen things to maximum, you'll still be dependent on external stuff.
atj
we'll always have a dependency on external factors
zas
to me LE certs are a good trade-off for non-critical websites, easy to create, deploy, update
atj
IMO there is a far greater risk of a Hetzner failure than LE
and LE failure needs to persists for 3+ weeks to impact us anyway
(should be renewing at ~30 days before expiry)
I mention Hetzner in the context of external dependencies (which we can't avoid)
zas
the problem with LE certs is related to software needed to manage those... experience shows those are failing at some point, go unmaintained, contain bugs, not updated to keep up with new protocols, etc... a hardcoded cert is much simpler, so more reliable.
(but costs more)
but I think mayhem just decided the right solution, keep the cool staticbrainz.org and pay for a cert, until we made LE stuff robust enough.
we'll see if we can improve things next week with the work on openresty
atj
zas: honestly, if we can't make LE robust we should quit our jobs
huge companies use LE on their critical apps
zas
and they have huge failures...
atj
*citation needed
zas
and we're not a huge company
DjSlash
re: related software, isn't that an issue with all software you use?
reosarevok
I see our base docker image is based on bionic, that's 5 years old, right?
zas
yes
reosarevok
Is there a plan to upgrade that in the near term?
(fixing wikidata-bot which expected *xenial* which is even worse, lol)
zas
that's the problem with docker
base images are expected to be upgraded over time, the fact is we don't keep up fast enough. That's something we need to improve for sure.
yvanzo
bitmap: please merge #2945 if you approve it.
zas
reosarevok: if we upgrade base images we need to ensure the stuff using them still works, usually it requires few changes (because dependencies can break)
reosarevok
Sure, I get it's not just changing the name :) hence asking if it's in the plans, not if we should do it today :D
I know you just finished a lot of moving part changes
yvanzo
yes we should
zas
yes, definitively
yvanzo
musicbrainz-docker is using focal base image atm
you can use it to test even more recent base image for musicbrainz-web* containers at least.
zas
we have plenty of stuff using phusion base image
mayhem enjoys the atj zas banter
yvanzo
yes, this is just an example, other prod containers will require even more attention.
so adding a new one won't break anything until you change the version
zas
atj: it doesn't change the core problem, if you change underlying software (here a base image), you still need to ensure what you built over it works as expected, and atm we don't have much tools to do that. Just an example: if a command got its options or outputs changed, and you use it in a shell script, the shell script might break, and without proper testing that's hard to detect.
atj
zas: yeah I get that, what I meant was that just creating a newer base image doesn't actively break anything
the images that use it as a base still need to opt in
petitminion joined the channel
zas
nope it doesn't, each image can easily switch to another base image
atj
I don't like how Docker seems to encourage ossification
zas
it does, docker is a tradeoff, it eases devel & deployment on the plus side
about base images we could go through projects and list what they use as base images, and see which ones we can upgrade without too much risks
atj
i'm sure everyone is lining up to volunteer for that job ;)
yvanzo
This is just needed to keep stuff working.
mayhem
lucifer: which branch is running on metabrainz-prod right now?
Pratha-Fish
morena :D
reosarevok
Well, I can do the listing
zas
when we moved to docker we were quite new to it (docker itself was quite new), but overall it was a huge acceleration for our projects these last years, so that's rather a positive experience, but it has drawbacks we started to identify over the time. It also helped a lot in the transition from DWNI to Hetzner.
yvanzo
I’m not sure what stopped the focal update 2 years ago, but we can probably skip it and jump to jellyfish
reosarevok
No clue about the seeing which ones have few risks
atj was mentioning btw that apt-key is apparently going away soonish, but it seems it still works with jammy
atj
the custom compilation of python in the docker-python images is a bit gnarly
reosarevok
Most of our projects seem to still use apt-key, anyway