sigh. trying to restart my modem did shit all. fakk
2023-06-09 16038, 2023
CatQuest
sorry for the flakey connection guys
2023-06-09 16004, 2023
CatQuest
what's this I read about staticbrainz urls changing? :/
2023-06-09 16007, 2023
CatQuest
reosarevok
2023-06-09 16009, 2023
aerozol
mayhem: pinch me if I'm dreaming, but we didn't do a blog post re. datasets do we? It's just drafted in a Google doc? Let me know if you want me to post it
aerozol: yep, we forgot it. yep, we should post it.
2023-06-09 16028, 2023
mayhem
and next week post the second installment.
2023-06-09 16051, 2023
mayhem
and then do a HN post -- I think the roll your own tagger bit might catch some attention
2023-06-09 16059, 2023
atj
i feel like this change away from staticbrainz is a lot of work in order to mitigate a small risk
2023-06-09 16040, 2023
mayhem
atj: I agree.
2023-06-09 16054, 2023
atj
also, not a good use of MeB $$$
2023-06-09 16058, 2023
atj
just buy a cert
2023-06-09 16011, 2023
atj
instead of devs spending hours making code changes
2023-06-09 16034, 2023
mayhem
and staticbrainz is just cool. period. :)
2023-06-09 16052, 2023
atj
if there is a concern about auto-renew of LE certificates we should resolve the root cause, seems like an infrastructure problem rather than something LE specific
2023-06-09 16045, 2023
reosarevok
It'd be a fairly trivial change code-wise I think, but if you don't feel it's worth it, I certainly don't mind either way :)
2023-06-09 16009, 2023
mayhem
engage the power of slackgin!
2023-06-09 16012, 2023
reosarevok
Just let us know about the decision
2023-06-09 16018, 2023
mayhem
slack-gin? I should try and make some.
2023-06-09 16009, 2023
reosarevok
Is that like normal gin, but with less of the "I'm drinking wood" taste? If so, then it can only be an improvement
2023-06-09 16021, 2023
CatQuest
slaking <-- that pokemon
2023-06-09 16025, 2023
reosarevok
(admittedly, teenage me only drank very cheap gin, which is probably the problem)
atj, mayhem: hardcoded certs have also a cost in term of maintenance, renewal, deployment. LE certs are more convenient from this point of view. The argument staticbrainz is cool ... well, yes, but that's not a website domain (it was meant at the start to store our static files, not to be accessed by humans).
2023-06-09 16020, 2023
atj
i have no opinion on the coolness of the domain ;)
2023-06-09 16024, 2023
zas
Fixing the root cause is easier to say than to do, because LE certs creates a dependency on external services and networks not under our control. We have a software reliability issue atm, but even if we strengthen things to maximum, you'll still be dependent on external stuff.
2023-06-09 16012, 2023
atj
we'll always have a dependency on external factors
2023-06-09 16013, 2023
zas
to me LE certs are a good trade-off for non-critical websites, easy to create, deploy, update
2023-06-09 16052, 2023
atj
IMO there is a far greater risk of a Hetzner failure than LE
2023-06-09 16008, 2023
atj
and LE failure needs to persists for 3+ weeks to impact us anyway
2023-06-09 16023, 2023
atj
(should be renewing at ~30 days before expiry)
2023-06-09 16036, 2023
atj
I mention Hetzner in the context of external dependencies (which we can't avoid)
2023-06-09 16034, 2023
zas
the problem with LE certs is related to software needed to manage those... experience shows those are failing at some point, go unmaintained, contain bugs, not updated to keep up with new protocols, etc... a hardcoded cert is much simpler, so more reliable.
2023-06-09 16006, 2023
zas
(but costs more)
2023-06-09 16059, 2023
zas
but I think mayhem just decided the right solution, keep the cool staticbrainz.org and pay for a cert, until we made LE stuff robust enough.
2023-06-09 16046, 2023
zas
we'll see if we can improve things next week with the work on openresty
2023-06-09 16011, 2023
atj
zas: honestly, if we can't make LE robust we should quit our jobs
2023-06-09 16041, 2023
atj
huge companies use LE on their critical apps
2023-06-09 16051, 2023
zas
and they have huge failures...
2023-06-09 16009, 2023
atj
*citation needed
2023-06-09 16031, 2023
zas
and we're not a huge company
2023-06-09 16036, 2023
DjSlash
re: related software, isn't that an issue with all software you use?
2023-06-09 16008, 2023
reosarevok
I see our base docker image is based on bionic, that's 5 years old, right?
2023-06-09 16017, 2023
zas
yes
2023-06-09 16018, 2023
reosarevok
Is there a plan to upgrade that in the near term?
2023-06-09 16034, 2023
reosarevok
(fixing wikidata-bot which expected *xenial* which is even worse, lol)
2023-06-09 16052, 2023
zas
that's the problem with docker
2023-06-09 16023, 2023
zas
base images are expected to be upgraded over time, the fact is we don't keep up fast enough. That's something we need to improve for sure.
2023-06-09 16028, 2023
yvanzo
bitmap: please merge #2945 if you approve it.
2023-06-09 16057, 2023
zas
reosarevok: if we upgrade base images we need to ensure the stuff using them still works, usually it requires few changes (because dependencies can break)
2023-06-09 16017, 2023
reosarevok
Sure, I get it's not just changing the name :) hence asking if it's in the plans, not if we should do it today :D
2023-06-09 16041, 2023
reosarevok
I know you just finished a lot of moving part changes
2023-06-09 16000, 2023
yvanzo
yes we should
2023-06-09 16003, 2023
zas
yes, definitively
2023-06-09 16031, 2023
yvanzo
musicbrainz-docker is using focal base image atm
2023-06-09 16034, 2023
yvanzo
you can use it to test even more recent base image for musicbrainz-web* containers at least.
2023-06-09 16030, 2023
zas
we have plenty of stuff using phusion base image
2023-06-09 16034, 2023
mayhem enjoys the atj zas banter
2023-06-09 16003, 2023
yvanzo
yes, this is just an example, other prod containers will require even more attention.
so adding a new one won't break anything until you change the version
2023-06-09 16040, 2023
zas
atj: it doesn't change the core problem, if you change underlying software (here a base image), you still need to ensure what you built over it works as expected, and atm we don't have much tools to do that. Just an example: if a command got its options or outputs changed, and you use it in a shell script, the shell script might break, and without proper testing that's hard to detect.
2023-06-09 16018, 2023
atj
zas: yeah I get that, what I meant was that just creating a newer base image doesn't actively break anything
2023-06-09 16037, 2023
atj
the images that use it as a base still need to opt in
2023-06-09 16042, 2023
petitminion joined the channel
2023-06-09 16044, 2023
zas
nope it doesn't, each image can easily switch to another base image
2023-06-09 16056, 2023
atj
I don't like how Docker seems to encourage ossification
2023-06-09 16059, 2023
zas
it does, docker is a tradeoff, it eases devel & deployment on the plus side
2023-06-09 16035, 2023
zas
about base images we could go through projects and list what they use as base images, and see which ones we can upgrade without too much risks
2023-06-09 16008, 2023
atj
i'm sure everyone is lining up to volunteer for that job ;)
2023-06-09 16029, 2023
yvanzo
This is just needed to keep stuff working.
2023-06-09 16034, 2023
mayhem
lucifer: which branch is running on metabrainz-prod right now?
2023-06-09 16014, 2023
Pratha-Fish
morena :D
2023-06-09 16028, 2023
reosarevok
Well, I can do the listing
2023-06-09 16029, 2023
zas
when we moved to docker we were quite new to it (docker itself was quite new), but overall it was a huge acceleration for our projects these last years, so that's rather a positive experience, but it has drawbacks we started to identify over the time. It also helped a lot in the transition from DWNI to Hetzner.
2023-06-09 16034, 2023
yvanzo
I’m not sure what stopped the focal update 2 years ago, but we can probably skip it and jump to jellyfish
2023-06-09 16036, 2023
reosarevok
No clue about the seeing which ones have few risks
2023-06-09 16038, 2023
reosarevok
atj was mentioning btw that apt-key is apparently going away soonish, but it seems it still works with jammy
2023-06-09 16051, 2023
atj
the custom compilation of python in the docker-python images is a bit gnarly
2023-06-09 16053, 2023
reosarevok
Most of our projects seem to still use apt-key, anyway