#metabrainz

/

      • CatQuest joined the channel
      • CatQuest has quit
      • CatQuest joined the channel
      • CatQuest has quit
      • CatQuest joined the channel
      • CatQuest has quit
      • CatQuest joined the channel
      • kingbass[m] joined the channel
      • CatQuest joined the channel
      • CatQuest has quit
      • CatQuest joined the channel
      • CatQuest
        sigh. trying to restart my modem did shit all. fakk
      • sorry for the flakey connection guys
      • what's this I read about staticbrainz urls changing? :/
      • reosarevok
      • aerozol
        mayhem: pinch me if I'm dreaming, but we didn't do a blog post re. datasets do we? It's just drafted in a Google doc? Let me know if you want me to post it
      • d4rk has quit
      • d4rk joined the channel
      • BrainzGit
        [design-system] 14Aerozol opened pull request #109 (03master…master): New Weblate icon https://github.com/metabrainz/design-system/pul...
      • [design-system] 14Aerozol closed pull request #109 (03master…master): New Weblate icon https://github.com/metabrainz/design-system/pul...
      • [design-system] 14Aerozol reopened pull request #109 (03master…master): New Weblate icon https://github.com/metabrainz/design-system/pul...
      • [design-system] 14Aerozol merged pull request #109 (03master…master): New Weblate icon https://github.com/metabrainz/design-system/pul...
      • aerozol
        Oops :D
      • q3lont joined the channel
      • CatQuest
        uh why like 4
      • weird :D
      • mayhem
        aerozol: yep, we forgot it. yep, we should post it.
      • and next week post the second installment.
      • and then do a HN post -- I think the roll your own tagger bit might catch some attention
      • atj
        i feel like this change away from staticbrainz is a lot of work in order to mitigate a small risk
      • mayhem
        atj: I agree.
      • atj
        also, not a good use of MeB $$$
      • just buy a cert
      • instead of devs spending hours making code changes
      • mayhem
        and staticbrainz is just cool. period. :)
      • atj
        if there is a concern about auto-renew of LE certificates we should resolve the root cause, seems like an infrastructure problem rather than something LE specific
      • reosarevok
        It'd be a fairly trivial change code-wise I think, but if you don't feel it's worth it, I certainly don't mind either way :)
      • mayhem
        engage the power of slackgin!
      • reosarevok
        Just let us know about the decision
      • mayhem
        slack-gin? I should try and make some.
      • reosarevok
        Is that like normal gin, but with less of the "I'm drinking wood" taste? If so, then it can only be an improvement
      • CatQuest
        slaking <-- that pokemon
      • reosarevok
        (admittedly, teenage me only drank very cheap gin, which is probably the problem)
      • atj
        i'd say it's all gin personally
      • CatQuest
        it does taste woody, yea
      • reosarevok
        !recall oh no.
      • BrainzBot
      • atj
        most gin seems to go to great lengths to disguise the taste of gin
      • mayhem
        my gin oaxaca is trying to hide its hoja santa taste. rather an odd gin.
      • lucifer
        mayhem: to be clear, this never is actually never in the past X days. https://github.com/metabrainz/troi-recommendati...
      • mayhem
        Oh.
      • lucifer
        oh nvm
      • i forget we attach timestamps to recs as well
      • mayhem
        Uh oh?
      • lucifer
        false alarm. all good.
      • mayhem
        Yes, that was the key that made it all easy.
      • Thanks for adding those. :-)
      • petitminion joined the channel
      • zas
        atj, mayhem: hardcoded certs have also a cost in term of maintenance, renewal, deployment. LE certs are more convenient from this point of view. The argument staticbrainz is cool ... well, yes, but that's not a website domain (it was meant at the start to store our static files, not to be accessed by humans).
      • atj
        i have no opinion on the coolness of the domain ;)
      • zas
        Fixing the root cause is easier to say than to do, because LE certs creates a dependency on external services and networks not under our control. We have a software reliability issue atm, but even if we strengthen things to maximum, you'll still be dependent on external stuff.
      • atj
        we'll always have a dependency on external factors
      • zas
        to me LE certs are a good trade-off for non-critical websites, easy to create, deploy, update
      • atj
        IMO there is a far greater risk of a Hetzner failure than LE
      • and LE failure needs to persists for 3+ weeks to impact us anyway
      • (should be renewing at ~30 days before expiry)
      • I mention Hetzner in the context of external dependencies (which we can't avoid)
      • zas
        the problem with LE certs is related to software needed to manage those... experience shows those are failing at some point, go unmaintained, contain bugs, not updated to keep up with new protocols, etc... a hardcoded cert is much simpler, so more reliable.
      • (but costs more)
      • but I think mayhem just decided the right solution, keep the cool staticbrainz.org and pay for a cert, until we made LE stuff robust enough.
      • we'll see if we can improve things next week with the work on openresty
      • atj
        zas: honestly, if we can't make LE robust we should quit our jobs
      • huge companies use LE on their critical apps
      • zas
        and they have huge failures...
      • atj
        *citation needed
      • zas
        and we're not a huge company
      • DjSlash
        re: related software, isn't that an issue with all software you use?
      • reosarevok
        I see our base docker image is based on bionic, that's 5 years old, right?
      • zas
        yes
      • reosarevok
        Is there a plan to upgrade that in the near term?
      • (fixing wikidata-bot which expected *xenial* which is even worse, lol)
      • zas
        that's the problem with docker
      • base images are expected to be upgraded over time, the fact is we don't keep up fast enough. That's something we need to improve for sure.
      • yvanzo
        bitmap: please merge #2945 if you approve it.
      • zas
        reosarevok: if we upgrade base images we need to ensure the stuff using them still works, usually it requires few changes (because dependencies can break)
      • reosarevok
        Sure, I get it's not just changing the name :) hence asking if it's in the plans, not if we should do it today :D
      • I know you just finished a lot of moving part changes
      • yvanzo
        yes we should
      • zas
        yes, definitively
      • yvanzo
        musicbrainz-docker is using focal base image atm
      • you can use it to test even more recent base image for musicbrainz-web* containers at least.
      • zas
        we have plenty of stuff using phusion base image
      • mayhem enjoys the atj zas banter
      • yvanzo
        yes, this is just an example, other prod containers will require even more attention.
      • BrainzGit
        [listenbrainz-android] 14kingbass01 opened pull request #176 (03main…fix157): Fix theme and colors for About and NewsBrainz Activities in Compose https://github.com/metabrainz/listenbrainz-andr...
      • petitminion has quit
      • zas
        We need better tests too, because changing base image may create issues we're not able to identify through automated testing atm
      • atj
        aren't you using versioned base images though?
      • kingbass[m]
      • atj
        so adding a new one won't break anything until you change the version
      • zas
        atj: it doesn't change the core problem, if you change underlying software (here a base image), you still need to ensure what you built over it works as expected, and atm we don't have much tools to do that. Just an example: if a command got its options or outputs changed, and you use it in a shell script, the shell script might break, and without proper testing that's hard to detect.
      • atj
        zas: yeah I get that, what I meant was that just creating a newer base image doesn't actively break anything
      • the images that use it as a base still need to opt in
      • petitminion joined the channel
      • zas
        nope it doesn't, each image can easily switch to another base image
      • atj
        I don't like how Docker seems to encourage ossification
      • zas
        it does, docker is a tradeoff, it eases devel & deployment on the plus side
      • about base images we could go through projects and list what they use as base images, and see which ones we can upgrade without too much risks
      • atj
        i'm sure everyone is lining up to volunteer for that job ;)
      • yvanzo
        This is just needed to keep stuff working.
      • mayhem
        lucifer: which branch is running on metabrainz-prod right now?
      • Pratha-Fish
        morena :D
      • reosarevok
        Well, I can do the listing
      • zas
        when we moved to docker we were quite new to it (docker itself was quite new), but overall it was a huge acceleration for our projects these last years, so that's rather a positive experience, but it has drawbacks we started to identify over the time. It also helped a lot in the transition from DWNI to Hetzner.
      • yvanzo
        I’m not sure what stopped the focal update 2 years ago, but we can probably skip it and jump to jellyfish
      • reosarevok
        No clue about the seeing which ones have few risks
      • atj was mentioning btw that apt-key is apparently going away soonish, but it seems it still works with jammy
      • atj
        the custom compilation of python in the docker-python images is a bit gnarly
      • reosarevok
        Most of our projects seem to still use apt-key, anyway
      • yvanzo
        It's been mentioned for a long time.
      • atj
        it's easy to migrate away from apt-key to be fair
      • probably a 3 line patch
      • lucifer
        mayhem: should be master
      • yvanzo
      • reosarevok
        That seems a lot more complicated, fwiw :D
      • atj
        you don't need to do the gpg dance, at least on 20.04
      • if the key is armored, use .asc if not use .gpg
      • apt works it out
      • yvanzo
        just giving dates at least
      • It's been deprecated because of a potential security issue, not for simplification.
      • mayhem
        lucifer: I am trying to debug sending invoices.
      • when I add debug statements and deploy to prod for sorting out what is happening, I get this:
      • lucifer
        mayhem: oh!
      • mayhem
        :)
      • lucifer
        did you make a new deploy?
      • mayhem
        yes
      • I have to debug invoice sending in prod, unfortunately.
      • lucifer
      • need to apply this script
      • mayhem
        ok.
      • mayhem is confused
      • lucifer
        isn't it possible to setup a couple of dummy supporters on test.meb and test with it?
      • mayhem
        I need to send stuff that is in the production DB.
      • lucifer
        applied the script. try now
      • mayhem: what's the new release's tag?
      • mayhem
        works now, thanks!
      • invoice-debug
      • lucifer
        cool
      • don't revert to 2023-05-05.1 as that won't work now.