#metabrainz

/

      • atj
        don't say that!
      • 2022-02-28 05956, 2022

      • alastairp
        monkey: sounds good, do you need me to help with filling in some of those sections?
      • 2022-02-28 05907, 2022

      • lucifer
        mayhem: to be on the safer side, we should probably also list expected outcomes separately?
      • 2022-02-28 05947, 2022

      • zas
        atj: please commit to ansible repo along, so I can follow changes
      • 2022-02-28 05925, 2022

      • lucifer
      • 2022-02-28 05947, 2022

      • atj
        yes, sorry I'm a bit behind
      • 2022-02-28 05921, 2022

      • atj
        I had to make some shorewall role changes to match the manual changes I made on Friday for fail2ban
      • 2022-02-28 05926, 2022

      • monkey
        alastairp: Probably. Let me write down a blurb and I'll send it your way for corrections/additions
      • 2022-02-28 05927, 2022

      • lucifer
        ah ok.
      • 2022-02-28 05949, 2022

      • BrainzGit
        [listenbrainz-server] 14amCap1712 merged pull request #1880 (03master…now-playing): LB-1066: Clarify now playing listens are not persisted in DB https://github.com/metabrainz/listenbrainz-server…
      • 2022-02-28 05905, 2022

      • BrainzGit
        [listenbrainz-server] 14amCap1712 opened pull request #1887 (03master…cont-agg-drop): Remove all traces of listen count cont. agg https://github.com/metabrainz/listenbrainz-server…
      • 2022-02-28 05931, 2022

      • BrainzGit
        [listenbrainz-server] 14amCap1712 merged pull request #1882 (03master…msb-simplify): Cleanup mbids lookup in MsB code https://github.com/metabrainz/listenbrainz-server…
      • 2022-02-28 05928, 2022

      • atj
        zas: all updates committed and pushed
      • 2022-02-28 05937, 2022

      • atj
        do you want to reboot?
      • 2022-02-28 05935, 2022

      • atj
        afk for a minute, be right back
      • 2022-02-28 05906, 2022

      • zas
        atj: yes, I'll reboot
      • 2022-02-28 05938, 2022

      • BrainzGit
        [listenbrainz-server] release 03untagged-3a835ebc9e7492ffcd8d has been published by 14github-actions[bot]: https://github.com/metabrainz/listenbrainz-server…
      • 2022-02-28 05947, 2022

      • lucifer
        uhhh
      • 2022-02-28 05941, 2022

      • lucifer
        reverted.
      • 2022-02-28 05949, 2022

      • zas
        atj: aphex's back
      • 2022-02-28 05919, 2022

      • lucifer
        mayhem: alastairp: listenbrainz.org/messybrainz was added 2 releases ago. so very unlikely we'll need to revert that back. after today's release, lets take down msb.org container and redirect it to lb.org/msb . thoughts?
      • 2022-02-28 05938, 2022

      • mayhem
        sure
      • 2022-02-28 05952, 2022

      • alastairp
        what do you mean unlikely that we need to revert it?
      • 2022-02-28 05919, 2022

      • lucifer
        in the past, we sometime have had to roll back releases because something broke.
      • 2022-02-28 05900, 2022

      • alastairp
        ah, got it
      • 2022-02-28 05907, 2022

      • alastairp
        sounds good
      • 2022-02-28 05911, 2022

      • lucifer
        this is a few releases ago so we'd likely not have to roll back so far that the page gets removed.
      • 2022-02-28 05922, 2022

      • lucifer
        great!
      • 2022-02-28 05938, 2022

      • atj
        zas: great, looks good
      • 2022-02-28 05950, 2022

      • zas
        atj: all containers are up, and work, I checked open ports from outside, looks good too
      • 2022-02-28 05939, 2022

      • yvanzo
        bitmap, reosarevok: I added a difficulty field of GSoC ideas.
      • 2022-02-28 05952, 2022

      • atj
        zas: what would you like to do next?
      • 2022-02-28 05906, 2022

      • zas
        atj: can we improve a bit users part? https://github.com/atj/metabrainz-ansible/commit/…
      • 2022-02-28 05922, 2022

      • atj
        as in, add all users?
      • 2022-02-28 05931, 2022

      • zas
        in our cases, most users share same rights
      • 2022-02-28 05943, 2022

      • atj
        right I see
      • 2022-02-28 05950, 2022

      • zas
        Is is possible to deduce users from existing keys?
      • 2022-02-28 05906, 2022

      • BrainzGit
        [listenbrainz-server] 14amCap1712 merged pull request #1886 (03master…cleanup-delete): LB-1055: Cleanup user delete code https://github.com/metabrainz/listenbrainz-server…
      • 2022-02-28 05942, 2022

      • zas
        well, perhaps something we can do later
      • 2022-02-28 05953, 2022

      • atj
        so you'd like to be able to just add a key and the user is automatically created
      • 2022-02-28 05902, 2022

      • zas
        ideally yes
      • 2022-02-28 05912, 2022

      • tandy1000
        <akshaaatt> "Hi monkey! For when you're awake..." <- yayy, things are moving along :))
      • 2022-02-28 05942, 2022

      • tandy1000
        did we decide on keeping youtube in or out?
      • 2022-02-28 05956, 2022

      • zas
        also, what happens when we remove a user from this config?
      • 2022-02-28 05929, 2022

      • atj
        nothing, we need to specify "state: absent"
      • 2022-02-28 05934, 2022

      • atj
        to remove the user
      • 2022-02-28 05909, 2022

      • monkey
        alastairp: Feel free to change anything: https://wiki.musicbrainz.org/Development/Summer_o…
      • 2022-02-28 05951, 2022

      • atj
      • 2022-02-28 05951, 2022

      • zas
        Ok, good
      • 2022-02-28 05906, 2022

      • atj
        I feel that we should try to be implicit rather than explicit, whilst trying to remove as much duplication as possible
      • 2022-02-28 05916, 2022

      • atj
        sorry "explicit rather than implicit"
      • 2022-02-28 05943, 2022

      • atj
        so that someone who looks at the variables can understand what it means
      • 2022-02-28 05953, 2022

      • atj
        without too much hidden magic
      • 2022-02-28 05955, 2022

      • zas
        I agree, we don't want to remove users by error
      • 2022-02-28 05922, 2022

      • atj
        so, for instance we could add a global variable like "admin_groups"
      • 2022-02-28 05940, 2022

      • zas
        Yes
      • 2022-02-28 05951, 2022

      • atj
        and set that to ['sudo', 'docker'], then use "{{ admin_groups}}" for the groups var
      • 2022-02-28 05923, 2022

      • atj
        I know everyone is an admin at the moment effectively, but that might change in future
      • 2022-02-28 05934, 2022

      • zas
        yup
      • 2022-02-28 05931, 2022

      • atj
        what else needs installing on this base image? NRPE, prometheus, ...
      • 2022-02-28 05927, 2022

      • atj
        crowdsec looks quite nice, and I think I can write a role to install and manage that reasonably easily
      • 2022-02-28 05938, 2022

      • zas
        I think we should add monitoring, telegraf to start with
      • 2022-02-28 05946, 2022

      • atj
        yep
      • 2022-02-28 05903, 2022

      • atj
        is telegraf installed from third party repos?
      • 2022-02-28 05932, 2022

      • zas
        for now, that's configured using https://github.com/metabrainz/telegraf-fabric
      • 2022-02-28 05910, 2022

      • zas
        but there's some complexity
      • 2022-02-28 05945, 2022

      • mayhem
        zas: atj : hetzner says: "IPv6-Only is now default for orders via the Robot Web Service"
      • 2022-02-28 05958, 2022

      • mayhem
        what is our IPv6 readiness like?
      • 2022-02-28 05910, 2022

      • zas
        ouch, near zero
      • 2022-02-28 05919, 2022

      • atj
        I was going to mention that
      • 2022-02-28 05925, 2022

      • mayhem
        I suppose we dont need IPv4 addresses for all of our machines.
      • 2022-02-28 05931, 2022

      • mayhem
        really only the gateways need them.
      • 2022-02-28 05937, 2022

      • zas
        yes
      • 2022-02-28 05906, 2022

      • atj
        yes, but I would bet on it causing issues for apps somewhere due to assumptions about IPv4 addresses
      • 2022-02-28 05911, 2022

      • mayhem
        and IPv4 addresses now become an add-on.
      • 2022-02-28 05925, 2022

      • atj
        IPv4 addresses are worth upwards of $45 each now
      • 2022-02-28 05942, 2022

      • mayhem
        wow.
      • 2022-02-28 05910, 2022

      • atj
        it's become a major obstacle for new services
      • 2022-02-28 05947, 2022

      • zas
      • 2022-02-28 05900, 2022

      • zas
      • 2022-02-28 05925, 2022

      • atj
      • 2022-02-28 05947, 2022

      • CatQuest
        monkey: editing a tiny bit
      • 2022-02-28 05952, 2022

      • zas
      • 2022-02-28 05924, 2022

      • zas
        reason: we whitelist containers to monitor
      • 2022-02-28 05919, 2022

      • zas
        I think we should start from scratch on this, using ansible features instead
      • 2022-02-28 05927, 2022

      • zas
      • 2022-02-28 05959, 2022

      • CatQuest
        monkey: where is the "create relationship editing interface" one? :|
      • 2022-02-28 05912, 2022

      • CatQuest
        also make that one #! :D
      • 2022-02-28 05914, 2022

      • CatQuest
        eh
      • 2022-02-28 05918, 2022

      • CatQuest
        also make that one #1 :D
      • 2022-02-28 05911, 2022

      • atj
        zas: I'll have a look and how much is required
      • 2022-02-28 05912, 2022

      • monkey
        It's in there CatQuest, a sub-part of the admin syste
      • 2022-02-28 05913, 2022

      • monkey
        m
      • 2022-02-28 05946, 2022

      • BrainzGit
        [listenbrainz-server] 14amCap1712 merged pull request #1887 (03master…cont-agg-drop): Remove all traces of listen count cont. agg https://github.com/metabrainz/listenbrainz-server…
      • 2022-02-28 05915, 2022

      • lucifer
      • 2022-02-28 05959, 2022

      • monkey
        I'm adding a line about the rel editor page specificly CatQuest
      • 2022-02-28 05955, 2022

      • CatQuest
        one line? come on man, you gotta sell it!
      • 2022-02-28 05911, 2022

      • mayhem
        MBS-12225
      • 2022-02-28 05912, 2022

      • BrainzBot
        MBS-12225: Decide on inclusive Terminology for local MusicBrainz service instances https://tickets.metabrainz.org/browse/MBS-12225
      • 2022-02-28 05921, 2022

      • alastairp
        lucifer: let's go ahead with it
      • 2022-02-28 05936, 2022

      • lucifer
        👍, i'll resolve the merge conflicts
      • 2022-02-28 05917, 2022

      • atj
        zas: so if I understand correctly, you maintain a list of docker container names, and the entire list is then added to the telegraf configuration at "container_name_include = [__CONTAINERS__]"
      • 2022-02-28 05926, 2022

      • zas
        yes
      • 2022-02-28 05932, 2022

      • atj
        alright, fairly straightforward
      • 2022-02-28 05904, 2022

      • zas
        apart that each host can have a different config, depending on what's run on it, or hardware
      • 2022-02-28 05911, 2022

      • atj
        yeah, I was just looking at that. Should be possible to do all that via ansible group vars
      • 2022-02-28 05927, 2022

      • zas
        I guess so
      • 2022-02-28 05942, 2022

      • zas
        for aphex, we can use default-telegraf-sensors.conf
      • 2022-02-28 05948, 2022

      • atj
        just looking for some existing roles that we can use to manage telegraf. not finding much unfortunately
      • 2022-02-28 05945, 2022

      • zas
        atj: also we miss nagios nrpe in current playbook
      • 2022-02-28 05926, 2022

      • zas
      • 2022-02-28 05939, 2022

      • atj
        zas: let me get that installed then, as it's less work
      • 2022-02-28 05952, 2022

      • zas
        yup
      • 2022-02-28 05922, 2022

      • BrainzGit
        [listenbrainz-server] 14amCap1712 merged pull request #1781 (03master…ports-again): Update LB public ports. again. https://github.com/metabrainz/listenbrainz-server…
      • 2022-02-28 05950, 2022

      • elomatreb[m]
        lucifer: thanks for the info!
      • 2022-02-28 05950, 2022

      • atj
        zas: sorry, just having to do some minor updates to the role for Ubuntu 20.04
      • 2022-02-28 05934, 2022

      • zas
        Np
      • 2022-02-28 05939, 2022

      • atj
        zas: it's installed and running, I'm just addubg some configuration for Shorewall
      • 2022-02-28 05949, 2022

      • atj
        s/addubg/adding
      • 2022-02-28 05911, 2022

      • atj
        zas: you should be able to run NRPE checks now after this commit https://github.com/atj/metabrainz-ansible/commit/…
      • 2022-02-28 05959, 2022

      • atj
        I'll add a monitoring_plugins role shortly to install your custom plugins
      • 2022-02-28 05920, 2022

      • atj
        I'm going to grab some lunch now, I have a short call at 14:00 UTC
      • 2022-02-28 05926, 2022

      • lucifer
        alastairp, mayhem, monkey, akshaaatt: any PRs to merge, intending to do a release.
      • 2022-02-28 05933, 2022

      • alastairp
        not for LB, thanks
      • 2022-02-28 05933, 2022

      • mayhem
        None from me, none from monkey.
      • 2022-02-28 05945, 2022

      • lucifer
        monkey, alastairp, mayhem, akshaaatt, riksucks: i merged the ports PR. so you'll probably need to update the ports in your config again. also on MB oauth app.
      • 2022-02-28 05900, 2022

      • lucifer
        👍
      • 2022-02-28 05916, 2022

      • akshaaatt
        monkey: https://github.com/metabrainz/listenbrainz-server… looks good to me to merge if you've reviewed it
      • 2022-02-28 05945, 2022

      • alastairp
        CatQuest: thanks for reporting CB-428
      • 2022-02-28 05946, 2022

      • BrainzBot
        CB-428: Remove Gravatar from CritiqueBrainz https://tickets.metabrainz.org/browse/CB-428
      • 2022-02-28 05956, 2022

      • alastairp
        I noticed that when I was in the code last week and already planned to do so
      • 2022-02-28 05948, 2022

      • alastairp
        Ansh: if you're interested in continuing on some CB tasks, this should be straight forward if you want to do ti
      • 2022-02-28 05942, 2022

      • Ansh
        Sure! I'll do it
      • 2022-02-28 05941, 2022

      • alastairp
        (and if you want to something else and you're not sure about what then feel free to ask me or lucifer to recommend something!)
      • 2022-02-28 05930, 2022

      • Ansh
        Thanks. I was planing to look into LB as well.
      • 2022-02-28 05944, 2022

      • lucifer
        mayhem: i am looking at the code around inserted_timestamp and alastairp mentioned this PR in the ticket. i don't think we ever dumped these timestamps. the code only seems to include those in api.
      • 2022-02-28 05948, 2022

      • lucifer
      • 2022-02-28 05955, 2022

      • lucifer
      • 2022-02-28 05917, 2022

      • lucifer
      • 2022-02-28 05927, 2022

      • lucifer
        to_api has inserted_at but not to_json
      • 2022-02-28 05957, 2022

      • alastairp
        has anyone else complained about the format of the date in the API?
      • 2022-02-28 05921, 2022

      • alastairp
        I remember seeing something somewhere (here in irc?) about the lisened_at, but that's something else to deal with later
      • 2022-02-28 05922, 2022

      • lucifer
        LB-754
      • 2022-02-28 05923, 2022

      • BrainzBot
        LB-754: Make inserted_at a standard unix timestamp https://tickets.metabrainz.org/browse/LB-754
      • 2022-02-28 05934, 2022

      • CatQuest
        alastairp: :)
      • 2022-02-28 05955, 2022

      • lucifer
        i don't think inserted_at is of use to api users anyways. it was used in spark but other than that i don't know its purpose in dumps.
      • 2022-02-28 05909, 2022

      • alastairp
        oh, interesting
      • 2022-02-28 05953, 2022

      • alastairp
        so it's useful/needed for _making dumps_, but perhaps not as useful once you have them loaded somewhere else?
      • 2022-02-28 05919, 2022

      • lucifer
        i'll download and spot check the latest inc dump to see if it is even present in dumps. looking at the code i don't think it should be
      • 2022-02-28 05949, 2022

      • alastairp
        as I mentioned in LB-987, it would be useful if you wanted to perfectly recreate the database from a full dump + incremental dump