#metabrainz

/

      • bitmap
        that's all for me. go zas
      • zas
        Hey
      • last week we released Picard 2.8 (https://blog.metabrainz.org/2022/05/24/picard-2...)
      • we add a small meeting with new GSoC student skelly37 with outsidecontext
      • we discussed a bit about Picard dev priorities for upcoming 2.9 & 3.0
      • I also deployed a workaround for an annoying warning about SSD failure on trille (according to Hetzner the problem isn't real, and the net seems to confirm that)
      • I started few polls about auto-editor system (https://community.metabrainz.org/t/auto-editors...), I find elections too rare
      • plus usual upgrades, supervision, user support, minor issues handling. fin. atj?
      • atj
        hi
      • last week I made some changes to our smartmontools and netplan ansible roles. the first was to resolve the issue zas mentioned just now and the second was prep work for moving the spark cluster to ansible.
      • zas
        atj: thanks for your work on ansible, much appreciated
      • atj
        I had a look at the spark cluster configuration, and created a WIP PR for our ansible playbook
      • we've been focused on physical servers up until this point, so including VMs requires some reorganisation
      • hopefully I'll find some more time to work on that this week so we can get the jackson VMs into ansible ASAP
      • mayhem
        those are bare metal, not VMs.
      • not sure it matters.
      • atj
        heh, are they?
      • mayhem
        yep
      • zas
        yes, but different group, SOLR cloud is on VMs though
      • mayhem
        the server auction type, if I am not mistaken. (read: cheap)
      • alastairp
        that being said, if you want to practise with VMs then I can help to work on brainzbot and sentry VMs
      • zas
        alastairp: this will come at some point ;)
      • atj
        sorry, ignore my mention of VMs then :) however they have VLAN interfaces which is the primary difference
      • alastairp
        yes, not pressuring, just offering some servers if necessary :)
      • odnes_ joined the channel
      • mayhem
        sorry, we're a bunch of pedants
      • atj
        I'm glad you pointed it out
      • odnes has quit
      • I think I got confused between SOLR and Spark
      • yvanzo
        the correct writing is Solr
      • atj
        thanks yvanzo :)
      • CatQuest
        mbssssssss
      • zas
        lol :)
      • yvanzo
        I’m just following the team :)
      • alastairp
        solR, it's a mid-2010s social network
      • atj
        that's about it I think, FYI I'll be AFK on Thursday and Friday, as we have been bestowed with an extra bank holiday thanks to our archaic system of government involving strange woman lying in ponds distributing swords
      • CatQuest
        mfw i realise "mid 2010s" is like a decade ago
      • king arthur day is a thing?
      • atj
        who else is up?
      • monkey
        +1 for Python reference
      • CatQuest
        listen buddy, no watery tart has ever given me a sword!
      • atj
        we may well see the violence inherent in the system
      • reosarevok
        I thought only covid parties were inherent in the system
      • yvanzo
        atj: nobody else, just your chapter about securing MeB infra
      • CatQuest
        i prefer corvid parties
      • they're a murder!
      • reosarevok groans
      • Success!
      • atj
        is that the name for a group of crows?
      • CatQuest
        thatsthejoke.png
      • yes :D
      • alastairp
        shall we move on?
      • atj
        I was just checking I understood it
      • yep
      • CatQuest
        absolutely
      • reosarevok
        Freso: time to move on? :)
      • alastairp
        if we understand correctly, that's it for reviews
      • Freso
        reosarevok: Probably. :)
      • alastairp
        next up, part 5 of the trilogy on securing infrastructure
      • yvanzo
        atj: we were at “Reducing docker container capabilities” which you mostly wrote (IIRC)
      • CatQuest
        part 5, the trillening
      • yvanzo
        actually should be part 4 again, last time we just stated that the author was missing.
      • atj
        yes, sorry about that
      • CatQuest
        part 5: part 4,5
      • CatQuest watches the entire chat go apeshit at their norwegian desimals
      • yvanzo
        Don't worry, we didn’t think about looking at it in advance either.
      • atj
        so, the main thrust of my thoughts on this topic are summarised by the Docker docs: "Docker containers are, by default, quite secure; especially if you run your processes as non-privileged users inside the container."
      • "quite" doing a bit of work there
      • akshaaatt gets reminded of AoT anime which has `seasons` for the final season
      • my understanding is that pretty much everything in docker is running as root at present
      • alastairp
        atj: so I think we agreed that we should focus on moving to non-root
      • yvanzo
        Changing the users inside the container is definitely something that can be done at project-level, a TODO item has already been added.
      • alastairp
        some services are root, others use non-privileged users
      • atj
        if reducing that is feasible, then I think it would be a win from a security perspectivre
      • *perspective
      • alastairp
        what are your thoughts on capabilities and rootless?
      • CatQuest
        hmm iirc "quite" menas diff in AE and BE ?
      • monkey
        Quite.
      • atj
        capabilities are good, but complex as they can often make things fail in odd ways
      • yvanzo
        atj: The last point (6min left) is about permissions to run docker commands.
      • atj
        alastairp: rootless is probably more hassle than it's worth I think, but I need to investigate further
      • yvanzo: I'm not sure how we can feasibly reduce that attack surface but am open to suggestions
      • giving users docker access is akin to granting root, as we know
      • and everyone who has docker access has sudo anyway
      • yvanzo
        I don’t know if removing sudo users from docker group would make a difference?
      • atj
        somewhat, but I think most people probably need both?
      • yvanzo
        IIRC it can make a difference in some context, which is both are supported by some Docker Compose projects.
      • odnes_ has quit
      • atj
        personally I think we should try to focus on use of non-root users as much as possible, and examine the feasibility of reducing capabilities
      • yvanzo
        atj: Support for both 'docker' and 'sudo docker' command has sometimes be implemented through DOCKER_CMD: http://livegrep.metabrainz.org/search/livegrep?...
      • zas
        Yes, most users need both.
      • atj
        trying to move to fine grained user permissions is likely to cause sysadmin headaches
      • and frustrated users
      • yvanzo
        Would generalizing the use of DOCKER_CMD removes this need for docker group?
      • Etua joined the channel
      • alastairp
        I think that moving to non-root users is a pretty straightforward step that we can do, and so we should focus on that first
      • yvanzo: maybe for scripts, but for general interaction on servers - running one-off commands etc I'm used to running without sudo
      • atj
        yvanzo: standardise one or the other, personally I'd probably avoid sudo, but not sure on other peoples thoughts
      • yvanzo
        I agree, not suggested to move to fine grained permissions right away, but to find a mid-term path to it.
      • alastairp
        I'm sure it's a small thing to get used to, but...
      • atj
        yvanzo: probably more like long term at this stage I think
      • yvanzo
        alastairp: you can also set a shell alias.
      • Freso
        Are we at a point where we can wrap up?
      • atj
        we have 2 different attack surfaces: a) application vuln resulting in RCE and b) user account compromise
      • Freso
        Or should we end general meeting and continue talk for involved and interested parties?
      • yvanzo
        I’m good.
      • atj
        we can end the general meeting, I'll try to look into docker capabilities more and if we can make a note to focus on moving to non-root users in containers when feasible
      • mayhem needs to go
      • Freso
        Alright then.
      • Thank you all for your time! Take care, stay safe, drink water, wear your masks!
      • </BANG>
      • monkey
        FWIW, I checked last week for BookBrainz images, they are running with a non-sudo user
      • akshaaatt
        Thank you!
      • monkey
        non-root*
      • atj
        thanks monkey
      • alastairp
        monkey: great!
      • thanks everyone
      • one thing that trips me up for non-root is one-off installations in a container
      • monkey
        This is the default for node-based Docker images, which the BB ones are based on.
      • monkey gotta go too, byyeeeeee
      • alastairp
        e.g. suddenly I need to apt instlal something extra
      • or pip install for that matter (as we install packages to /usr/local)
      • so having docker run start up as a regular user is really annoying
      • could be worked around in development by using --user flag to docker run. not sure how many times it's come up that we need to install something in a `docker exec` on prod
      • atj
        shouldn't really be doing that in prod tbh :)
      • reosarevok
        bitmap: https://github.com/metabrainz/musicbrainz-serve... should be fine to merge then
      • atj
        but I understand needs must
      • alastairp
        yes yes :)
      • reosarevok
        yvanzo, bitmap: let's merge anything that's left to merge in the next 20 min or so and then I'll start a release
      • yvanzo
        atj: Added two TODOs to the doc for the last item we discussed during the meeting.
      • atj
        yvanzo: thank you
      • BrainzGit
        [musicbrainz-server] 14mwiencek merged pull request #2539 (03master…upgrade.sh-enable-reptg-last): Run all sql before enabling replication triggers https://github.com/metabrainz/musicbrainz-serve...
      • alastairp
        yvanzo: my question about MB on wolf - I see that we have sir local dev enabled on the checkout, and a `sir` directory in musicbrainz home directory
      • I don't know if we had live indexing (or solr at all) set up here
      • one thing, which I will open a ticket for unless you point me to some documentation that I can read: step 5 says "If you were using the live search indexing, uninstall its triggers as follows", it'd be great to have a command to run to see if we are using live indexing (helpful for people who are maintaining a system that someone else set up)
      • yvanzo
        alastairp: Is MB search needed on wolf instance?
      • (including MB Search API)
      • alastairp
        I'm 80% sure no, but I'm not sure if anyone else is using it
      • yvanzo
        I may have enabled it for the purpose of schema change test.
      • alastairp
        ah :)
      • not going to lie, feeling like I'm missing out a bit: https://twitter.com/emfcamp/status/153133754057...
      • > We also support important phone services such as dial-up, fax, ISDN, ADSL, and bulletin boards. If you’d like to bring along your modem and fax machine the CuTEL wiki page has the details.
      • so yes, they have fibre to a field in the middle of nowhere. but you _can_ dial in to get online if you want
      • BrainzGit
        [troi-recommendation-playground] 14amCap1712 opened pull request #57 (03main…raw-recs): Add raw recommendations to recs_to_playlist patch https://github.com/metabrainz/troi-recommendati...
      • yvanzo
        mayhem: That ^ reminds me that I installed a development version of SIR for VolumIO mirror, they should just use the current SIR instead now, but they probably figured that out already.
      • alastairp: to answere your initial question: The command "admin/configure show" will mention "live-indexing-search" if it is still on.
      • alastairp
        yvanzo: ah right, I understand. so if admin/configure was used to create an indexer service, then it's in use
      • I was concentrating too much on setup-amqp-triggers script to install/uninstall triggers rather than looking for the running service
      • yvanzo
        alastairp: Yes, it is enabled with point no. 4 in https://github.com/metabrainz/musicbrainz-docke...
      • alastairp
        (I had a very quick look for a script that would print a list of the triggers if they were installed)