#metabrainz

/

      • agatzk29 joined the channel
      • 2021-05-14 13433, 2021

      • agatzk29 has quit
      • 2021-05-14 13459, 2021

      • agatzk22 joined the channel
      • 2021-05-14 13408, 2021

      • thomasross joined the channel
      • 2021-05-14 13456, 2021

      • thomasross has quit
      • 2021-05-14 13455, 2021

      • BrainzGit
        [musicbrainz-server] reosarevok opened pull request #2100 (master…MBS-11662): MBS-11662: Lowercase untitled/unknown if in brackets https://github.com/metabrainz/musicbrainz-server/…
      • 2021-05-14 13432, 2021

      • Freso
        Wb reosarevok 👋
      • 2021-05-14 13440, 2021

      • reosarevok
        Hi! I got a break, it was nice, can recommend
      • 2021-05-14 13401, 2021

      • reosarevok
        Hope nothing got too messed up while I was away
      • 2021-05-14 13441, 2021

      • Mr_Monkey
        We managed. Welcome back !
      • 2021-05-14 13454, 2021

      • ruaok
        and happy birthday too!
      • 2021-05-14 13459, 2021

      • ruaok
        (mooin!)
      • 2021-05-14 13406, 2021

      • reosarevok
        Thanks!
      • 2021-05-14 13450, 2021

      • alastairp
        "Instrucció per a la recuperació progressiva de la presencialitat del PAS a la UPF"
      • 2021-05-14 13451, 2021

      • alastairp
        :(
      • 2021-05-14 13412, 2021

      • sumedh joined the channel
      • 2021-05-14 13405, 2021

      • ruaok
        as in its time to go back to the office?
      • 2021-05-14 13426, 2021

      • alastairp
        yep, starting from next week!
      • 2021-05-14 13429, 2021

      • alastairp
        that was fast
      • 2021-05-14 13410, 2021

      • ruaok
        no way to say "nope, not until vaccine!" ?
      • 2021-05-14 13421, 2021

      • alastairp
        well, that's definitely what I'm going to say
      • 2021-05-14 13432, 2021

      • alastairp
        we'll see if there's any fallout from it
      • 2021-05-14 13406, 2021

      • BrainzGit
        [listenbrainz-server] amCap1712 opened pull request #1463 (master…master): Use docker buildx to build production images https://github.com/metabrainz/listenbrainz-server…
      • 2021-05-14 13454, 2021

      • reg[m] has quit
      • 2021-05-14 13400, 2021

      • reosarevok
        ruaok (and Freso maybe?): opinions on https://wiki.musicbrainz.org/index.php?title=Data… ?
      • 2021-05-14 13424, 2021

      • _lucifer
        alastairp, based on your whether cron is running PR, how about we also add an action to validate and display the crontabs in human readable form if a crontab is modified?
      • 2021-05-14 13441, 2021

      • _lucifer
        we do not need to do this now but seems a useful future enhancement
      • 2021-05-14 13449, 2021

      • ruaok
        reosarevok: seems good.
      • 2021-05-14 13419, 2021

      • Freso
        I would remove the "if you ask nicely", but otherwise seems good.
      • 2021-05-14 13420, 2021

      • alastairp
        mmm, maybe itercron will raise an exception if a pattern is invalid? I don't know if it can turn a pattern into a descriptive text
      • 2021-05-14 13448, 2021

      • _lucifer
        there's a is_valid function to check if the pattern is valid
      • 2021-05-14 13457, 2021

      • alastairp
        I guess ruaok and I have been around long enough that (most of the time) we know how to write a cron without screwing it up
      • 2021-05-14 13416, 2021

      • alastairp
        would be happy to add additional cron management commands if you'd find it useful
      • 2021-05-14 13456, 2021

      • ruaok snaps his suspenders in agreement
      • 2021-05-14 13407, 2021

      • reosarevok
        Freso: well, the point is we are not obliged to remove the name AFAICT and I certainly won't do it if they are dicks about it
      • 2021-05-14 13411, 2021

      • _lucifer
        👍, human readable text would have been nice but i do not see that in the library docs so let's leave it.
      • 2021-05-14 13417, 2021

      • reosarevok
        So I thought a word of warning might be useful
      • 2021-05-14 13453, 2021

      • alastairp
        https://bradymholt.github.io/cron-expression-desc… it looks like there are tools to do it, though?
      • 2021-05-14 13412, 2021

      • sumedh has quit
      • 2021-05-14 13427, 2021

      • alastairp
        "At 12:00 AM, on day 1 and 15 of the month"
      • 2021-05-14 13449, 2021

      • alastairp
        "At 04:00 AM, only on Monday"
      • 2021-05-14 13401, 2021

      • _lucifer
        thanks! bookmarked, that should do the job
      • 2021-05-14 13417, 2021

      • ruaok
        what's not human readable about crontabs? there is a perfect translator available. its called "man"
      • 2021-05-14 13433, 2021

      • alastairp
        man 5 crontab not on Macs :( I always have to ssh somewhere else
      • 2021-05-14 13450, 2021

      • ruaok
      • 2021-05-14 13404, 2021

      • ruaok
        yep. another reason why macs are being phased out of my life.
      • 2021-05-14 13417, 2021

      • ruaok
        linux on the desktop has quietly arrived.
      • 2021-05-14 13431, 2021

      • reosarevok
        I thought it had been here for 10 years
      • 2021-05-14 13441, 2021

      • alastairp
        though, mixing up 0 and * in the minutes column can be problematic
      • 2021-05-14 13427, 2021

      • reg[m] joined the channel
      • 2021-05-14 13414, 2021

      • ruaok
        `0 * * * * dd if=/dev/null of=/dev/md0` is my favorite crontab entry
      • 2021-05-14 13426, 2021

      • ruaok
        long walk off a short pier.
      • 2021-05-14 13441, 2021

      • alastairp
        don't do this at home, kids
      • 2021-05-14 13419, 2021

      • ruaok
        okk, less destructive... `while 1: fork()`
      • 2021-05-14 13430, 2021

      • ruaok
        dont do that one at home or on our servers, either. :)
      • 2021-05-14 13444, 2021

      • alastairp looks for a zipbomb to email reosarevok
      • 2021-05-14 13402, 2021

      • alastairp
      • 2021-05-14 13414, 2021

      • ruaok
        hmm. interesting. I should read up on the USB-3 spec. I wonder how the data is multiplexed
      • 2021-05-14 13427, 2021

      • alastairp
        there's so much going on there
      • 2021-05-14 13437, 2021

      • alastairp
        usbc/thunderbolt/alt modes, etc
      • 2021-05-14 13419, 2021

      • alastairp
        a friend of mine has a 4k monitor with built in ethernet and usb hub, and if he wants 60hz he can't run the hub at usb-3/gigabit ethernet because there's not enough bandwidth
      • 2021-05-14 13455, 2021

      • BrainzGit
        [musicbrainz-server] reosarevok opened pull request #2101 (master…MBS-10205): MBS-10205: Readd link to the Data Removal Policy to the menu https://github.com/metabrainz/musicbrainz-server/…
      • 2021-05-14 13420, 2021

      • SamThursfield[m] has quit
      • 2021-05-14 13449, 2021

      • _lucifer
        alastairp, ruaok: we do not have buildx on bono. can we install that?
      • 2021-05-14 13456, 2021

      • _lucifer
        *docker buildx
      • 2021-05-14 13422, 2021

      • ruaok
        sure, have at it. bono is our wild west, right?
      • 2021-05-14 13437, 2021

      • alastairp
        you'll have to install a new version of docker
      • 2021-05-14 13443, 2021

      • _lucifer
        lol
      • 2021-05-14 13455, 2021

      • _lucifer
        i checked we have 19.03 so it should work
      • 2021-05-14 13420, 2021

      • _lucifer
        only need to install buildx
      • 2021-05-14 13440, 2021

      • alastairp
        maybe there's a setting to add in the daemon.json file
      • 2021-05-14 13457, 2021

      • ruaok
        alastairp: for `COPY ./docker/services/spark_reader/consul-template-spark-reader.conf /etc/consul-template-spark-reader.conf` how does the base image know to look at this file?
      • 2021-05-14 13412, 2021

      • alastairp
        ruaok: sorry - missed one more item
      • 2021-05-14 13419, 2021

      • alastairp
        we add `down` to all services
      • 2021-05-14 13427, 2021

      • alastairp
        and then in rc.local rm the down based on env variables
      • 2021-05-14 13446, 2021

      • ruaok
        understood, but this is a one service container that I always want up. so I should not need down, right?
      • 2021-05-14 13418, 2021

      • alastairp
        right, so add a new runit service with a `run` file
      • 2021-05-14 13428, 2021

      • ruaok
        got that.
      • 2021-05-14 13448, 2021

      • alastairp
        `exec run-consul-template -config /etc/consul-template-spotify-reader.conf`
      • 2021-05-14 13450, 2021

      • ruaok
        `exec /code/metrics/metric_writer.py`
      • 2021-05-14 13458, 2021

      • ruaok
        oh, I see.
      • 2021-05-14 13402, 2021

      • ruaok
        got it.
      • 2021-05-14 13406, 2021

      • alastairp
        👍
      • 2021-05-14 13431, 2021

      • ruaok
        because the exec that I was missing in in template...conf
      • 2021-05-14 13437, 2021

      • alastairp
        yep, right
      • 2021-05-14 13452, 2021

      • alastairp
        render the config file with consul _then_ call the thing in the exec block
      • 2021-05-14 13400, 2021

      • _lucifer
        https://github.com/docker/buildx/#docker-ce does mention that it can be enabled by adding a entry to config.json
      • 2021-05-14 13439, 2021

      • ruaok
        for metric support, what should my base image be?
      • 2021-05-14 13423, 2021

      • alastairp
        I think the python one is a good choice
      • 2021-05-14 13411, 2021

      • ruaok
        clearly. but which version?
      • 2021-05-14 13413, 2021

      • ruaok
        ARG PYTHON_BASE_IMAGE_VERSION=3.7-20210115
      • 2021-05-14 13422, 2021

      • ruaok
        for this I might as well go with 3.9.
      • 2021-05-14 13438, 2021

      • alastairp
        yeah, I see no reason not to
      • 2021-05-14 13406, 2021

      • ruaok
        but 20210115 won't have metric support in, will it?
      • 2021-05-14 13413, 2021

      • ruaok
        where do I find what versions we have available?
      • 2021-05-14 13416, 2021

      • alastairp
        metric support is part of BU
      • 2021-05-14 13426, 2021

      • alastairp
        so just add that in requirements
      • 2021-05-14 13405, 2021

      • _lucifer
      • 2021-05-14 13407, 2021

      • ruaok
        ah and BU is not part of the base image. I see.
      • 2021-05-14 13427, 2021

      • ruaok
        _lucifer: got it, thanks.
      • 2021-05-14 13445, 2021

      • ruaok
        so requirements.txt has `git+https://github.com/metabrainz/brainzutils-python.git@v2.0.1` ?
      • 2021-05-14 13451, 2021

      • alastairp
        yes
      • 2021-05-14 13409, 2021

      • ruaok
        ok, i think that completes all the pieces. time to give it whirl.
      • 2021-05-14 13425, 2021

      • ruaok
      • 2021-05-14 13454, 2021

      • ruaok
        any tips on how to debug this? the flags for run-consul-template are not terribly helpful.
      • 2021-05-14 13425, 2021

      • alastairp
        are you just running it from a `docker run`?
      • 2021-05-14 13433, 2021

      • _lucifer
        i woudl suggest copying stuff from the statup improvements PR
      • 2021-05-14 13403, 2021

      • alastairp
        consul-agent requires syslog to be started up (which runit will do)
      • 2021-05-14 13411, 2021

      • ruaok
        the container just restarts, so I am doing a `docker run --rm -it ... bash`
      • 2021-05-14 13403, 2021

      • ruaok
        yep, i have a valid looking /etc/service/syslog-ng/run
      • 2021-05-14 13434, 2021

      • alastairp
        I normally see this happening - consul-template starts up, doesn't see syslog, prints the error, syslog starts up, consul-template restarts, connects to syslog, and goes silent
      • 2021-05-14 13400, 2021

      • _lucifer
        reosarevok, i need to create a couple of fake users on MB for testing. can't use test.mb. what's the best way to go about it. i go ahead and create. after i am done, delete the account?
      • 2021-05-14 13418, 2021

      • alastairp
        but I'm unsure about the restart, sorry. I'm in a call, but can take a look at it afterwards if you get the code up
      • 2021-05-14 13431, 2021

      • reosarevok
        _lucifer: or don't delete them
      • 2021-05-14 13444, 2021

      • reosarevok
        I see no problem with having ListenBrainz Test User or something
      • 2021-05-14 13400, 2021

      • _lucifer
        cool, thanks!
      • 2021-05-14 13400, 2021

      • ruaok
      • 2021-05-14 13405, 2021

      • ruaok
        any bits in particular?
      • 2021-05-14 13442, 2021

      • _lucifer
        yes, one sec. let me see which ones are needed.
      • 2021-05-14 13434, 2021

      • _lucifer
      • 2021-05-14 13445, 2021

      • _lucifer
        these 3 files, you can leave out the sentry stuff
      • 2021-05-14 13416, 2021

      • _lucifer
        plus you'll need a few changes to consul config like https://github.com/metabrainz/listenbrainz-server…
      • 2021-05-14 13431, 2021

      • _lucifer
        if you have the code in a repo, i can set it up
      • 2021-05-14 13439, 2021

      • ruaok
        <3 <3 <3
      • 2021-05-14 13443, 2021

      • ruaok
        one sec
      • 2021-05-14 13445, 2021

      • alastairp
        ruaok: is `run` +x ?
      • 2021-05-14 13449, 2021

      • ruaok
        yes
      • 2021-05-14 13453, 2021

      • alastairp
        👍
      • 2021-05-14 13422, 2021

      • ruaok
        _lucifer: git@github.com:metabrainz/metric-writer.git
      • 2021-05-14 13429, 2021

      • _lucifer
        👍
      • 2021-05-14 13444, 2021

      • _lucifer
        ruaok, just saw your consul template, i think the startup improvements won't benefit you :(. those improvements are useful when loading stuff like keys from consul.
      • 2021-05-14 13453, 2021

      • ruaok
        ah, I feel better now. I looked at your links and wasn't certain what I should be copying.
      • 2021-05-14 13403, 2021

      • ruaok
        thanks though.
      • 2021-05-14 13413, 2021

      • _lucifer
        one thing though, you might need a `cd /code/metrics` somewhere.
      • 2021-05-14 13435, 2021

      • _lucifer
      • 2021-05-14 13448, 2021

      • _lucifer
        for example we have this in LB
      • 2021-05-14 13456, 2021

      • _lucifer
        i see you do specify the complete file name, but i am not sure we don't do that in LB so thought to mention.
      • 2021-05-14 13459, 2021

      • ruaok
        did that, no change.
      • 2021-05-14 13448, 2021

      • _lucifer
        :(
      • 2021-05-14 13415, 2021

      • _lucifer
        also not related to current errors but https://github.com/metabrainz/metric-writer/blob/… you probably want to lowercase REDIS_SERVERS here.
      • 2021-05-14 13410, 2021

      • ruaok
        thx, fixed that already, just not committed yet.
      • 2021-05-14 13415, 2021

      • ruaok
        happy to hit THAT error. :)
      • 2021-05-14 13407, 2021

      • _lucifer
        :D
      • 2021-05-14 13430, 2021

      • _lucifer
        next up, try passing -log-level debug to run-consul-template
      • 2021-05-14 13456, 2021

      • ruaok
        did, that seems to be the case already, no change.
      • 2021-05-14 13431, 2021

      • ruaok
        even with no stuff in consul_config.py.ctmpl nothing gets generated.
      • 2021-05-14 13452, 2021

      • sumedh joined the channel
      • 2021-05-14 13426, 2021

      • ruaok
        alastairp: when you're done with your meeting, I would love help on this.
      • 2021-05-14 13439, 2021

      • _lucifer
        one option would be to not enable syslog. i think for testing that's fine?
      • 2021-05-14 13418, 2021

      • ruaok
        can't hurt to try, but I doubt the error messages it is printing are really relevant. those are ALWAYS printed. 🙄