#metabrainz

/

      • alastairp
        right, it uses the build/image trick
      • 2021-05-06 12601, 2021

      • alastairp
        if you have both in a docker-compose file, it performs the build and then tags it with `image:`
      • 2021-05-06 12602, 2021

      • ruaok
        oy. I think some tuning on ts is necessary.
      • 2021-05-06 12619, 2021

      • ruaok
        it drives gaga to 98% disk space twice a day.
      • 2021-05-06 12641, 2021

      • alastairp
        oops. anything in logs about what it's doing?
      • 2021-05-06 12654, 2021

      • alastairp
        recreating those materialised views?
      • 2021-05-06 12605, 2021

      • ruaok
        I'm guessing that is it.
      • 2021-05-06 12615, 2021

      • ruaok
        I'll have to have a read on how to tune that.
      • 2021-05-06 12647, 2021

      • _lucifer
        ah ok! makes sense.
      • 2021-05-06 12650, 2021

      • alastairp
        _lucifer: I just added ./annoy_indices to dockerignore
      • 2021-05-06 12655, 2021

      • alastairp
        I told you it was hacky :)
      • 2021-05-06 12636, 2021

      • _lucifer
        a lot of our work is :D
      • 2021-05-06 12653, 2021

      • alastairp
        oh - the other hacky part is that we're using postgres running directly on bono, and mounting in /var/run/postgresql to the image so that it can connect
      • 2021-05-06 12624, 2021

      • alastairp
        in other news, I just tested a build and BU + setuptools dependency with no versions makes it try and install a version which isn't python-2 compatible
      • 2021-05-06 12625, 2021

      • _lucifer
        yup, i guessed that was the case seeing the volumes.
      • 2021-05-06 12637, 2021

      • _lucifer
        in BU 2 or 1.18?
      • 2021-05-06 12649, 2021

      • alastairp
        weirdly it's using 1.15
      • 2021-05-06 12621, 2021

      • _lucifer
        oh i see i changed the branch to download_ds.
      • 2021-05-06 12639, 2021

      • _lucifer
        i should probably merge master into the branch first.
      • 2021-05-06 12620, 2021

      • alastairp
      • 2021-05-06 12642, 2021

      • alastairp
        so we should be able to fix it by explicitly installing a lower version of that before installing from requirements.txt
      • 2021-05-06 12601, 2021

      • alastairp
        (the discussion we had about the model training PR will allow us to upgrade to py 3)
      • 2021-05-06 12603, 2021

      • _lucifer
        the latest AB prod is using BU 1.18.1 and that builds fine right?
      • 2021-05-06 12621, 2021

      • alastairp
        I don't know the answer to either of those questions :)
      • 2021-05-06 12630, 2021

      • alastairp
        we updated AB to 1.18.1
      • 2021-05-06 12636, 2021

      • _lucifer
      • 2021-05-06 12610, 2021

      • alastairp
        but because the dependency for setuptools_scm in BU's setup.py has no version number, it seems like they released a new version which dropped py 2 support
      • 2021-05-06 12621, 2021

      • _lucifer
        oh!
      • 2021-05-06 12631, 2021

      • _lucifer
        makes sense.
      • 2021-05-06 12642, 2021

      • agatzk joined the channel
      • 2021-05-06 12642, 2021

      • alastairp
        we could release BU 1.18.2 which includes a version restriction for AB
      • 2021-05-06 12608, 2021

      • _lucifer
        sure, let's do that.
      • 2021-05-06 12649, 2021

      • alastairp
        OK, let me do that. we discussed that we'll need a branch if we continue to make 1.18 releases. what do you want to call it? 1.18, or 1.x ?
      • 2021-05-06 12609, 2021

      • _lucifer
        there's already one iirc.
      • 2021-05-06 12626, 2021

      • _lucifer
        v1.18
      • 2021-05-06 12603, 2021

      • alastairp
        oops, missed the fact that we made one of those. thanks
      • 2021-05-06 12639, 2021

      • alastairp
        although - now I'm a bit confused. latest setuptools_scm was released March 17, but our change was made March 31
      • 2021-05-06 12650, 2021

      • alastairp
        maybe it's a dependency of a dependency
      • 2021-05-06 12654, 2021

      • _lucifer
        yeah, just saw that.
      • 2021-05-06 12611, 2021

      • _lucifer
        they dropped python2 support in 6.0.0.
      • 2021-05-06 12644, 2021

      • _lucifer
        maybe on March 31, somehow we still got the 5.* version so it worked?
      • 2021-05-06 12650, 2021

      • alastairp
      • 2021-05-06 12610, 2021

      • alastairp
      • 2021-05-06 12632, 2021

      • _lucifer
        actually, its building fine now.
      • 2021-05-06 12635, 2021

      • alastairp
        yeah, it's possible that something like that happened?
      • 2021-05-06 12642, 2021

      • alastairp
        huh. right
      • 2021-05-06 12650, 2021

      • _lucifer
        i merged master into the download_ds branch and now it installed correctly.
      • 2021-05-06 12629, 2021

      • _lucifer
        so something happened between 1.15 and 1.18.1?
      • 2021-05-06 12632, 2021

      • _lucifer
        AB is up on bono now but it is returning 502s.
      • 2021-05-06 12640, 2021

      • alastairp
        possibly. let's just ignore it :)
      • 2021-05-06 12650, 2021

      • _lucifer
        yup.
      • 2021-05-06 12614, 2021

      • _lucifer
        why do we need nginx on bono to run AB?
      • 2021-05-06 12646, 2021

      • alastairp
        because we're running it in uwsgi, not the dev server. so we need something that speaks http
      • 2021-05-06 12615, 2021

      • _lucifer
        ah ok!
      • 2021-05-06 12628, 2021

      • _lucifer
        i see consul issues in the logs.
      • 2021-05-06 12657, 2021

      • alastairp
        it's possible that the last time I ran this was before I made startup changes to AB
      • 2021-05-06 12603, 2021

      • _lucifer
        probably. so how do we update the consul config here?
      • 2021-05-06 12656, 2021

      • alastairp
        the easiest way may be to just have a local config.py in the repo, and then in docker-compose.yml set the `command` to run. this way it won't do any of the prod startup process, and will skip consul
      • 2021-05-06 12620, 2021

      • alastairp
        in fact, there is already a config.py, so that might be what I was using before
      • 2021-05-06 12612, 2021

      • alastairp
        I've added command: to docker-compose.yml, restarting
      • 2021-05-06 12603, 2021

      • alastairp
      • 2021-05-06 12613, 2021

      • alastairp
        well, it's running. doesn't look good :)
      • 2021-05-06 12634, 2021

      • _lucifer
        thanks!
      • 2021-05-06 12632, 2021

      • _lucifer
        the logs are clean though.
      • 2021-05-06 12618, 2021

      • _lucifer
        alastairp: its up here http://bono.metabrainz.org:8001/
      • 2021-05-06 12636, 2021

      • alastairp
        it's possible that 1) nginx is trying to serve static files and not pass through to web, or 2) they've not been built in web for some reason
      • 2021-05-06 12612, 2021

      • _lucifer
        yeah, seems the static files not built. css and js not working.
      • 2021-05-06 12607, 2021

      • _lucifer
        can we just bring up a dev instance?
      • 2021-05-06 12615, 2021

      • Zhele has quit
      • 2021-05-06 12616, 2021

      • alastairp
        yeah, that should work too. just take everything down, find a port that's not used, and consider using the local database so that you have some content
      • 2021-05-06 12625, 2021

      • Zhele joined the channel
      • 2021-05-06 12628, 2021

      • ROpdebee has quit
      • 2021-05-06 12627, 2021

      • ruaok
        alastairp: the issue with gaga running out of diskspace has to do with the old continuous aggregates. we should really work to merge the time ranges PR ASAFP.
      • 2021-05-06 12644, 2021

      • ruaok
        I think the new continuous aggregate will use less diskspace.
      • 2021-05-06 12638, 2021

      • ROpdebee joined the channel
      • 2021-05-06 12640, 2021

      • BrainzGit
        [bookbrainz-site] the-good-boy opened pull request #622 (master…Column-min-width): fix(BB-405):Seach: Tabulated results columns have dynamic width https://github.com/bookbrainz/bookbrainz-site/pul…
      • 2021-05-06 12627, 2021

      • alastairp
        ruaok: ok cool. I'll prioritiese that tomorrow
      • 2021-05-06 12614, 2021

      • _lucifer
      • 2021-05-06 12640, 2021

      • _lucifer
        looks super interesting
      • 2021-05-06 12658, 2021

      • ruaok
        alastairp: ok, ping me when you start work tomorrow. lets tag team the PR and release tomorrow.
      • 2021-05-06 12645, 2021

      • _lucifer
      • 2021-05-06 12611, 2021

      • _lucifer
        ruaok, alastairp: any idea about ^?
      • 2021-05-06 12649, 2021

      • _lucifer
        http is being redirected to https on bono
      • 2021-05-06 12616, 2021

      • ruaok
        yes.
      • 2021-05-06 12621, 2021

      • ruaok
        that is intended.
      • 2021-05-06 12659, 2021

      • ruaok
        in order for letsencrypt to assign certs it needs to have access to port 80 (unencrypted) to start the process. once the process is done, all port 80 requests are sent to 443.
      • 2021-05-06 12617, 2021

      • ruaok
        what do you want to do?
      • 2021-05-06 12626, 2021

      • _lucifer
        but this is running on another port not 80
      • 2021-05-06 12635, 2021

      • _lucifer
        bring up a dev instance of AB
      • 2021-05-06 12610, 2021

      • ruaok
        ah. its a different side of the same setup.
      • 2021-05-06 12647, 2021

      • ruaok
        I guess you are connecting to the musicbrainz_docker (or whatever) network? if so, any container that is found on that network will be shunted through SSL.
      • 2021-05-06 12609, 2021

      • _lucifer
        yeah right. oh ok.
      • 2021-05-06 12627, 2021

      • _lucifer
        is there a way to get around this?
      • 2021-05-06 12630, 2021

      • ruaok
        the setup is quite hacky, it could do with moving to a separate account.
      • 2021-05-06 12659, 2021

      • ruaok
        I'll set it up for you/
      • 2021-05-06 12610, 2021

      • _lucifer
        thanks!
      • 2021-05-06 12612, 2021

      • ruaok
        you want to expose port 888, which is HTTP, yes?
      • 2021-05-06 12621, 2021

      • _lucifer
        yes
      • 2021-05-06 12652, 2021

      • ruaok
        ok, you can do it yourself, actually. :)
      • 2021-05-06 12609, 2021

      • ruaok
      • 2021-05-06 12629, 2021

      • ruaok
        if you set these 3 env vars on your -web containers, magic will work.
      • 2021-05-06 12639, 2021

      • ruaok
        we'll need to setup dns for this. what should we call it.
      • 2021-05-06 12651, 2021

      • ruaok
      • 2021-05-06 12609, 2021

      • ruaok
        don't request that address, yet. don't fill DNS caches with bad data.
      • 2021-05-06 12616, 2021

      • _lucifer
        maybe dev.lb.org?
      • 2021-05-06 12642, 2021

      • _lucifer
        i'll probably reuse that for different services then
      • 2021-05-06 12655, 2021

      • ruaok
        ok
      • 2021-05-06 12629, 2021

      • ruaok
        ok, DNS records set.
      • 2021-05-06 12639, 2021

      • ruaok
        go ahead and set those env vars.
      • 2021-05-06 12651, 2021

      • ruaok
        then watch the `le` container
      • 2021-05-06 12608, 2021

      • _lucifer
        on it
      • 2021-05-06 12652, 2021

      • ruaok
        nooice!
      • 2021-05-06 12619, 2021

      • ruaok
        well, it looked to come up ok.
      • 2021-05-06 12626, 2021

      • ruaok
        but page doesn't load.
      • 2021-05-06 12636, 2021

      • _lucifer
        getting too many redirects error. yeah.
      • 2021-05-06 12652, 2021

      • ruaok nods
      • 2021-05-06 12656, 2021

      • _lucifer
        i put this on the web container directly and that'll be running a flask dev server. not uswgi or nginx could that be an issue?
      • 2021-05-06 12645, 2021

      • ruaok
        in theory if it speaks HTTP is should work.
      • 2021-05-06 12610, 2021

      • _lucifer
        i think alastairp mentioned above that nginx is needed for that.
      • 2021-05-06 12653, 2021

      • ruaok
      • 2021-05-06 12656, 2021

      • ruaok
        this is what we're using.
      • 2021-05-06 12642, 2021

      • _lucifer
        ok makes sense. alastairp, had such a setup on bono. it didn't work as expected so i wanted to try a usual dev instance.
      • 2021-05-06 12653, 2021

      • _lucifer
        i'll try again with his setup.
      • 2021-05-06 12605, 2021

      • ruaok
        this is that setup, methinks.
      • 2021-05-06 12612, 2021

      • ruaok
        I doubt we have two of them.
      • 2021-05-06 12601, 2021

      • ruaok
        do you need access to the MB database for this work
      • 2021-05-06 12602, 2021

      • ruaok
        ?
      • 2021-05-06 12624, 2021

      • ruaok
        if not, simply connected to a different docker network and no magic will happen.
      • 2021-05-06 12603, 2021

      • _lucifer
        i am not sure, i haven't worked that much with AB before.
      • 2021-05-06 12617, 2021

      • _lucifer
        let me try without MB db.
      • 2021-05-06 12609, 2021

      • _lucifer
        ruaok, i moved it to a different network but still it is trying to redirect to https.
      • 2021-05-06 12657, 2021

      • ruaok
        if you see the le container still spitting out messages, then you've not escaped it.
      • 2021-05-06 12649, 2021

      • _lucifer
        2021/05/06 16:46:27 Received event start for container 772bc81ffca8
      • 2021-05-06 12655, 2021

      • _lucifer
        yeah :(, this is my container
      • 2021-05-06 12637, 2021

      • _lucifer
        but i rechecked its on a different network
      • 2021-05-06 12644, 2021

      • nelgin
        yvanzo, Is it going to be fixed? Do I need to open a ticket to track it?
      • 2021-05-06 12655, 2021

      • _lucifer
        nelgin, you should probably open a ticket.
      • 2021-05-06 12633, 2021

      • Mr_Monkey
        _lucifer: I'm hitting my head on an issue with a test, and can't for the life of me understand why it's failing.
      • 2021-05-06 12633, 2021

      • Mr_Monkey
        If you've got some time, could you please have a look? It's on PR 1390, the error message here: https://github.com/metabrainz/listenbrainz-server…
      • 2021-05-06 12652, 2021

      • _lucifer
        sure
      • 2021-05-06 12602, 2021

      • Mr_Monkey
        Thank you :) I'm slowly going mad.
      • 2021-05-06 12648, 2021

      • BrainzGit
        [bookbrainz-site] dependabot[bot] opened pull request #623 (master…dependabot/npm_and_yarn/underscore-1.13.1): chore(deps): bump underscore from 1.10.2 to 1.13.1 https://github.com/bookbrainz/bookbrainz-site/pul…
      • 2021-05-06 12629, 2021

      • agatzk has quit
      • 2021-05-06 12630, 2021

      • agatzk joined the channel
      • 2021-05-06 12604, 2021

      • BrainzGit
        [listenbrainz-server] dependabot-preview[bot] opened pull request #1431 (master…dependabot/npm_and_yarn/ua-parser-js-0.7.28): [Security] Bump ua-parser-js from 0.7.21 to 0.7.28 https://github.com/metabrainz/listenbrainz-server…
      • 2021-05-06 12639, 2021

      • travis-ci joined the channel
      • 2021-05-06 12639, 2021

      • travis-ci
        Project bookbrainz-site build #3953: passed in 4 min 47 sec: https://travis-ci.org/bookbrainz/bookbrainz-site/…
      • 2021-05-06 12639, 2021

      • travis-ci has left the channel
      • 2021-05-06 12649, 2021

      • Mr_Monkey
        _lucifer: actually I finally figured out the issue, and fixing it now.
      • 2021-05-06 12624, 2021

      • _lucifer
        Mr_Monkey: oh nice! what was it?