#metabrainz

OK, I'll do a release now then!

fanks!

I'm heading to OfficeBrainz !

_lucifer: did you make all of the spotify reader changes that we discussed? Is it waiting on my review now?

alastairp, yes

great, thanks

building and pushing LB v-2021-03-24.0

ruaok: Wanna have some lunch with alastair and I?

alastairp, updated https://github.com/metabrainz/listenbrainz-serv...

going to release new version of all services except spotify reader - _lucifer, is that right?

yes

I just ate, Mr_Monkey. :-(

Are you two going back on a Wednesday at the office schedule?

I'm going to try and do wednesdays again

see you when we get back from lunch, then!

👍

> I just ate, Mr_Monkey

ruaok: https://news.ycombinator.com/item?id=18978251

Jan 23 2019: Travis will likely slow rot under Idera, looking at how its owned by a private equity firm.

right. let's check out github actions then.

ruaok: Mr_Monkey:

anything else we talked about?

AFAIK, GH is completely free for open source projects. no limits.

*GH ACtions

_lucifer: yeah, that's my understanding

dockerfile evolution? "if a linear build path for a dockerfile is not suited, make a new dockerfile" that stuff.

ah yeah

also, OTHER-375

OTHER-375: Use Jenkins to automate building and pushing docker images when tag is pushed https://tickets.metabrainz.org/browse/OTHER-375

> GitHub Actions usage is free for both public repositories and self-hosted runners. 

_lucifer: yes

ruaok: this is definitely in the wrong place anyway, so should be an easy win

_lucifer: yes! exactly

oh good, that would drastically improve my life.

we just had a good talk about questioning a bunch of our existing assumptions with our build/test/deploy stacks, which I think was really useful

indeed. and one of the thoughts I put on the table: if our lives could be made easier by paying some money to free up our own resources, that should be considered as a possibility.

its my discover weekly, which incidentally is listenable again. I turned of "keep playing music after this album finishes" and thus am picking much more of what i want to listen to. in turn DW works well -- at least this week.

[listenbrainz-server] amCap1712 opened pull request #1356 (master…validate-auth): LB-793: validate-token endpoint takes token as an argument instead of header https://github.com/metabrainz/listenbrainz-serv...

just noting that I'm pretty happy with Github actions for Picard. But requirements are likely a bit different, so not necessarily comparable

should there be a certain limit of free runtime minutes per organization we are probably already using a bit of that. But I'm not aware there is such a limit :D

thanks outsidecontext

yeah, I use it on another (closed) project. a free plan has 2000 minutes/month, and the lowest level paid plan has 3000 I think (and I believe our MeB non-profit plan is approx equal to this one). but as _lucifer said, theoretically it's free for open source projects, so we should try it

alastairp, should we proceed with the lb releases?

_lucifer: I'm just doing that right now

as we speak

nice :D

let me know if i can assist in some way

yeah, same here.

cron, timescale, web restarted

websockets, api compat, spark reader, labs api restarted

_lucifer: spark reader makes no noise when it starts up. is this something that we improved in a PR?

I thought I did, but I may not have committed that.

yes, i added a log statement before it attempts to connect to rabbitmq

but I don't see this log message: `current_app.logger.info('Spark consumer started!')`

which was already there

is `basic_qos` blocking call?

the process is running in the container (I can see it with ps)

it might just be an issue with logging

alastairp: it seems that a good pattern is to always include a top level `try: except Exception` block as the first thing in a thread's run() method.

ruaok: we've moved an apt-get install in the Dockerfile further up, so changes to crontab won't cause apt-get install node to happen again and again

great. is that in master?

you had one of them swallowing your exceptions?

I guess Mr_Monkey is opening a PR

multiple levels of swallowing.

in the future I will just put those in as a habit to save myself this frustration.

we've still got another issue, where according to dockerfile best-practises, items that change frequently should happen later in the file

so we have 1) runit scripts, 2) crontabs, 3) npm install and npm build

and one single execution chain. :(

to us, logic dictates that js code changes most frequently, and so should be put last

yeah

but do the install of js code change frequently? not so much.

we just came to the realisation that while all services use the same image, only web needs the js assets (most other images need the python assets)

does

well, it means that if you have to do a dev cycle of changing crontab/rebuild, the docker image cache will be invalidated and you'll have to rebuild the js files

remember that js installation is two steps - installation of dependencies, and then compilation of code

should we separate those two steps?

installation first, code last?

we were considering that

so we could do 1) runit scripts, 2) js dependencies, 3) crontabs, 4) build js files

it kind of feels like we're hyper-optimising your specific use-case though, and we're not sure if this happens frequently enough that it's worth the effort?

not sure, it's difficult to gauge

I dont think the order of 1-3 is all that important, but having 4 last makes sense to me.

that should not be that much work, right?

agreed, having 4 last makes sense to me, and this is easy to do

lets go for that and if it still is slow, we'll take another look.

👍

ruaok: ^ fyi, that's all I have to run our own docker registry server

or in non-docker-compose syntax: `docker run registry -v whatever -p whatever`

zas: do you remember why we chose to use docker hub instead of our own registry?

sometimes pushing to docker hub is freaking slooooooow.

yes, it is. I guess we didn't really made a choice, I tried to set up our own registry at some point for caching purpose. You can build locally and push to docker hub, I usually use autobuilds (I tag the git repo, it builds), but in all cases there are delays

I think it would be nice to have our own, but it would gobs of diskspace, I think.

now, setting up our own registry is likely possible, but as usual, it implies resources, redundancy, backups, upgrades (well, you know that), that's the usual reason we use external resources ;)

yeah.

github has a container registry too, no?

I think so

well, its something we should consider once we get past some of the infrastructure improvements we're currently making.

Is there something special about https://musicbrainz.org/release/eebd73dd-9c83-4... ?

When I view the release group https://musicbrainz.org/release-group/2e254f5a-... the release shows as greyed out.

Trying to load it into Picard yields an error.

Same with any other Digital Media releases for the band.

Okay. Thanks. I'll have to check my (recently installed) firewall settings. Thanks.

Yup. That was the problem. Sorry to bother y'all.

your firewall doesn't like anything to do with hippies

metadata hippies at that.

alastairp, is spark reader prod process up?

bitmap: yvanzo: reosarevok: does the CAA redirect have rate limits?

ruaok: afaik, only at the openresty level

ruaok, alastairp: is it fine if I send a couple of requests to and from spark to test spark reader?

_lucifer: `ps aux`

so 1req/sec/IP ?

shows

52 ? S 0:04 python3 -m listenbrainz.spark.spark_reader

ah my bad!