does anyone know what a real world size limit of an RMQ message is? iliekcomputers, I am thinking about the data returned from the user similarity algorithm... can we return everything in one giant chunk of data?
2021-02-26 05713, 2021
_lucifer
the default limit imposed by rabbitmq is 128Mib but not sure if you were asking about this
2021-02-26 05701, 2021
ruaok
that mostly answers my question... now we know the default limit, what is sensical?
2021-02-26 05728, 2021
ruaok
one message will suffice for now, I think. we don't have that many users.
2021-02-26 05710, 2021
roger_that joined the channel
2021-02-26 05741, 2021
roger_that
yvanzo: what do you guys have set for your indexer limit? What are the implications of not processing certain messages?
_lucifer: are you following along in ghe planning doc?
2021-02-26 05736, 2021
ruaok
I've written more on the second page, have a look when you can.
2021-02-26 05752, 2021
_lucifer
yes, i am. will do.
2021-02-26 05758, 2021
ruaok
thx
2021-02-26 05724, 2021
ruaok
alastairp: _lucifer iliekcomputers shivam-kapila : I've updated the LB social features doc with what I understand to be the current state of things. Please edit/leave comments for anything that isn't quite right.
2021-02-26 05718, 2021
iliekcomputers
left a few comments
2021-02-26 05726, 2021
ruaok
thanks, integrated those already.
2021-02-26 05759, 2021
ruaok
it currently doesn't feel like we have enough work to keep all of us busy for an entire week, but I am sure this task will guppy up.
2021-02-26 05726, 2021
ruaok
I could take on the troi-bot posting as a stretch goal.
2021-02-26 05709, 2021
ruaok
hmm. the feature to recommend a track -- is that part of the timeline or is that separate?
2021-02-26 05743, 2021
iliekcomputers
i don't think it was, originally.
2021-02-26 05754, 2021
ruaok
ok, adding a new section.
2021-02-26 05736, 2021
iliekcomputers
oh btw
2021-02-26 05755, 2021
_lucifer
alastairp: should we work on lb consul upgrade today?
2021-02-26 05741, 2021
alastairp
yes, that'd be a great idea
2021-02-26 05725, 2021
alastairp
I'm in a meeting for 1-1.5 hours more, but I'm kind of available here from time to time if I'm not paying too much attention to the meeting
2021-02-26 05734, 2021
alastairp
let me open the PRs
2021-02-26 05711, 2021
d4rkie has quit
2021-02-26 05750, 2021
Nyanko-sensei joined the channel
2021-02-26 05715, 2021
moufl has quit
2021-02-26 05719, 2021
Darkloke joined the channel
2021-02-26 05756, 2021
moufl joined the channel
2021-02-26 05702, 2021
Darkloke has quit
2021-02-26 05754, 2021
reosarevok
legoktm: should we be using the Wikimedia REST API for getting the extracts rather than the PHP API?
_lucifer: can you look at each of these sections and double-check that it appears that 1) each config block acts on a single directory in /etc/service/, 2) each block takes files from a single directory in ./docker/services/, and 3) each block adds a `down` file to /etc/service/[service]
2021-02-26 05744, 2021
_lucifer
yes, sure.
2021-02-26 05707, 2021
shivam-kapila
Good evening. ruaok: I have added some comments to the planning doc. overall it looks fine
2021-02-26 05722, 2021
sumedh has quit
2021-02-26 05710, 2021
_lucifer
alastairp, one nitpick, follow_dispatcher service is now called websockets. we should probably rename that here as well.
2021-02-26 05721, 2021
alastairp
good catch, please update it
2021-02-26 05758, 2021
_lucifer
👍
2021-02-26 05736, 2021
alastairp
this has already been updated in docker-server-configs. The container name is now `listenbrainz-websockets-$DEPLOY_ENV`
I made this quick script to put things into consul for quick testing, I wasn't sure if there is a better tool for using, but this is fine for me for now
2021-02-26 05707, 2021
alastairp
I installed consul, and in a venv python-consul
2021-02-26 05743, 2021
alastairp
I start consul with `consul agent -dev`
2021-02-26 05700, 2021
alastairp
and then I put some stuff into it with `python publish_consul.py LB/config.prod.json consul/LB/config.prod.json` (from in the docker-server-configs checkout)
2021-02-26 05709, 2021
alastairp
what OS/docker stack are you running?
2021-02-26 05729, 2021
_lucifer
Ubuntu 20.04
2021-02-26 05739, 2021
alastairp
ah, because I have `CONSUL_HOST: host.docker.internal` in docker-compose.yml
2021-02-26 05700, 2021
alastairp
so that I can run consul on my local machine, however this dns name only works on mac/windows
2021-02-26 05714, 2021
alastairp
we might have to run the consul server in docker-compose
it has `runit` installed. this manages the startup scripts that we put in /etc/service
2021-02-26 05748, 2021
alastairp
when a docker container starts, it first executes /etc/rc.local (this is why we put stuff here), and then it looks in /etc/service for any directory _without_ a `down` file, and executes its `run` file
this connects to a consul server (think of consul as a key/value store for application configuration, it has some other magic that isn't necessary to know now)
2021-02-26 05743, 2021
alastairp
consul-template takes a config file (e.g. https://github.com/metabrainz/listenbrainz-server…) performs susbstitution on the template, generating our config.py, and then runs the script in the `exec {}` block
2021-02-26 05726, 2021
alastairp
so on MeB servers there's a centralised consul server, it loads its config magically from the contents of the consul directory in docker-server-configs
2021-02-26 05731, 2021
alastairp
so if we want to run the production server workflow locally, we need to start up a consul server, populate it with relevant config information, and then make the `webserver` container connect to it (with the CONSUL_HOST env variable)
2021-02-26 05714, 2021
_lucifer
that clears it up. thanks for the clear explanation :D
2021-02-26 05718, 2021
_lucifer
one question though where does git2consul come into play/
2021-02-26 05728, 2021
alastairp
git2consul is the magic that reads those files and pushes them into consul
2021-02-26 05718, 2021
_lucifer
ok, makes sense.
2021-02-26 05705, 2021
_lucifer
now to run consul server locally, i should add the consulagent and git2consul service to my docker-compose.yml and modify CONSUL_HOST accordingly?
2021-02-26 05700, 2021
alastairp
I _think so_. I've not done this before sorry
2021-02-26 05758, 2021
_lucifer
no worries. let me try this.
2021-02-26 05724, 2021
BrainzGit
[musicbrainz-server] yvanzo opened pull request #1944 (master…mbs-10416-cntrl-chars): MBS-10416: Remove invalid characters from newly entered annotations https://github.com/metabrainz/musicbrainz-server/…
