<10theashikagupta> @monkey[m] hey are you a mentor of MetaBrainz?
pite has quit
Kladky joined the channel
monkey[m]
theashikagupta: yes I am :)
discordbrainz
<10theashikagupta> Hello Sir I am Ashika Gupta I want to contribute in MetaBrainz for Gsoc'25 I just need your guidance. Can I get your any social handle to connect with u ?
Hi all, the agenda item from yesterday was about upgrading to Docker Compose v2 in all MetaBrainz projects.
It is mostly used for development, except for MusicBrainz where it is also used for mirrors.
bitmap, reosarevok: Would it be OK to announce dropping support for Docker Compose v1 in mirrors in 3 months from now (Nov 18)?
mayhem[m]
ericd: can you please open a new PR with your work since the mid-term? I really need to start reviewing it. Thanks
monkey[m]
<yvanzo[m]> "Hi all, the agenda item from..." <- We have some custom code in LB and BB scripts to detect the docker compose version (use `docker-compose` or `docker compose`), but it would indeed be nice to drop that
monkey: Yes, indeed, it is mostly about replacing `docker-compose` with `docker compose` in scripts and docs, dropping `version` in compose files (but most projects did that already because of recently added warnings by docker compose), and maybe making use of improvements.
Also the word separator in the earliest versions of v1 was underscore _ whereas it is - in v2 (and in latest versions of v1).
reosarevok[m]
yvanzo: that seems sensible to me, especially since in most cases end users won't need to change anything I expect since they will be using our provided scripts and images
reosarevok[m] goes back to packing for some camping
reosarevok: It changes the installation dependencies for MB mirrors.
It isn’t much an issue since docker-compose-v2 is packaged in the latest Ubuntu LTS.
mayhem[m]
lucifer: yes, in about 2 hours I will be on a mainline train where I expect decent mobile coverage .
reosarevok[m]
yvanzo: sounds fine to me still then :)
derat[m]
reosarevok: sorry to interrupt the packing, but are you okay with https://github.com/metabrainz/musicbrainz-serve... doing an edit search for "view other users' notes", or would you prefer that it link to /edits/notes-received?
reosarevok[m]
I can live with either. I see you were discussing it with yvan and he can also live with either, so in that case what we have now works
derat[m]
cool, that was my assessment of the situation too -- thanks. :-) i'll ask for a merge on the PR
reosarevok[m]
I'm phone only until Friday so don't wait for me with things :)
discordbrainz
<10theashikagupta> Hello Sir I am Ashika Gupta I want to contribute in MetaBrainz for Gsoc'25 I just need your guidance. Can I get your any social handle to connect with u ?
mayhem[m]
Available for discussion now and for the next two hours.
lucifer[m]
mayhem: i prototyped replacing rabbitmq with redis for spark results processing. the peak memory usage can go upto ~48G in the full tests i ran. i am thinking about running a separate redis instance for this purpose and put a max memory policy of 64GB on it.
kiss has a lot of network usage peaks, anyone knows about those? can those be smoothed out somehow?
mayhem[m]
Sorry for the slow response...
That seems like a workable approach. Does the size of the data grow with the number of users?
Coverage is still quite spotty
zas[m]: Sounds like Lucifer doing testing. See above
lucifer: something to consider as well. Network saturation
Could we compress the data before we send it to redis?
airwaves[m] joined the channel
airwaves[m]
<mayhem[m]> "Available for discussion now and..." <- I'd like to contribute [to LB] in some way too, but I'm not entirely sure as to where to start -- preferably frontend stuff as I have quite a lot of JS experience lol
mayhem[m]
Hi and welcome!
We have a ticket category for easy first bugs that could be a starting point for you.
monkey and ansh: do the LB front end dev, so asking them is an even better way.
Also, if there is something that you don't like about our UI, fix that?
monkey[m]
Hi airwaves ! As mayhem said above, the easy option (and something I love doing myself) is finding something in the UI that you find annoying, and fixing it :P
That being said, I'm here if you would like more precise suggestions or need some help finding the relevant code or getting your environment set up
airwaves[m]
Link this Listen not autofocusing is a source of annoyance
* Link this Listen not autofocusing has been a source of annoyance
do you guys have like a local demo docker server I can point the front-end at?
lucifer[m]
[@zas666:matrix.org](https://matrix.to/#/@zas666:matrix.org) , [@mayhem:chatbrainz.org](https://matrix.to/#/@mayhem:chatbrainz.org): the data is same as what was sent through rabbitmq on clash and at the same rate.
<lucifer[m]> "[@zas666:matrix.org](https://..." <- Except in this case we were not consuming the data as it was being written, therefore the large memory use, right?
So real word usage would actually likely be much lower.
<airwaves[m]> "do you guys have like a local..." <- Our development setup is docker based and won't take long to setup to get started on this problem
Get it running, add a listen and you're good to go
lucifer[m]
[@mayhem:chatbrainz.org](https://matrix.to/#/@mayhem:chatbrainz.org) no we are using the data. But consuming it at a slower rate than what it is produced at.
So high memory usage.
Rabbitmq used to persist some part to the disk and keep some in memory.
I am not sure why the network io would be higher with redis than rabbitmq. Afaik neither do any compression.
client side at least
mayhem[m]
Could we slow the production of the data?
bitmap[m]
<yvanzo[m]> "bitmap, reosarevok: Would it..." <- makes sense to me if V1 is EOL. I'm not sure what the requirements are though. I tried installing docker-compose-v2 on hendrix yesterday in order to prebuild the musicbrainz-docker release, but couldn't due to package conflicts
yvanzo[m]
bitmap: Docker Compose v2 is already available on hendrix.
You don’t need the package docker-compose-v2 due to the way Docker is installed on our servers.
bitmap[m]
ah, I got "compose is not a command" on a few servers, maybe hendrix was not one of them
Jade[m]
<bitmap[m]> "the empty variables in the..." <- I looked into this a bit on Monday, by the way - sending the request directly works fine on the Rust side. There's the potential it was an old version of the image, so I've tagged a new version. Maybe try pulling the image again?
yvanzo[m]
bitmap: `docker compose version` works for me on `hendrix`atm.
bitmap[m]
yvanzo[m]: confirmed, a few servers it didn't work on which I'd tried before hendrix were aphex, hip, and patton
yvanzo[m]
bitmap: It seems that the package `docker-compose-plugin` might be missing on these servers.
bitmap[m]
(I did eventually get the prebuild to work on hendrix via DOCKER_COMPOSE_CMD='docker compose' ./admin/repository/prebuild-musicbrainz, so it's not waiting to be built)
initially I tried running the prebuild script on my MacBook, but some Perl packages failed to build
yvanzo[m]
bitmap: Thanks for yesterday’s MB release by the way!
bitmap[m]
it was about time I tried executing all the steps you improved recently :)
yvanzo[m]
It is very weird that it failed on macOS. Can you please provide details if you still have these?
Good to see this doc being actually used too.
monkey[m]
<airwaves[m]> "Link this Listen not autofocusin..." <- Very much agree! So much in fact that I already opened (and merged) a PR for it :D LB#2948
It isn't deployed at the moment, but let me see if I can deploy it
[listenbrainz-server] 14MonkeyDo merged pull request #2959 (03master…LB-1254): LB-1254: Graph for month view in statistics may cut of days at the end due to varying length of months https://github.com/metabrainz/listenbrainz-serv...
minimal joined the channel
airwaves[m]
<monkey[m]> "Very much agree! So much in fact..." <- > <@monkey:chatbrainz.org> Very much agree! So much in fact that I already opened (and merged) a PR for it :D LB#2948
> It isn't deployed at the moment, but let me see if I can deploy it
<yvanzo[m]> "It is very weird that it..." <- `Date::Calc` and one other module failed to build, though I didn't have access to the internal build.log files which would have contained the compilation errors. now I wonder if the base image is multi-arch and it was performing the build using an arm64 image? (as it's an M2 Mac)
airwaves[m]
wow, that was quick monkey :D
monkey[m]
airwaves[m]: Yes, the pull request description is from before I realized and fixed that too.
Still need to build the image and deploy it, but should be pretty auick too
quick*
pite joined the channel
monkey[m] is updating LB prod
airwaves[m]
monkey[m]: man fair play, you guys work fast :D
monkey[m]
OK, it's up airwaves if you want to test it. If you find more improvements, please don't hesitate :)
airwaves[m]
sure will haha, looking forward to it :D
may as well sort through missing data now lol
monkey[m]
For example, the link listen modal will autofocus when you click the copy text button only, which might still be annoying for some
Oh, you'll like this too !
bitmap[m]
<Jade[m]> "I looked into this a bit on..." <- indeed, restarting the container worked 👍 thanks!
bitmap: It might not be multi-architecture indeed, but there should be a way to cross compile from M2.
bitmap[m]
yeah, there is a way to force the platform in the compose file IIRC. but I guess the version of the Linux kernel it's emulating on macOS may differ still, which could cause other reproducibility issues...
yvanzo[m]
bitmap: Multi-platform support is a feature of Docker Compose v2.
bitmap, reosarevok: I drafted a blog post for requiring Docker Compose v2 for mirrors. Actually, I used the template for announcing schema changes. Should we make a small schema change at the same time?
We can discuss this next week when reosarevok is back 😎
bitmap[m]
ok, let's wait for Reo :) there are a few minor changes we could include IIRC
yvanzo: BTW, on several [MB Docker releases](https://github.com/metabrainz/musicbrainz-docker/releases) prior to the latest one, it says:
> Assuming your checked out version of musicbrainz-docker is v-2024-05-13-mbdb29-pg16, then just run:
But I assume this should actually say "v-2024-05-13-mbdb29-pg16 or later"? (I added "or later" to the latest release.)
<mayhem[m]> "ericd: can you please open a new..." <- will do
yvanzo[m]
bitmap: Yes, _or later_ has been omitted for the release after the schema change and never came back.
bitmap[m]
gotcha, just wanted to make sure I was correct in adding that back
ericd[m]
Hi aerozol, I wonder where I can find the feed icon and icon for the copy button in your mock up? Last week I just use some text button for testing purpose.
<BrainzGit> "[listenbrainz-server] ericd23..." <- mayhem: ta-da. and, is the new mb oauth up?
mayhem[m]
<ericd[m]> "mayhem: ta-da. and, is the new..." <- Not yet and it won't be before the coding period is done. We should get it finished ready to merge, but we'll need to disable the feature for now. Once the oauth stuff is done, I'll take care of deploying it.
lucifer[m]
mayhem: not easy to slow down production without blocking the cluster but how about we host the new redis instance on michael?
eventually, we'll hopefully have incremental stats and that would considerably reduce traffic for old time ranges but it will take lot of time to get there.
monkey: do you want to make any changes to the OAuth related screens i shared the other day?
mayhem[m]
lucifer[m]: Tricky. Because if our users double or triple, this system breaks. Not very future proof.
airwaves[m]
<monkey[m]> "ariwaves: Now a bit easier to..." <- oh man, that should make filling that in x10 easier lol
mayhem[m]
Why is writing to couchdb so slow?
What binds it? Disk or CPU?
lucifer[m]
i am not sure.
but we are following all of their recommended steps.
aerozol[m]
reosarevok: Should I open a ticket to add ‘promoter
...’ to label types?
aerozol[m] uploaded an image: (26KiB) < https://matrix.chatbrainz.org/_matrix/media/v3/download/matrix.org/WgOoLruCdFaFkhaGYFJuqntD/image.png >
p.s. I wonder if atj is familiar with that promoters events
We have “Relationship types / Event-Label / Presented” just wondering if it’s worth adding more types or nah
allen
Hi there, can I pm someone from security@org with a question regarding one api endpoint?
