#metabrainz

<03metadataismydrug> I think LB might be down? 504 Gateway Timeout

aerozol: hi! what broke?

[listenbrainz-server] 14amCap1712 merged pull request #2793 (03master…fresh-releases-duration): Add a threshold to fresh releases calculation https://github.com/metabrainz/listenbrainz-serv...

[listenbrainz-server] release 03v-2024-02-23.0 has been published by 14github-actions[bot]: https://github.com/metabrainz/listenbrainz-serv...

lucifer: It might have already been broken, here’s the ticket: https://tickets.metabrainz.org/browse/LB-1526

LB-1526: Fresh Releases range selector doesn't work

aerozol: ah yes, that sounds like a frontend issue.

Gotcha, guess nobody noticed until now

aerozol: forgot if you get notified, but a quick look at MBS-13494 would be nice (no huge rush though since I won't implement it before the Event Art stuff is merged at the earliest)

MBS-13494: Allow batch-removing cover art https://tickets.metabrainz.org/browse/MBS-13494

Wow we now have a new picture on home screen. Looks sooo cool !!

bitmap: I tried to run upgrade.sh to test some of my schema change stuff, and I got "ERROR: DROP INDEX CONCURRENTLY cannot run inside a transaction block" which is part of your 20231005-edit-data-idx-link-type.sql

I will run it without that file for now, but probably something you need to amend

ansh: When I inspect a listen on my "Recent listens" page, the page's scrollbar disapppears and does not reappear once I close the inspect dialog. Is this a known issue?

I haven't seen this before, so it must have been introduced by an update which was deployed to production this week.

O’Moin

bitmap, lucifer, reosarevok: `test.mb.o` now has full search indexes and live indexing enabled 🙂

kellnerd: I can replicate. Maybe my fault, I'm the last one to have touched the modals

Looks like bootstrap is not removing a class that makes the body non-scrollable when modals are open

Hm, it does once in a while if I try again and again. I think I know where that's coming from.

yvanzo: sweet!

[mbsssss] 14yvanzo merged pull request #63 (03master…solr9): SEARCH-685 (Solr Schema): Support Solr 9 https://github.com/metabrainz/mbsssss/pull/63

What testing should we do? Just usual "add stuff and make sure it indexes it" is obvious but are there specific changes that should be tested in depth

[mmd-schema] 14yvanzo merged pull request #33 (03master…java17): SEARCH-714 (MMD Schema): Upgrade Java SE version to 17 https://github.com/metabrainz/mmd-schema/pull/33

[mb-solr] 14yvanzo merged pull request #54 (03master…solr9-new): SEARCH-714, SEARCH-685: Upgrade to Java 17 and Solr 9.4.0 https://github.com/metabrainz/mb-solr/pull/54

reosarevok: Solr 9.5.0 and GitHub Actions

[mb-solr] 14yvanzo opened pull request #55 (03master…github-action): Add GitHub action to test packaging MB Solr with Maven https://github.com/metabrainz/mb-solr/pull/55

reosarevok: Here is WIP for GitHub action ^

[musicbrainz-server] 14reosarevok opened pull request #3183 (03master…MBS-13421): MBS-13421: Add support for genre collections https://github.com/metabrainz/musicbrainz-serve...

yvanzo: so I tried to generate a new mmd-schema binding for ^

[ERROR] Failed to execute goal org.jvnet.jaxb:jaxb-maven-plugin:4.0.0:generate (default-cli) on project mmd2: Execution default-cli of goal org.jvnet.jaxb:jaxb-maven-plugin:4.0.0:generate failed: Unable to load the mojo 'generate' in the plugin 'org.jvnet.jaxb:jaxb-maven-plugin:4.0.0' due to an API incompatibility: org.codehaus.plexus.component.repository.exception.ComponentLookupException: org/jvnet/jaxb/maven/XJCMojo has been

compiled by a more recent version of the Java Runtime (class file version 55.0), this version of the Java Runtime only recognizes class file versions up to 52.0

I guess I need some setup steps?

[mb-solr] 14yvanzo opened pull request #56 (03master…solr-9.5.0): Upgrade Solr version to 9.5.0 https://github.com/metabrainz/mb-solr/pull/56

reosarevok: And the WIP for Solr 9.5.0 ^

reosarevok: Which commands did you use?

Just pulled master and did docker compose --project-directory=docker/brainz-mmd2-jaxb run --rm builder

Thank you monkey :)

Easy fix actually, just about to open a PR :)

(I wasn't sure whom to ping by the way, as you were marked as away and I was suspecting a connection to the SPA treatment, but apparently I was wrong 😇)

[listenbrainz-server] 14MonkeyDo merged pull request #2786 (03master…tags-auth-retry): TagsComponent: improve token refresh and error catching https://github.com/metabrainz/listenbrainz-serv...

[listenbrainz-server] 14MonkeyDo opened pull request #2794 (03master…fix-modal-backdrop): Modals: remove backdrop class manually when closing modals https://github.com/metabrainz/listenbrainz-serv...

[mmd-schema] 14yvanzo opened pull request #36 (03master…doc.md): Convert documentation text files to Markdown https://github.com/metabrainz/mmd-schema/pull/36

akshaaatt: could you please review https://github.com/metabrainz/listenbrainz-ios/... , whenever you get time :D

reosarevok: Never saw this error before, maybe something cached from a previous incomplete version?

Try rebuilding the image: `docker compose --project-directory=docker/brainz-mmd2-jaxb build --no-cache --pull`

If it still doesn’t work, try also removing the Maven cache: `sudo rm -f -r docker/brainz-mmd2-jaxb/maven_cache`

[listenbrainz-ios] 14akshaaatt merged pull request #18 (03main…fix-model): Model cleanup and UX enhancements https://github.com/metabrainz/listenbrainz-ios/...

[listenbrainz-server] 14MonkeyDo merged pull request #2794 (03master…fix-modal-backdrop): Modals: remove backdrop class manually when closing modals https://github.com/metabrainz/listenbrainz-serv...

ansh: Hello! I see some failing tests in LB#2731 which seem to be caused by a missing react-router context to wrap components in the front-end tests (looking at https://github.com/metabrainz/listenbrainz-serv...) according to https://stackoverflow.com/a/75728680

Migrate user dashboard and Explore pages to SPA: https://github.com/metabrainz/listenbrainz-serv...

I'm gonna deploy the PR to test.LB for further testing in the meantime

Thanks yvanzo, rebuilding the image was enough :)

[mmd-schema] 14reosarevok opened pull request #37 (03master…MBS-13421): MBS-13421: Add support for genre collections https://github.com/metabrainz/mmd-schema/pull/37

yvanzo: is there value in having mmd-schema and mbsssss separate from mb-solr? i feel we could merge all three in 1 repo.

lucifer, aerozol: For what it's worth, the extended history for the fresh releases makes the page much better for me too. I'm not 100% convinced we should use the entire history, because of what MetadataIsMyDrug said, but IMO we should definitely use at least 5-6 years (maybe more) of history. Maybe all of it if it's easy to ignore the "too much" aspect. In any case, much much better !

lucifer: yes, mmd-schema is also used by musicbrainz-server and sir, while mbsssss is deployed independently to SolrCloud.

yvanzo: ah okay, i guess we could still move the brainz-mmd-jaxb bindings and the generating script to mb-solr because sir and mb-server probably only need the relax-ng schema files.

they do

for mbsssss, i am not sure about the deployment details but if ansible automates it sufficiently its probably simpler to still have it in the solr repo.

we can probably talk about it after solr upgrade in prod is done with.

yes, brainz-mmd2-jaxb could be its own repository but that would probably not make a big difference

lucifer: Do you need any help with https://tickets.metabrainz.org/browse/LB-1106 ?

LB-1106: Improve LB's documentation about RabbitMQ

That was two years ago, but we discussed writing services doc again two weeks ago after the big maintenance week.

yvanzo: i meant more like moving brainz-mmd2-jaxb into mb-solr repository instead of a separate repo.

yes, sorry for the delay. i'll try to do the docs asap.

RabbitMQ is probably not the most critical, but then it’s probably the easiest to start with. :)

Here is what we have for SIR usage of RabbitMQ: https://sir.readthedocs.io/en/latest/service/in...

’s

So I've been looking around for project ideas for GSoC and wanted to ask - is there any interest in me doing a project to reduce the time it takes to process the data in the spark pipeline?

I've been thinking about rewriting the pipeline in Rust using something like https://docs.rs/differential-dataflow/

Which should allow much faster processing (being in Rust) and continuous streaming of new data and updates

Potentially even handling deletions without rerunning the pipeline

I saw that there's a ticket about it too - https://tickets.metabrainz.org/browse/LB-491

LB-491: Create performant rust/go based listen ingester

This is a good question for LB devs :)

lucifer, mayhem, monkey ^

For MB, there was a couple of Rust ideas in 2022: https://wiki.musicbrainz.org/Development/Summer...

Good news! I could fetch playlists form spotify! Now importing these playlists to LB is left :)

fun fact: there is a file with all spotify api services and all functions there except one is not used at all

yvanzo The thing that sparked this idea was https://wiki.musicbrainz.org/Development/Summer...

There is no Rust development at the moment in MeB but we would likely welcome contributors that could help bootstrapping its usage.

I was thinking about the application to make use of the API, and I though processing the data to see things like which tags are used together the most would be interesting

Yes, there is a ton of MB (new) background tasks that could be implemented in Rust.

JadedBlueEyes: i don't know the internals of differential-dataflow, but iiuc it seems like an in memory thing. not a distributed data processing platform which spark is.

I think the point of it is taking an input, processing it across a set of worker nodes incrementally, and then outputting that

Which is probably a subset of what spark does

Ok, I understand now, that’s correct that LB is doing some MB data processing but I don’t know the details.

Anyway the ideas are just indications, it’s not set in stone, so fresh propositions are very welcome.