#metabrainz

<11xmemphistox> If I host MusicBrainz in docker, can I specify where to put the actual DB? IE) a mount point to a nas?

yvanzo: bitmap hi, I have refractored code for polling and notifying and made it a bit more simpler, and fixed a couple of mistakes.

Can you have a look at `poller` and `archival` modules ?

Hi yellowhatpro, I’ve read the updated description, it seems to be promising so far.

There should be more tests for first index, for example when is_saved is set to true.

(edit_data is being handled by postgres trigger) seems to be an outdated comment.

extract_last_rows_idx_from_internet_archive_table seems to assume that there will always be a row with a from_table matching every source table (edit_data and edit_note in the current implementation). How is that guaranteed?

Aren’t these rows deleted eventually?

(Not sure that part is implemented though.)

Also, if a edit_data row contains a URL that is already in the URL queue, the edit_data row won’t be mentioned in the URL queue.

[musicbrainz-server] 14reosarevok merged pull request #3309 (03master…MBS-13582): MBS-13582: Add statistics for genre collections https://github.com/metabrainz/musicbrainz-serve...

mayhem: added comments to pr. Feels so fresh to read python code after a long time xD

yvanzo: remind me, if I wanted to start collecting the new stats and make sure they work fine during beta, should I be updating the cron container? And is that a bad idea?

reosarevok: That’s what I’ve been doing so far. Making patch for the production branch and deploying it directly in the cron container.

Making a patch as in? Cherry-pick them into prod?

No, into a local branch.

based on production

So make an mbs branch cron-stats-2024-06 off prod, then cherry-picking those, then what, logging into the container and checking that out? I haven't changed container branches before, just used the update scripts :)

https://github.com/metabrainz/musicbrainz-serve... is too long a line btw, circleci complained, please amend again :)

I just copied the changed files, there is no git into the containers.

Oopsie

So then what's the point of having a local branch to begin with? :D Shouldn't I just copy the changed files and that's it? 😕

Should be fixed now.

(sorry if I'm being really dumb, docker + not that much sleep 😅)

Even such tiny changes are worth a PR ^^

It's a full export day so I guess I should be waiting quite a bit anyway

yvanzo[m]: Heh. Always easy to forget!

reosarevok: Making a local branch is for basing your patch on production.

Very often files can be modified by more than one commit or PR.

Yes, have to wait for full export to complete anyway.

reosarevok: Updated the cron container and calculated the missing statistics (eaa/genre collections) for beta.

Oh thanks!

Seems like they work :)

[musicbrainz-server] 14yvanzo opened pull request #3310 (03master…admin-run-logs): Make admin/run log the (full) command it ran https://github.com/metabrainz/musicbrainz-serve...

<yvanzo[m]> "There should be more tests for..." <- Yeah right, I will keep adding more ✅

<yvanzo[m]> "extract_last_rows_idx_from_inte..." <- I think I should rename the function to be `fetch_edit_data_and_note_start_id` or something.

https://github.com/yellowHatpro/mb-exurl-ia-ser... : In this function, if there is nothing in the `internet archive urls` table, we will fetch the latest rows from edit data and edit columns.

I think I can improve the logic btw, there is still a problem I can spot.... (full message at <https://matrix.chatbrainz.org/_matrix/media/v3/...>)

<yvanzo[m]> "Aren’t these rows deleted..." <- Yeah we will eventually have a task which will periodically run over `internet_archive_urls` table, and clean the rows already saved, and retry the URLs which could not be saved

<yvanzo[m]> "Also, if a edit_data row..." <- Ummmm, I didn't get this one, do you mean I should take care of duplicates?

What happens if the same URL (example.com) is in two different edit notes?

Yes

I do have a check for this:... (full message at <https://matrix.chatbrainz.org/_matrix/media/v3/...>)

There is a lot of potential for code factorization about edit_data and edit_note.

It would also make it easier to handle more tables in the future.

Oh, I will make it more modular then

yellowhatpro[m]: > <@yellowhatpro:matrix.org> I do have a check for this:... (full message at <https://matrix.chatbrainz.org/_matrix/media/v3/...>)

Rental car 😆