#metabrainz

ruaok: Ig matchable should be only used for look up (join), but for getting distinct etc we should use mbids or some id which we know is distinct.

ruaok: we can fetch all the top artists listen to by the user in the last week, the user most probably likes these artists, we can then fetch the top artists similar to these artists and that will be the candidate set for a user.

ruaok: where have you copied the recs? I forgot :( I just see mapping dump in my account on new leader.

iliekcomputers: can you please have a look at #1115? I see you have requested changes in there. Let's merge it once you approve.

>pristine___: on lemmy in /home/pristine is recommendation.cf_recording.sql.bz2

pristine___: probably this ^

Whoops, I was checking on leader.

Thanks

Morning

[listenbrainz-server] ishaanshah opened pull request #1152 (master…restart-cron-jobs): Re-enable cronjob for importing incremental dumps everyday https://github.com/metabrainz/listenbrainz-serv...

[musicbrainz-server] reosarevok merged pull request #1752 (master…MBS-11186): MBS-11186: Add missing space between class names https://github.com/metabrainz/musicbrainz-serve...

MBS-11186: Inconsistent username font-weight for edit owner https://tickets.metabrainz.org/browse/MBS-11186

[musicbrainz-server] reosarevok merged pull request #1749 (master…MBS-11177): MBS-11177: Drop "Description" label from type_bubble https://github.com/metabrainz/musicbrainz-serve...

MBS-11177: Improve the display of entity’s type attribute’s description https://tickets.metabrainz.org/browse/MBS-11177

shivam-kapila: 🎉, which one?

[musicbrainz-server] reosarevok opened pull request #1763 (master…MBS-11200): MBS-11200: Add works to VGMdb autocleanup https://github.com/metabrainz/musicbrainz-serve...

MBS-11200: Add works to VGMdb autocleanup https://tickets.metabrainz.org/browse/MBS-11200

iliekcomputers: thanks for the approval. I will merge it :)

👍🏽

[listenbrainz-server] vansika merged pull request #1115 (master…move-mapping-utils): Move pre-mapping code to utils https://github.com/metabrainz/listenbrainz-serv...

[musicbrainz-server] reosarevok opened pull request #1764 (master…MBS-11199): MBS-11199: Remove superfluous 7digital parameters https://github.com/metabrainz/musicbrainz-serve...

MBS-11199: Update 7digital.com URL cleanup https://tickets.metabrainz.org/browse/MBS-11199

mo’’in’

moin!

Recommendations dump is on ftp now.

[listenbrainz-server] vansika opened pull request #1153 (master…dataframe-utils): Utils for shareable dataframe functions https://github.com/metabrainz/listenbrainz-serv...

pristine___: https://github.com/metabrainz/listenbrainz-serv... the SQL query here is heinous. What requires it to be so complex?

We are first fetching all the recording mbids associated with a user. From those recording mbids we are looking for the ones which are supplied as an argument, the recording_list

why is it more complex than `select * from recommendation_feedback where user_id = u and recording_mbid in (...)` ?

Yeah, I used IN operator but there was an error, which seemed to me like a list/array issue.

Thanks for the feedback will try to simplify :)

yes, lets debug the IN issue -- this query is asking for trouble -- we should make it as easy as possible, which is likely to be the fastest way.

reconstruct the query with in and then post the error, we'll sort it out.

Trouble like?

[musicbrainz-server] reosarevok merged pull request #1755 (master…MBS-11192): MBS-11192: Add voting-icon for Approve https://github.com/metabrainz/musicbrainz-serve...

MBS-11192: Add voting-icon for Approved https://tickets.metabrainz.org/browse/MBS-11192

Will do after lunch :)

[critiquebrainz] alastair merged pull request #315 (master…CB-383): CB-383: Move tests to separate packages https://github.com/metabrainz/critiquebrainz/pu...

CB-383: ws views test isn't in separate test package https://tickets.metabrainz.org/browse/CB-383

_lucifer: hi, were you going to add tests to https://github.com/metabrainz/critiquebrainz/pu... and https://github.com/metabrainz/critiquebrainz/pu..., and determine the state of https://github.com/metabrainz/critiquebrainz/pu... ?

yes, alastairp it is in the works. i expect to complete all three by this sundayy

cool, just checking :) thanks

do you want me to give a quick review to these items in their current state as well, or should I wait until you complete them?

if possible then sure please review these as well. i'll then incorporate those changes as well.

before you add tests?

sure

yes!

perfect

I'm just looking at abhinavohri's ones too, they look mostly good

nice! i had also reviewed them once. seemed to be in order to me as well.

ok

I guess the ones with the new entities is mostly copying existing code

yeah right

what was the final result of the investigation into the no language codes?

does #314 fix it?

ruaok, bitmap, yvanzo: didn't https://github.com/metabrainz/sir/pull/65 add batching to the reindex code? if https://github.com/metabrainz/musicbrainz-docke... are the defaults for replication - the batch size of 100 seems very small for use cases of for example daily updates of a mirror

pybabel does not support the no code. but i think the issue is that the current code was mixing pycountry with pybabel

did you find the issue of the exception when visiting some user profile pages?

yes it does

I think it was https://sentry.metabrainz.org/metabrainz/cb/iss...

yes, I think you're right. mixing the difference between language and countries could be problematic

yes it happen when the review of an entity is tried to opened

I'll double check your work

its an issue to be fixed in brainzutils i suppose

*merged entity

oh, interesting

do you understand the reason for it?

if so, could you open a ticket, link the sentry issue, and explain what you understand about the issue?

no not yet, i was thinking to clone and modify brainzutils it to add some logging and use it to test locally

sure i will

Mineo: ah, thanks for that link -- I hadn't looked at the RMQ side of things yet.

I see a 'live_index_batch_size' for batching live index stuff. clear.

is there a batch size setting that is used for replicated instances as well?

:D

some love for LB. :)

alastairp: remember you posted a link about importlib.util.spec_from_file_location yesterday?

it doesn't return a module that contains all the classes -- just about everything else.

I've spent over an hour on it trying to improve it, but it doesn't work. the sys.path approach does, so I'll go with that.

hah, right. sorry about that wild goose chase then

agreed, changing the path should be good enough then

all good. I'll work on getting the existing tests working again.

Thanks Mineo, we should probably change this value for https://github.com/metabrainz/musicbrainz-docke...

is there a version of this for non-live indexing? do non-live rmq messages get batched? if not, this may be a good way to improve things.

Hi zas: Just created MBH-545. More generally, I would like to set it up for a musicbrainz-docker instance too.

MBH-545: Add stats for musicbrainz-solr-test and sir-test containers https://tickets.metabrainz.org/browse/MBH-545

ruaok: rmq is used for live indexing only.

yvanzo: solr-test and sir-test mean the ones used by test.mb?

_lucifer: yes

bitmap, reosarevok: gh:MBS#1518 is deployed on test.mb.o

MBS-10829: Fix SOLR syntax for recording with no length: https://github.com/metabrainz/musicbrainz-serve...

I suspect that test will only help a bit and we will only get proper feedback from at the earliest beta, but

yvanzo: ok I'll have a look. Does musicbrainz-docker instance contain solr too? For collecting metrics using telegraf access to port 8983 is needed (of course exposed port can be different, that's the internal solr one). Perhaps there's some access control too to take care of

reosarevok: feel free to edit the test db directly to test anything you want to.

zas: yes, it would be nice to use a different port. that would be for use on a dedicated cloud node.

ruaok, zas: about creating the musicbrainz-docker test node, we need 160GB disk space, would a dedicated CCX21 instance be ok?

ccx21 is expensive, does it need dedicated vcpu? CX41 could do I think

how much memory does it need?

16GB

the purpose is to test server load, if CX41 works for that purpose, go for it

Imho we can try CX41 (4 non-dedicated vCPUs, 16gb ram, 160gb storage) for half the price of CCX21, at worse, we'll upgrade

works for me :)

(server load when reindexing)

hi zas, I have a similar request to yvanzo, to add stats to granfa, remember our discussion from some months back about reading json stats over http from listenbrainz?

if you have some time either this week or early next week could we test this?

sure, we release a new version of Picard today, what about tomorrow around same time?

perfect, thank you

is there a repo with telegraf configuration that I can look at myself? (I had a look but didn't find anything obvious)

iliekcomputers: hi, I guess you won't be around for LB this time tomorrow, but if you have time after work today we could release my LB PR on test so that stats are ready for capturing tomorrow

no permission :)

I'll add you

telegraf-services, yes

alastairp: I can deploy your branch to beta in the evening, sure.

iliekcomputers: I can do the same ;) wasn't sure if you wanted to be around for it

Oh!

No worries, go ahead.

yvanzo: when do you need this VM?

I'd Just rebase over master once, to make sure nothing gets reverted unintentionally, but even that is optional.

sure thing

Thanks!

zas: can use it from today

zas: cause it is monthly cost not 30days?

yes; in fact the cost is per used hour, but they give prices for one month usage

of course if you don't need the machine all the time, we can take a more expensive one

but you have to take care to shut it down when unused

I would prefer that then

Since it's only to test CPU intensive tasks.

pristine___: it seems that there has been a regression -- there are only 200 CF tracks available.

ruaok: yeah, 200 only for you. I tested received for you and a user yesterday. Didn't do --top=1000. My bad

Recs*