#metabrainz

/

      • ephemer0l_ has quit
      • 2020-10-22 29659, 2020

      • CatQuest has quit
      • 2020-10-22 29647, 2020

      • pristine___
        ruaok: Ig matchable should be only used for look up (join), but for getting distinct etc we should use mbids or some id which we know is distinct.
      • 2020-10-22 29619, 2020

      • CatQuest joined the channel
      • 2020-10-22 29633, 2020

      • ephemer0l_ joined the channel
      • 2020-10-22 29638, 2020

      • pristine___
        ruaok: we can fetch all the top artists listen to by the user in the last week, the user most probably likes these artists, we can then fetch the top artists similar to these artists and that will be the candidate set for a user.
      • 2020-10-22 29634, 2020

      • pristine___
        ruaok: where have you copied the recs? I forgot :( I just see mapping dump in my account on new leader.
      • 2020-10-22 29618, 2020

      • pristine___
        iliekcomputers: can you please have a look at #1115? I see you have requested changes in there. Let's merge it once you approve.
      • 2020-10-22 29627, 2020

      • D4RK-PH0ENiX has quit
      • 2020-10-22 29610, 2020

      • D4RK-PH0ENiX joined the channel
      • 2020-10-22 29627, 2020

      • _lucifer
        >pristine___: on lemmy in /home/pristine is recommendation.cf_recording.sql.bz2
      • 2020-10-22 29654, 2020

      • _lucifer
        pristine___: probably this ^
      • 2020-10-22 29607, 2020

      • pristine___
        Whoops, I was checking on leader.
      • 2020-10-22 29610, 2020

      • pristine___
        Thanks
      • 2020-10-22 29641, 2020

      • shivam-kapila
        Morning
      • 2020-10-22 29607, 2020

      • shivam-kapila got a new laptop finally
      • 2020-10-22 29642, 2020

      • BrainzGit
        [listenbrainz-server] ishaanshah opened pull request #1152 (master…restart-cron-jobs): Re-enable cronjob for importing incremental dumps everyday https://github.com/metabrainz/listenbrainz-server…
      • 2020-10-22 29630, 2020

      • BrainzGit
        [musicbrainz-server] reosarevok merged pull request #1752 (master…MBS-11186): MBS-11186: Add missing space between class names https://github.com/metabrainz/musicbrainz-server/…
      • 2020-10-22 29632, 2020

      • BrainzBot
        MBS-11186: Inconsistent username font-weight for edit owner https://tickets.metabrainz.org/browse/MBS-11186
      • 2020-10-22 29652, 2020

      • BrainzGit
        [musicbrainz-server] reosarevok merged pull request #1749 (master…MBS-11177): MBS-11177: Drop "Description" label from type_bubble https://github.com/metabrainz/musicbrainz-server/…
      • 2020-10-22 29653, 2020

      • BrainzBot
        MBS-11177: Improve the display of entity’s type attribute’s description https://tickets.metabrainz.org/browse/MBS-11177
      • 2020-10-22 29627, 2020

      • _lucifer
        shivam-kapila: 🎉, which one?
      • 2020-10-22 29626, 2020

      • BrainzGit
        [musicbrainz-server] reosarevok opened pull request #1763 (master…MBS-11200): MBS-11200: Add works to VGMdb autocleanup https://github.com/metabrainz/musicbrainz-server/…
      • 2020-10-22 29628, 2020

      • BrainzBot
        MBS-11200: Add works to VGMdb autocleanup https://tickets.metabrainz.org/browse/MBS-11200
      • 2020-10-22 29600, 2020

      • pristine___
        iliekcomputers: thanks for the approval. I will merge it :)
      • 2020-10-22 29624, 2020

      • iliekcomputers
        👍🏽
      • 2020-10-22 29645, 2020

      • BrainzGit
        [listenbrainz-server] vansika merged pull request #1115 (master…move-mapping-utils): Move pre-mapping code to utils https://github.com/metabrainz/listenbrainz-server…
      • 2020-10-22 29628, 2020

      • BrainzGit
        [musicbrainz-server] reosarevok opened pull request #1764 (master…MBS-11199): MBS-11199: Remove superfluous 7digital parameters https://github.com/metabrainz/musicbrainz-server/…
      • 2020-10-22 29629, 2020

      • BrainzBot
      • 2020-10-22 29614, 2020

      • v6lur joined the channel
      • 2020-10-22 29602, 2020

      • yvanzo
        mo’’in’
      • 2020-10-22 29620, 2020

      • apetresc has quit
      • 2020-10-22 29657, 2020

      • apetresc joined the channel
      • 2020-10-22 29653, 2020

      • reosarevok
        moin!
      • 2020-10-22 29610, 2020

      • nelgin has quit
      • 2020-10-22 29628, 2020

      • nelgin joined the channel
      • 2020-10-22 29643, 2020

      • CatQuest has quit
      • 2020-10-22 29603, 2020

      • CatQuest joined the channel
      • 2020-10-22 29600, 2020

      • pristine___
      • 2020-10-22 29620, 2020

      • pristine___
        Recommendations dump is on ftp now.
      • 2020-10-22 29601, 2020

      • Gazooo79494 has quit
      • 2020-10-22 29607, 2020

      • BrainzGit
        [listenbrainz-server] vansika opened pull request #1153 (master…dataframe-utils): Utils for shareable dataframe functions https://github.com/metabrainz/listenbrainz-server…
      • 2020-10-22 29648, 2020

      • Gazooo79494 joined the channel
      • 2020-10-22 29642, 2020

      • ruaok
        pristine___: https://github.com/metabrainz/listenbrainz-server… the SQL query here is heinous. What requires it to be so complex?
      • 2020-10-22 29621, 2020

      • pristine___
        We are first fetching all the recording mbids associated with a user. From those recording mbids we are looking for the ones which are supplied as an argument, the recording_list
      • 2020-10-22 29656, 2020

      • ruaok
        why is it more complex than `select * from recommendation_feedback where user_id = u and recording_mbid in (...)` ?
      • 2020-10-22 29623, 2020

      • pristine___
        Yeah, I used IN operator but there was an error, which seemed to me like a list/array issue.
      • 2020-10-22 29638, 2020

      • pristine___
        Thanks for the feedback will try to simplify :)
      • 2020-10-22 29616, 2020

      • ruaok
        yes, lets debug the IN issue -- this query is asking for trouble -- we should make it as easy as possible, which is likely to be the fastest way.
      • 2020-10-22 29632, 2020

      • ruaok
        reconstruct the query with in and then post the error, we'll sort it out.
      • 2020-10-22 29640, 2020

      • pristine___
        Trouble like?
      • 2020-10-22 29600, 2020

      • BrainzGit
        [musicbrainz-server] reosarevok merged pull request #1755 (master…MBS-11192): MBS-11192: Add voting-icon for Approve https://github.com/metabrainz/musicbrainz-server/…
      • 2020-10-22 29601, 2020

      • BrainzBot
        MBS-11192: Add voting-icon for Approved https://tickets.metabrainz.org/browse/MBS-11192
      • 2020-10-22 29654, 2020

      • pristine___
        Will do after lunch :)
      • 2020-10-22 29631, 2020

      • BrainzGit
        [critiquebrainz] alastair merged pull request #315 (master…CB-383): CB-383: Move tests to separate packages https://github.com/metabrainz/critiquebrainz/pull…
      • 2020-10-22 29632, 2020

      • BrainzBot
        CB-383: ws views test isn't in separate test package https://tickets.metabrainz.org/browse/CB-383
      • 2020-10-22 29654, 2020

      • alastairp
      • 2020-10-22 29642, 2020

      • _lucifer
        yes, alastairp it is in the works. i expect to complete all three by this sundayy
      • 2020-10-22 29602, 2020

      • alastairp
        cool, just checking :) thanks
      • 2020-10-22 29640, 2020

      • alastairp
        do you want me to give a quick review to these items in their current state as well, or should I wait until you complete them?
      • 2020-10-22 29637, 2020

      • _lucifer
        if possible then sure please review these as well. i'll then incorporate those changes as well.
      • 2020-10-22 29615, 2020

      • alastairp
        before you add tests?
      • 2020-10-22 29619, 2020

      • alastairp
        sure
      • 2020-10-22 29623, 2020

      • _lucifer
        yes!
      • 2020-10-22 29628, 2020

      • alastairp
        perfect
      • 2020-10-22 29633, 2020

      • alastairp
        I'm just looking at abhinavohri's ones too, they look mostly good
      • 2020-10-22 29652, 2020

      • _lucifer
        nice! i had also reviewed them once. seemed to be in order to me as well.
      • 2020-10-22 29659, 2020

      • alastairp
        ok
      • 2020-10-22 29613, 2020

      • alastairp
        I guess the ones with the new entities is mostly copying existing code
      • 2020-10-22 29627, 2020

      • _lucifer
        yeah right
      • 2020-10-22 29629, 2020

      • alastairp
        what was the final result of the investigation into the no language codes?
      • 2020-10-22 29643, 2020

      • alastairp
        does #314 fix it?
      • 2020-10-22 29606, 2020

      • Mineo
        ruaok, bitmap, yvanzo: didn't https://github.com/metabrainz/sir/pull/65 add batching to the reindex code? if https://github.com/metabrainz/musicbrainz-docker/… are the defaults for replication - the batch size of 100 seems very small for use cases of for example daily updates of a mirror
      • 2020-10-22 29627, 2020

      • _lucifer
        pybabel does not support the no code. but i think the issue is that the current code was mixing pycountry with pybabel
      • 2020-10-22 29632, 2020

      • alastairp
        did you find the issue of the exception when visiting some user profile pages?
      • 2020-10-22 29632, 2020

      • _lucifer
        yes it does
      • 2020-10-22 29634, 2020

      • alastairp
      • 2020-10-22 29604, 2020

      • alastairp
        yes, I think you're right. mixing the difference between language and countries could be problematic
      • 2020-10-22 29612, 2020

      • _lucifer
        yes it happen when the review of an entity is tried to opened
      • 2020-10-22 29615, 2020

      • alastairp
        I'll double check your work
      • 2020-10-22 29634, 2020

      • _lucifer
        its an issue to be fixed in brainzutils i suppose
      • 2020-10-22 29648, 2020

      • _lucifer
        *merged entity
      • 2020-10-22 29653, 2020

      • alastairp
        oh, interesting
      • 2020-10-22 29609, 2020

      • alastairp
        do you understand the reason for it?
      • 2020-10-22 29632, 2020

      • alastairp
        if so, could you open a ticket, link the sentry issue, and explain what you understand about the issue?
      • 2020-10-22 29610, 2020

      • _lucifer
        no not yet, i was thinking to clone and modify brainzutils it to add some logging and use it to test locally
      • 2020-10-22 29612, 2020

      • _lucifer
        sure i will
      • 2020-10-22 29652, 2020

      • ruaok
        Mineo: ah, thanks for that link -- I hadn't looked at the RMQ side of things yet.
      • 2020-10-22 29641, 2020

      • ruaok
        I see a 'live_index_batch_size' for batching live index stuff. clear.
      • 2020-10-22 29601, 2020

      • ruaok
        is there a batch size setting that is used for replicated instances as well?
      • 2020-10-22 29627, 2020

      • ruaok
      • 2020-10-22 29628, 2020

      • ruaok
        :D
      • 2020-10-22 29634, 2020

      • ruaok
        some love for LB. :)
      • 2020-10-22 29614, 2020

      • ruaok
        alastairp: remember you posted a link about importlib.util.spec_from_file_location yesterday?
      • 2020-10-22 29636, 2020

      • ruaok
        it doesn't return a module that contains all the classes -- just about everything else.
      • 2020-10-22 29600, 2020

      • ruaok
        I've spent over an hour on it trying to improve it, but it doesn't work. the sys.path approach does, so I'll go with that.
      • 2020-10-22 29602, 2020

      • alastairp
        hah, right. sorry about that wild goose chase then
      • 2020-10-22 29618, 2020

      • alastairp
        agreed, changing the path should be good enough then
      • 2020-10-22 29605, 2020

      • ruaok
        all good. I'll work on getting the existing tests working again.
      • 2020-10-22 29618, 2020

      • yvanzo
        Thanks Mineo, we should probably change this value for https://github.com/metabrainz/musicbrainz-docker/…
      • 2020-10-22 29623, 2020

      • ruaok
        is there a version of this for non-live indexing? do non-live rmq messages get batched? if not, this may be a good way to improve things.
      • 2020-10-22 29600, 2020

      • yvanzo
        Hi zas: Just created MBH-545. More generally, I would like to set it up for a musicbrainz-docker instance too.
      • 2020-10-22 29601, 2020

      • BrainzBot
        MBH-545: Add stats for musicbrainz-solr-test and sir-test containers https://tickets.metabrainz.org/browse/MBH-545
      • 2020-10-22 29638, 2020

      • yvanzo
        ruaok: rmq is used for live indexing only.
      • 2020-10-22 29644, 2020

      • _lucifer
        yvanzo: solr-test and sir-test mean the ones used by test.mb?
      • 2020-10-22 29656, 2020

      • yvanzo
        _lucifer: yes
      • 2020-10-22 29639, 2020

      • yvanzo
        bitmap, reosarevok: gh:MBS#1518 is deployed on test.mb.o
      • 2020-10-22 29640, 2020

      • BrainzBot
        MBS-10829: Fix SOLR syntax for recording with no length: https://github.com/metabrainz/musicbrainz-server/…
      • 2020-10-22 29617, 2020

      • reosarevok
        I suspect that test will only help a bit and we will only get proper feedback from at the earliest beta, but
      • 2020-10-22 29641, 2020

      • zas
        yvanzo: ok I'll have a look. Does musicbrainz-docker instance contain solr too? For collecting metrics using telegraf access to port 8983 is needed (of course exposed port can be different, that's the internal solr one). Perhaps there's some access control too to take care of
      • 2020-10-22 29643, 2020

      • yvanzo
        reosarevok: feel free to edit the test db directly to test anything you want to.
      • 2020-10-22 29658, 2020

      • yvanzo
        zas: yes, it would be nice to use a different port. that would be for use on a dedicated cloud node.
      • 2020-10-22 29644, 2020

      • yvanzo
        ruaok, zas: about creating the musicbrainz-docker test node, we need 160GB disk space, would a dedicated CCX21 instance be ok?
      • 2020-10-22 29652, 2020

      • zas
        ccx21 is expensive, does it need dedicated vcpu? CX41 could do I think
      • 2020-10-22 29629, 2020

      • zas
        how much memory does it need?
      • 2020-10-22 29642, 2020

      • yvanzo
        16GB
      • 2020-10-22 29656, 2020

      • yvanzo
        the purpose is to test server load, if CX41 works for that purpose, go for it
      • 2020-10-22 29629, 2020

      • zas
        Imho we can try CX41 (4 non-dedicated vCPUs, 16gb ram, 160gb storage) for half the price of CCX21, at worse, we'll upgrade
      • 2020-10-22 29609, 2020

      • yvanzo
        works for me :)
      • 2020-10-22 29656, 2020

      • yvanzo
        (server load when reindexing)
      • 2020-10-22 29615, 2020

      • alastairp
        hi zas, I have a similar request to yvanzo, to add stats to granfa, remember our discussion from some months back about reading json stats over http from listenbrainz?
      • 2020-10-22 29625, 2020

      • alastairp
        if you have some time either this week or early next week could we test this?
      • 2020-10-22 29614, 2020

      • zas
        sure, we release a new version of Picard today, what about tomorrow around same time?
      • 2020-10-22 29651, 2020

      • alastairp
        perfect, thank you
      • 2020-10-22 29611, 2020

      • alastairp
        is there a repo with telegraf configuration that I can look at myself? (I had a look but didn't find anything obvious)
      • 2020-10-22 29602, 2020

      • alastairp
        iliekcomputers: hi, I guess you won't be around for LB this time tomorrow, but if you have time after work today we could release my LB PR on test so that stats are ready for capturing tomorrow
      • 2020-10-22 29616, 2020

      • zas
      • 2020-10-22 29626, 2020

      • alastairp
        no permission :)
      • 2020-10-22 29634, 2020

      • zas
      • 2020-10-22 29638, 2020

      • zas
        I'll add you
      • 2020-10-22 29600, 2020

      • alastairp
        telegraf-services, yes
      • 2020-10-22 29601, 2020

      • iliekcomputers
        alastairp: I can deploy your branch to beta in the evening, sure.
      • 2020-10-22 29616, 2020

      • alastairp
        iliekcomputers: I can do the same ;) wasn't sure if you wanted to be around for it
      • 2020-10-22 29624, 2020

      • iliekcomputers
        Oh!
      • 2020-10-22 29628, 2020

      • iliekcomputers
        No worries, go ahead.
      • 2020-10-22 29603, 2020

      • zas
        yvanzo: when do you need this VM?
      • 2020-10-22 29605, 2020

      • iliekcomputers
        I'd Just rebase over master once, to make sure nothing gets reverted unintentionally, but even that is optional.
      • 2020-10-22 29614, 2020

      • alastairp
        sure thing
      • 2020-10-22 29637, 2020

      • iliekcomputers
        Thanks!
      • 2020-10-22 29641, 2020

      • yvanzo
        zas: can use it from today
      • 2020-10-22 29626, 2020

      • yvanzo
        zas: cause it is monthly cost not 30days?
      • 2020-10-22 29625, 2020

      • zas
        yes; in fact the cost is per used hour, but they give prices for one month usage
      • 2020-10-22 29642, 2020

      • zas
        of course if you don't need the machine all the time, we can take a more expensive one
      • 2020-10-22 29606, 2020

      • zas
        but you have to take care to shut it down when unused
      • 2020-10-22 29631, 2020

      • yvanzo
        I would prefer that then
      • 2020-10-22 29644, 2020

      • yvanzo
        Since it's only to test CPU intensive tasks.
      • 2020-10-22 29656, 2020

      • ruaok
        pristine___: it seems that there has been a regression -- there are only 200 CF tracks available.
      • 2020-10-22 29643, 2020

      • ruaok manually requests 1000
      • 2020-10-22 29638, 2020

      • pristine___
        ruaok: yeah, 200 only for you. I tested received for you and a user yesterday. Didn't do --top=1000. My bad
      • 2020-10-22 29644, 2020

      • pristine___
        Recs*