#metabrainz

/

      • lucifer
        yes makes sense. 1.4 deprecates stuff which will be removed in 2.0
      • 2021-08-31 24315, 2021

      • reosarevok
        So I guess we should try and get these tests working, and then look at the warnings?
      • 2021-08-31 24334, 2021

      • reosarevok
        Well, maybe we should try to run the indexer on all entities first, to see if that works...
      • 2021-08-31 24342, 2021

      • lucifer
        i'll try to fix the tests. it looks like a real issue to me. in the meantime you can test other entities?
      • 2021-08-31 24326, 2021

      • monkey
        > Are many of your listens still without mbids? :o
      • 2021-08-31 24326, 2021

      • monkey
        jasondk: None of my listens have MBIDs unfortunately. That's true locally and on test.lb. I mainly use Spotify and local music players posting to LB, I guess neither send the MBIDs…
      • 2021-08-31 24349, 2021

      • MrClon
        reosarevok: Ok, how to setting links as ended?
      • 2021-08-31 24334, 2021

      • monkey
        What I would suggest is to wait until listens have MBIDs for all users before deploying the CB review feature. I know it's not ideal, but currently for a user like me the feature just looks broken and that can give off the wrong impression.
      • 2021-08-31 24322, 2021

      • reosarevok
        lucifer: ok, I'll do that after lunch
      • 2021-08-31 24332, 2021

      • lucifer
        thanks!
      • 2021-08-31 24359, 2021

      • reosarevok
        MrClon: it used to be super complicated, but we have new code in beta for that now :) So if you go to beta.musicbrainz.org you can set links as ended by clicking the pencil button on the same place you'd add them / remove them
      • 2021-08-31 24301, 2021

      • opal has quit
      • 2021-08-31 24327, 2021

      • opal joined the channel
      • 2021-08-31 24329, 2021

      • Guest36 joined the channel
      • 2021-08-31 24334, 2021

      • Guest36 has quit
      • 2021-08-31 24330, 2021

      • MrClon
        reosarevok: So i should POST to /ws/js/edit/create something like `{"edits":[{"id":3301206,"linkTypeID":75,"ended":false,"edit_type":91}],"makeVotable":false,"editNote":"some note"}`? `id` is relationship ID? How to get that ID?
      • 2021-08-31 24319, 2021

      • alastairp
        reosarevok: hi, looks like you're busy, but can I add one more thing to your list and ask you to give a quick look over https://musicbrainz.wordpress.com/wp-admin/post.p… when you have time?
      • 2021-08-31 24311, 2021

      • yvanzo
        yyoung[m]: yes, unless relationships to such links are modified
      • 2021-08-31 24320, 2021

      • reosarevok
        MrClon: from the database, I don't know if there's another way. As I said, this is not meant to be a normal way of doing edits :)
      • 2021-08-31 24332, 2021

      • reosarevok
        alastairp: sure, in a bit :)
      • 2021-08-31 24339, 2021

      • alastairp
        thanks!
      • 2021-08-31 24324, 2021

      • yvanzo
        MrClon: also, entering edits automatically should be done through a bot account: https://musicbrainz.org/doc/Bots
      • 2021-08-31 24352, 2021

      • MrClon
        i know
      • 2021-08-31 24310, 2021

      • MrClon
        If someone change broken link to correct, it will be new relationship with new reationship ID? Or it just update one row in urls table?
      • 2021-08-31 24322, 2021

      • yvanzo
        It depends: Small changes (such as http <-> https) can be made directly to the URL entity (just update one row in url table). Larger changes are expected to replace both the URL and the relationship with new ones.
      • 2021-08-31 24315, 2021

      • yvanzo
        But a relationship id cannot be reused to target another URL entity.
      • 2021-08-31 24352, 2021

      • reosarevok
        alastairp: "The fact that we can identify the same recording as being similar even with the MusicBrainz ID is different is interesting"
      • 2021-08-31 24353, 2021

      • reosarevok
        without?
      • 2021-08-31 24303, 2021

      • reosarevok
        "The data files generated by Annoy are stand-alone" this is the first mention of Annoy in the post. Either it's useful (maybe it can use more info) or it's not (and can be dropped) or it's a useful extra detail for people who know about that stuff (and then maybe it could be "The data files (generated by Annoy) are stand-alone")
      • 2021-08-31 24337, 2021

      • reosarevok
        "If you think that you might be interested in this, let us know" did you stop but want to write more, or is that the end? (since there's no punctuation, I'm not sure)
      • 2021-08-31 24345, 2021

      • reosarevok
        If it's the end, maybe "let us know!" ? :)
      • 2021-08-31 24352, 2021

      • reosarevok
        The general post seems good (and cool!)
      • 2021-08-31 24342, 2021

      • alastairp
        reosarevok: "even when the MBID is different", will fix
      • 2021-08-31 24336, 2021

      • alastairp
        yeah, that last paragraph was from the first draft, and I guess it survived. I'll add in a bit about annoy or make the last para more generic, and make a better ending
      • 2021-08-31 24344, 2021

      • alastairp
        in fact, I think there was a ! at the end
      • 2021-08-31 24349, 2021

      • alastairp
        thank you
      • 2021-08-31 24309, 2021

      • opal has quit
      • 2021-08-31 24322, 2021

      • lucifer
        reosarevok: can you please push your changes. i think i have a potential fix.
      • 2021-08-31 24332, 2021

      • opal joined the channel
      • 2021-08-31 24338, 2021

      • reosarevok
        lucifer: sure, one sec :)
      • 2021-08-31 24346, 2021

      • lucifer
        alastairp: i forgot my jenkins password 😓, do you know how i can reset it?
      • 2021-08-31 24339, 2021

      • reosarevok
        lucifer: done
      • 2021-08-31 24349, 2021

      • lucifer
        👍 thanks!
      • 2021-08-31 24357, 2021

      • reosarevok
        Can you push to my branch?
      • 2021-08-31 24306, 2021

      • reosarevok
        Or should I just push this on top of the existing PR?
      • 2021-08-31 24350, 2021

      • lucifer
        existing PR sounds good.
      • 2021-08-31 24322, 2021

      • reosarevok
        Ok
      • 2021-08-31 24318, 2021

      • reosarevok
        Wtf, I must be doing something stupid
      • 2021-08-31 24357, 2021

      • reosarevok
        It tells me everything up to date after a push but https://github.com/metabrainz/sir/pull/111/commits still doesn't have my commits
      • 2021-08-31 24332, 2021

      • reosarevok
        Oh, nevermind
      • 2021-08-31 24333, 2021

      • reosarevok
        Name clash
      • 2021-08-31 24342, 2021

      • reosarevok
        Now there
      • 2021-08-31 24352, 2021

      • reosarevok
        lucifer: all yours
      • 2021-08-31 24321, 2021

      • lucifer
        👍
      • 2021-08-31 24330, 2021

      • yvanzo
      • 2021-08-31 24343, 2021

      • lucifer
        yvanzo: about that, i faced the issue again today. adding `/code/venv-musicbrainz-docker/lib/python2.7/site-packages` to PYTHONPATH fixed it.
      • 2021-08-31 24329, 2021

      • lucifer
        i checked the default sys.path and it also has both /usr/local/lib/python2.7 and /usr/local/lib/python2.7/site-packages.
      • 2021-08-31 24307, 2021

      • lucifer
        but i am not sure if this is indeed the right way so will look further into it and let you know.
      • 2021-08-31 24356, 2021

      • yvanzo
        Ok, thanks
      • 2021-08-31 24359, 2021

      • reosarevok
        "I could not find docs on this but doing a `vars` dump"
      • 2021-08-31 24300, 2021

      • reosarevok
        Sigh
      • 2021-08-31 24317, 2021

      • lucifer
        :)
      • 2021-08-31 24324, 2021

      • reosarevok
        I wish stuff got documented properly, especially for huge libraries...
      • 2021-08-31 24340, 2021

      • lucifer
        woo. tests pass now.
      • 2021-08-31 24357, 2021

      • lucifer
        let's run it for all entities now.
      • 2021-08-31 24306, 2021

      • reosarevok
        I can try that on my sample data first
      • 2021-08-31 24313, 2021

      • reosarevok
        If that works I guess maybe we should try with non-sample?
      • 2021-08-31 24325, 2021

      • reosarevok
        But that'll take forever and I don't have it set up
      • 2021-08-31 24353, 2021

      • lucifer
        let's do it on sample. i don't have the full dump either.
      • 2021-08-31 24359, 2021

      • reosarevok
        Ok, I'll run it
      • 2021-08-31 24303, 2021

      • yvanzo
        Yes, sample data would be good enough.
      • 2021-08-31 24312, 2021

      • lucifer
        once it works on sample, we can either run on bono for a full run or sir_test.
      • 2021-08-31 24337, 2021

      • yvanzo
        I'm currently playing with sir-test.
      • 2021-08-31 24302, 2021

      • lucifer
        ah ok
      • 2021-08-31 24304, 2021

      • legoktm[m] has quit
      • 2021-08-31 24304, 2021

      • kepstin has quit
      • 2021-08-31 24305, 2021

      • elomatreb[m] has quit
      • 2021-08-31 24305, 2021

      • reg[m] has quit
      • 2021-08-31 24305, 2021

      • akashgp09[m] has quit
      • 2021-08-31 24305, 2021

      • JuniorJPDJ has quit
      • 2021-08-31 24306, 2021

      • yyoung[m] has quit
      • 2021-08-31 24306, 2021

      • tandy[m] has quit
      • 2021-08-31 24309, 2021

      • riksucks has quit
      • 2021-08-31 24309, 2021

      • MagnusSvensson[4 has quit
      • 2021-08-31 24309, 2021

      • Lorenzo[m] has quit
      • 2021-08-31 24311, 2021

      • akshaaatt[m] has quit
      • 2021-08-31 24333, 2021

      • reosarevok
        lucifer: uh-oh
      • 2021-08-31 24348, 2021

      • lucifer
        yup i ran and saw
      • 2021-08-31 24349, 2021

      • reosarevok
      • 2021-08-31 24352, 2021

      • reosarevok
        Oh, ok
      • 2021-08-31 24359, 2021

      • lucifer
        a mess it is
      • 2021-08-31 24320, 2021

      • lucifer
        load_only won't take the name, its wants the column object. undefer takes the name
      • 2021-08-31 24339, 2021

      • riksucks joined the channel
      • 2021-08-31 24352, 2021

      • reosarevok
        Hmmm. But load_only seemed to happily work with the same defer did
      • 2021-08-31 24304, 2021

      • reosarevok
        And https://docs.sqlalchemy.org/en/14/orm/loading_col… suggests that defer and undefer take the same
      • 2021-08-31 24307, 2021

      • elomatreb[m] joined the channel
      • 2021-08-31 24319, 2021

      • yyoung[m] joined the channel
      • 2021-08-31 24319, 2021

      • kepstin joined the channel
      • 2021-08-31 24331, 2021

      • akshaaatt[m] joined the channel
      • 2021-08-31 24331, 2021

      • JuniorJPDJ joined the channel
      • 2021-08-31 24331, 2021

      • tandy[m] joined the channel
      • 2021-08-31 24331, 2021

      • MagnusSvensson[m joined the channel
      • 2021-08-31 24331, 2021

      • akashgp09[m] joined the channel
      • 2021-08-31 24331, 2021

      • legoktm[m] joined the channel
      • 2021-08-31 24331, 2021

      • reg[m] joined the channel
      • 2021-08-31 24343, 2021

      • Lorenzo[m] joined the channel
      • 2021-08-31 24353, 2021

      • lucifer
        yes. acc to my understanding we have column names. defer and undefer take those.
      • 2021-08-31 24334, 2021

      • reosarevok
        But load_only seemed to be taking those too, when I ran just instrument
      • 2021-08-31 24347, 2021

      • lucifer
        right, it works for instrument which is surprising. the docs do not show load_only working with column names. even the args are named attrs and for defer and undefer its column key.
      • 2021-08-31 24348, 2021

      • lucifer
      • 2021-08-31 24301, 2021

      • lucifer
        heh load_only has been available since 0.9.0!
      • 2021-08-31 24325, 2021

      • reosarevok
        Hmm
      • 2021-08-31 24330, 2021

      • reosarevok
        I tried to run work only, got
      • 2021-08-31 24331, 2021

      • reosarevok
      • 2021-08-31 24341, 2021

      • reosarevok
        So, a second, different error ?
      • 2021-08-31 24307, 2021

      • lucifer
        yeah looks like it
      • 2021-08-31 24310, 2021

      • krishan has quit
      • 2021-08-31 24313, 2021

      • krishan joined the channel
      • 2021-08-31 24322, 2021

      • krishan has quit
      • 2021-08-31 24345, 2021

      • krishan joined the channel
      • 2021-08-31 24304, 2021

      • reosarevok
        How the hell do all tests pass, in any case. Are the tests not actually testing indexing at all? :D
      • 2021-08-31 24322, 2021

      • lucifer
        the wonders of mocks :)
      • 2021-08-31 24338, 2021

      • Lotheric joined the channel
      • 2021-08-31 24346, 2021

      • reosarevok
        I do feel, indeed, mocked
      • 2021-08-31 24321, 2021

      • Lotheric
        o.O
      • 2021-08-31 24320, 2021

      • reosarevok
        I wonder if it's somehow connected to PoolWorker-7 2021-08-31 12:49:03,663 DEBUG: Loading only [] on <class 'sir.schema.modelext.CustomLinkAttribute'>
      • 2021-08-31 24328, 2021

      • reosarevok
        (to it being passed nothing)
      • 2021-08-31 24352, 2021

      • krishan has quit
      • 2021-08-31 24356, 2021

      • reosarevok
        Nope
      • 2021-08-31 24314, 2021

      • krishan joined the channel
      • 2021-08-31 24321, 2021

      • lucifer
        i am inclined to get rid of all this entirely. i see that the default for relationship loading is lazy but cannot find the default for columns but it looks like undeferred by default.
      • 2021-08-31 24351, 2021

      • reosarevok
        Get rid of all this as in, skip eager loading completely?
      • 2021-08-31 24318, 2021

      • lucifer
        yes. for now. until we get this working.
      • 2021-08-31 24352, 2021

      • reosarevok
        Does it work if we remove the load_only call? Let's test
      • 2021-08-31 24327, 2021

      • reosarevok
        Looks like it's working
      • 2021-08-31 24336, 2021

      • reosarevok
        Slowly, but, sent 200 documents a few times
      • 2021-08-31 24345, 2021

      • lucifer
        i am curently running for url. ok so far.
      • 2021-08-31 24323, 2021

      • reosarevok
        Work, here
      • 2021-08-31 24322, 2021

      • reosarevok
        Seems crazy slow, but of course, it might have been crazy slow either way
      • 2021-08-31 24333, 2021

      • reosarevok
        Since I'm certainly below speca
      • 2021-08-31 24336, 2021

      • reosarevok
        Specs too
      • 2021-08-31 24358, 2021

      • reosarevok
        Heh. It crashed my Chrome, even
      • 2021-08-31 24308, 2021

      • reosarevok
        Too little memory I guess
      • 2021-08-31 24312, 2021

      • reosarevok
        Still working tho
      • 2021-08-31 24349, 2021

      • reosarevok
        lucifer: did URL finish for you?
      • 2021-08-31 24304, 2021

      • lucifer
        nope.
      • 2021-08-31 24326, 2021

      • lucifer
        how many urls are there?
      • 2021-08-31 24347, 2021

      • lucifer
        approximately in the sample dump
      • 2021-08-31 24353, 2021

      • reosarevok
        A lot, I guess :D
      • 2021-08-31 24300, 2021

      • reosarevok
        I can check, one sec