#metabrainz

/

      • lucifer
        +1 on both
      • 2021-06-28 17953, 2021

      • BrainzGit
        [listenbrainz-server] 14mayhem opened pull request #1529 (03master…local-user-similarity-scaling): Move to per-user scaling https://github.com/metabrainz/listenbrainz-server…
      • 2021-06-28 17947, 2021

      • TOPIC: MetaBrainz Community and Development channel | MusicBrainz non-development: #musicbrainz | BookBrainz: #bookbrainz | Channel is logged; see https://musicbrainz.org/doc/IRC for details | Agenda: Reviews, MBS-11750 (reo)
      • 2021-06-28 17952, 2021

      • lucifer
        alastairp: LB email PR is updated. also opened a docker-server-configs PR to log out all users. to be merged only after we deploy the email PR.
      • 2021-06-28 17936, 2021

      • alastairp
        nice, looking at it now
      • 2021-06-28 17932, 2021

      • lucifer
        thanks!
      • 2021-06-28 17908, 2021

      • TOPIC: MetaBrainz Community and Development channel | MusicBrainz non-development: #musicbrainz | BookBrainz: #bookbrainz | Channel is logged; see https://musicbrainz.org/doc/IRC for details | Agenda: Reviews, MBS-11750 (reo), MBBE-40 (reo)
      • 2021-06-28 17956, 2021

      • BrainzGit
        [musicbrainz-server] 14reosarevok opened pull request #2152 (03master…MBS-11732): MBS-11732 / MBS-11733: Remove LYRICSnMUSIC / WikiaParoles from lyrics whitelist https://github.com/metabrainz/musicbrainz-server/…
      • 2021-06-28 17938, 2021

      • BrainzGit
        [musicbrainz-server] 14reosarevok opened pull request #2153 (03master…MBS-11726): MBS-11726: Add missing space to Set Cover Art display https://github.com/metabrainz/musicbrainz-server/…
      • 2021-06-28 17905, 2021

      • lucifer
        alastairp: do you know if exceptions while using `map` are handled differently?
      • 2021-06-28 17907, 2021

      • lucifer
      • 2021-06-28 17946, 2021

      • lucifer
        a 400 is raised when iterating using map but its returned as a 500.
      • 2021-06-28 17946, 2021

      • alastairp
        lucifer: no, I'm not sure. can you reproduce it clearly by just changing the way that you iterate?
      • 2021-06-28 17932, 2021

      • alastairp
        my gut feel for it being a 500 is that the code that maps an exception to a specific status code isn't working properly (because this is a custom exception)
      • 2021-06-28 17901, 2021

      • lucifer
        possible. i am trying to iterate using a loop now, let's see if tests pass.
      • 2021-06-28 17940, 2021

      • ruaok
        we also need to record a new youtube video. I made the first one -- who wants to make the next one? alastairp monkey lucifer ?
      • 2021-06-28 17917, 2021

      • lucifer
        right, i forgot about that email. i'll try to do that.
      • 2021-06-28 17925, 2021

      • ruaok
        sweet, thanks!
      • 2021-06-28 17948, 2021

      • lucifer
        these connection closed errors are becoming more frequent and annoying in tests. do we have any idea why they occur?
      • 2021-06-28 17908, 2021

      • ruaok
        I've been wondering the same, but have no clue.
      • 2021-06-28 17911, 2021

      • lucifer
        yeah, i don't have any idea either. i'll open a ticket for further investigation.
      • 2021-06-28 17925, 2021

      • lucifer
        alastairp, tests passes after changing map to a loop.
      • 2021-06-28 17959, 2021

      • alastairp
        that's... odd
      • 2021-06-28 17947, 2021

      • wargreen joined the channel
      • 2021-06-28 17918, 2021

      • MRiddickW joined the channel
      • 2021-06-28 17921, 2021

      • lucifer
        yup, right. i am merging the loop version. will try to understand why this happened later.
      • 2021-06-28 17953, 2021

      • lucifer
        thanks for taking care of 1518, alastairp. i'll try to fix the failing test and then merge.
      • 2021-06-28 17919, 2021

      • alastairp
        lucifer: btw, I almost always use list comprehensions instead of map in python
      • 2021-06-28 17949, 2021

      • lucifer
        oh. thinking of it, comprehension looks better than loop as well. changing to it.
      • 2021-06-28 17909, 2021

      • BrainzGit
        [troi-recommendation-playground] 14alastair merged pull request #37 (03main…ab-patch): Fix AB patch argument parsing https://github.com/metabrainz/troi-recommendation…
      • 2021-06-28 17929, 2021

      • lucifer
        alastairp: regarding troi PR, i didn't reply to the user because i wasn't sure that similarity.ab is up anyways. i think it used to be the bono AB instance which now use for testing other PRs.
      • 2021-06-28 17953, 2021

      • alastairp
        right. this might be the same person I had a chat to on the forums who was using it
      • 2021-06-28 17908, 2021

      • alastairp
        after that discussion I started it up again, but who knows, maybe we replaced it again ;)
      • 2021-06-28 17942, 2021

      • alastairp
        I need to work out a better way of doing email. I filter meb stuff to a folder and then forget about it
      • 2021-06-28 17944, 2021

      • lucifer
        yeah probably. should we merge the relevant PR into master?
      • 2021-06-28 17901, 2021

      • alastairp
        which relevant PR?
      • 2021-06-28 17915, 2021

      • zas
        yvanzo: ping
      • 2021-06-28 17930, 2021

      • lucifer
        the PR which contains the endpoints that troi needs.
      • 2021-06-28 17945, 2021

      • zas
        yvanzo: SIR prod is stuck, usual cmp error
      • 2021-06-28 17944, 2021

      • alastairp
        right. that's definitely something I should do. after all of our testing I think it's probably good to merge
      • 2021-06-28 17930, 2021

      • akshaaatt[m]
        <outsidecontext "akshaaatt, lucifer: hi, regardin"> Thank you🙌☺️
      • 2021-06-28 17915, 2021

      • lucifer
        we can do that as well tomorrow.
      • 2021-06-28 17939, 2021

      • lucifer
        when we work on the ml stuff blocking py3.
      • 2021-06-28 17900, 2021

      • BrainzGit
        [listenbrainz-server] 14mayhem opened pull request #1530 (03master…terminate-cron): Terminate cron https://github.com/metabrainz/listenbrainz-server…
      • 2021-06-28 17959, 2021

      • BrainzGit
        [listenbrainz-server] 14amCap1712 merged pull request #1515 (03master…lb-validation): Silently drop empty or null mbids in listen submissions https://github.com/metabrainz/listenbrainz-server…
      • 2021-06-28 17951, 2021

      • ruaok
        mbid mapping writer PR updated, alastairp.
      • 2021-06-28 17908, 2021

      • BrainzGit
        [listenbrainz-server] 14amCap1712 merged pull request #1422 (03master…require-email): LB-849: Require users to have a confirmed email before creating an LB account https://github.com/metabrainz/listenbrainz-server…
      • 2021-06-28 17913, 2021

      • lucifer
        \o/
      • 2021-06-28 17919, 2021

      • lucifer
        finally
      • 2021-06-28 17933, 2021

      • lucifer
        alastairp: ruaok: let's deploy this :D
      • 2021-06-28 17934, 2021

      • ruaok
        go for it. I have nothing else that needs deploying -- unless you want to try the user similarity stuff.
      • 2021-06-28 17906, 2021

      • lucifer
        we'll need to do 3 things. 1) post the blog post 2) deploy this PR 3) merge docker-server-configs and restart web containers if they don't automatically
      • 2021-06-28 17934, 2021

      • lucifer
        user similarity only needs changes on spark side so that can be tested irrespective of this.
      • 2021-06-28 17908, 2021

      • ruaok
        agreed, I can try the spark side of things and am happy to post the blog post.
      • 2021-06-28 17915, 2021

      • lucifer
        +1
      • 2021-06-28 17923, 2021

      • ruaok
        mind if go ahead and test the similarity stuff now?
      • 2021-06-28 17948, 2021

      • ruaok
        and should I post the blog post now?
      • 2021-06-28 17952, 2021

      • lucifer
        sure np
      • 2021-06-28 17954, 2021

      • lucifer
        yes
      • 2021-06-28 17958, 2021

      • ruaok
        ok. ok.
      • 2021-06-28 17958, 2021

      • BrainzGit
        [listenbrainz-server] release 03v-2021-06-28.0 has been published by 14github-actions[bot]: https://github.com/metabrainz/listenbrainz-server…
      • 2021-06-28 17909, 2021

      • lucifer
        if you want i can update spark while the image builds
      • 2021-06-28 17928, 2021

      • ruaok
        plz!
      • 2021-06-28 17934, 2021

      • lucifer
        on it
      • 2021-06-28 17900, 2021

      • ruaok
      • 2021-06-28 17925, 2021

      • alastairp
        nice!
      • 2021-06-28 17904, 2021

      • ruaok
      • 2021-06-28 17918, 2021

      • lucifer
        spark updated. requesting similar users.
      • 2021-06-28 17927, 2021

      • lucifer
        alastairp: ruaok: https://github.com/metabrainz/docker-server-confi… . can one of you check and approve?
      • 2021-06-28 17959, 2021

      • alastairp
        hmm
      • 2021-06-28 17902, 2021

      • alastairp
        one sec
      • 2021-06-28 17936, 2021

      • shivam-kapila
        Congrats lucifer for the jab :)
      • 2021-06-28 17957, 2021

      • shivam-kapila
        And btw I hate python dates now :|
      • 2021-06-28 17927, 2021

      • lucifer
        thanks!
      • 2021-06-28 17928, 2021

      • alastairp
        lucifer: this is what "user".login_id is for, and why we set it up
      • 2021-06-28 17954, 2021

      • lucifer
        oh
      • 2021-06-28 17910, 2021

      • alastairp
        however, I don't think we added the ability to bulk revoke login ids, and I'm not sure what the recommended way to do it is
      • 2021-06-28 17945, 2021

      • alastairp
      • 2021-06-28 17901, 2021

      • alastairp
        "This way you are free to change the user’s alternative id to a new randomly generated value when the user changes their password, which would ensure their old authentication sessions will cease to be valid."
      • 2021-06-28 17954, 2021

      • alastairp
        so yes, I'm 99% sure that to log everyone out we should just give everyone a new login_id (then the login_id that they have in their cookie will become invalid and they'll be forced to re-login)
      • 2021-06-28 17951, 2021

      • lucifer
        alastairp: i see. makes sense. i had looked that at it earlier but didn't understand it correctly.
      • 2021-06-28 17931, 2021

      • ruaok
        lucifer: I see the spark error. will update PR.
      • 2021-06-28 17948, 2021

      • lucifer
        👍
      • 2021-06-28 17911, 2021

      • reosarevok
      • 2021-06-28 17912, 2021

      • BrainzBot
        STYLE-1887: Add lyricstranslate.com to Lyrics whitelist
      • 2021-06-28 17926, 2021

      • reosarevok
        Not sure if we want it or not, but at least it's legit
      • 2021-06-28 17922, 2021

      • ruaok
        yep, looks fine by me.
      • 2021-06-28 17937, 2021

      • BrainzGit
        [listenbrainz-server] 14amCap1712 opened pull request #1531 (03master…logout-all): Add SQL script to logout all users https://github.com/metabrainz/listenbrainz-server…
      • 2021-06-28 17926, 2021

      • alastairp
        lucifer: that looks ok. did you test on a database with multiple users? will it generate a new uuid per user?
      • 2021-06-28 17908, 2021

      • ruaok
        lucifer: if start-spark-request-consumer.sh is the right way to start the consumer, what is the right way to shut it down?
      • 2021-06-28 17906, 2021

      • lucifer
        alastairp: i am almost sure it does, but running it locally to confirm.
      • 2021-06-28 17956, 2021

      • alastairp
        me too
      • 2021-06-28 17918, 2021

      • lucifer
        ruaok: ps aux | grep request_consumer, ps aux | grep Spark . kill these two PIDs.
      • 2021-06-28 17924, 2021

      • ruaok
        k
      • 2021-06-28 17907, 2021

      • alastairp
        lucifer: approved, but I just added a note saying that we should document this script and say what it does and when to use it
      • 2021-06-28 17932, 2021

      • lucifer
        +1
      • 2021-06-28 17951, 2021

      • ruaok
        `docker/start-spark-request-consumer.sh: line 24: spark-submit: command not found`
      • 2021-06-28 17956, 2021

      • ruaok
        lucifer: any ideas ^^ ?
      • 2021-06-28 17905, 2021

      • lucifer
        need to execute as root user
      • 2021-06-28 17916, 2021

      • ruaok
        👍
      • 2021-06-28 17906, 2021

      • ruaok
        `sudo docker/start-spark-request-consumer.sh` gives the same.
      • 2021-06-28 17912, 2021

      • ruaok
        do I literally need to run it from root?
      • 2021-06-28 17927, 2021

      • lucifer
        yeah.
      • 2021-06-28 17956, 2021

      • lucifer
        the spark, hadoop stuff is only in root user's path that's why.
      • 2021-06-28 17922, 2021

      • lucifer
        i am logged in. i can update.
      • 2021-06-28 17909, 2021

      • ruaok
        its updated, I think. but go ahead and start as usual.
      • 2021-06-28 17943, 2021

      • lucifer
        much of this will go away once we daemonize spark.
      • 2021-06-28 17951, 2021

      • lucifer
        (at least i hope so)
      • 2021-06-28 17956, 2021

      • ruaok
        yeah, it should.
      • 2021-06-28 17947, 2021

      • lucifer
        alastairp: ruaok: i realised i forgot to add consul config in LB. the values are hard coded currently. opening PR to fix that. but it depends on https://github.com/metabrainz/docker-server-confi… and https://github.com/metabrainz/listenbrainz-server…
      • 2021-06-28 17908, 2021

      • lucifer
        (consul config for email PR)
      • 2021-06-28 17928, 2021

      • lucifer
        ruaok: similar users generated.
      • 2021-06-28 17944, 2021

      • ruaok
        1528 approved.
      • 2021-06-28 17950, 2021

      • ruaok
        (thanks!)
      • 2021-06-28 17947, 2021

      • ruaok
        176 also approved, based on my shoddy consul knowledge.
      • 2021-06-28 17928, 2021

      • ruaok
        those similarities look really nice!
      • 2021-06-28 17945, 2021

      • lucifer
        i had tested that on test with youtube so i almost it works.
      • 2021-06-28 17948, 2021

      • ruaok
        however, this is now utterly useless https://listenbrainz.org/similar-users
      • 2021-06-28 17908, 2021

      • ruaok
        almost is pretty much perfect, no?
      • 2021-06-28 17929, 2021

      • lucifer
        *i am almost sure
      • 2021-06-28 17934, 2021

      • ruaok
        :)
      • 2021-06-28 17923, 2021

      • lucifer
        regardless, everything only *almost* works, we'll always find bugs corner cases later :)
      • 2021-06-28 17931, 2021

      • ruaok
        the score column is useless. I think it will still find spammers.
      • 2021-06-28 17933, 2021

      • lucifer
        not sure about spammers but similar users looks nice indeed D
      • 2021-06-28 17935, 2021

      • lucifer
        :D
      • 2021-06-28 17902, 2021

      • ruaok
        I'll have a think more about the spammers aspects, but for now this is good improvement.
      • 2021-06-28 17929, 2021

      • alastairp
        lucifer: just to confirm - have you tested 1528 with a real consul server?
      • 2021-06-28 17947, 2021

      • lucifer
        alastairp: yes, on test.lb with youtube config.
      • 2021-06-28 17910, 2021

      • alastairp
        sounds good to me. thanks
      • 2021-06-28 17910, 2021

      • lucifer
        my primary use case is loading booleans from json though.
      • 2021-06-28 17929, 2021

      • alastairp
        json.loads("True") ?
      • 2021-06-28 17943, 2021

      • alastairp
        true, even
      • 2021-06-28 17948, 2021

      • alastairp
        oh yes, that works. nice
      • 2021-06-28 17923, 2021

      • lucifer
      • 2021-06-28 17929, 2021

      • alastairp
        so instead of doing a python-style "True" string in the consul config, you'll do "key": true instead?
      • 2021-06-28 17936, 2021

      • lucifer
        yes
      • 2021-06-28 17938, 2021

      • alastairp
        ok
      • 2021-06-28 17925, 2021

      • lucifer
        oh duh! 🤦 i didn't think "True" would work that's why i opened the PR in first place
      • 2021-06-28 17920, 2021

      • alastairp
        right - in fact in all of our string values we do '''{{template }}''' anyway - adding the string quote values manually
      • 2021-06-28 17919, 2021

      • ritiek joined the channel
      • 2021-06-28 17927, 2021

      • lucifer
        yeah, realised that now. don't put the quotes in this case and it would subsitute it correctly.
      • 2021-06-28 17922, 2021

      • yvanzo
        zas: on it
      • 2021-06-28 17950, 2021

      • reosarevok
        Releasing prod
      • 2021-06-28 17905, 2021

      • alastairp
        hello. should we <Bang> ?
      • 2021-06-28 17918, 2021

      • lucifer
        1 hr left for that?
      • 2021-06-28 17931, 2021

      • alastairp
        isn't it at 6? wtf. what's wrong with me?
      • 2021-06-28 17954, 2021

      • alastairp
        yeah, thanks. I guess I want the day to end
      • 2021-06-28 17901, 2021

      • lucifer
        lol :D
      • 2021-06-28 17923, 2021

      • lucifer
        i'll merge this as it is, and we can change to "True" if we want later.
      • 2021-06-28 17930, 2021

      • alastairp
        ok