#metabrainz

/

      • _lucifer
        now to fix it, i guess the easiest way would be to set a date before which it should never look and just error out

        • 2021-03-05 06407, 2021

      • _lucifer
        another way would be to just look at most a year or few months and error out otherwise?

        • 2021-03-05 06422, 2021

      • _lucifer
        i don't think we want to further back than that to find the latest listen

        • 2021-03-05 06432, 2021

      • _lucifer
        iliekcomputers: thoughts?

        • 2021-03-05 06447, 2021

      • iliekcomputers
        sorry what's the bug?

        • 2021-03-05 06417, 2021

      • iliekcomputers
        the function you mention is used to find the latest listen in the spark cluster.

        • 2021-03-05 06450, 2021

      • _lucifer
        the cluster was empty and it went back till the medivial times to find a listen

        • 2021-03-05 06459, 2021

      • iliekcomputers
        ah, interesting.

        • 2021-03-05 06415, 2021

      • iliekcomputers
        yeah, we should have a lower limit there i guess.

        • 2021-03-05 06455, 2021

      • iliekcomputers
        hesitant to set it to the current year as it could lead to errors on jan 1.

        • 2021-03-05 06402, 2021

      • iliekcomputers
        the year before the current year maybe.

        • 2021-03-05 06413, 2021

      • _lucifer
        6 months or 1 year back from today sounds good to me

        • 2021-03-05 06425, 2021

      • iliekcomputers
        yeah, sounds reasonable to me.

        • 2021-03-05 06451, 2021

      • _lucifer
        šŸ‘

        • 2021-03-05 06404, 2021

      • d4rkie joined the channel

        • 2021-03-05 06456, 2021

      • ruaok
        the absolute lower limit is jan 1 2005.

        • 2021-03-05 06409, 2021

      • Gore joined the channel

        • 2021-03-05 06448, 2021

      • ruaok
        yay dump is nearly imported.

        • 2021-03-05 06451, 2021

      • ruaok
        then mapping

        • 2021-03-05 06411, 2021

      • ruaok
        I'll test my code to write user similarity locally now.

        • 2021-03-05 06422, 2021

      • Gore has quit

        • 2021-03-05 06424, 2021

      • ruaok
        _lucifer: import succeeded. now importing mapping.

        • 2021-03-05 06441, 2021

      • _lucifer
        yay!

        • 2021-03-05 06411, 2021

      • _lucifer
        Done! Time taken: 0.04

        • 2021-03-05 06411, 2021

      • _lucifer
        lol

        • 2021-03-05 06422, 2021

      • ruaok
        do we have any code yet to display the user similarities?

        • 2021-03-05 06412, 2021

      • _lucifer
        shivam-kapila shared a mockup for that yesterday. he might know

        • 2021-03-05 06401, 2021

      • shivam-kapila
        The semi circular dial is almost done.

        • 2021-03-05 06419, 2021

      • ruaok
        got link?

        • 2021-03-05 06436, 2021

      • shivam-kapila
        To the mockup or code?

        • 2021-03-05 06440, 2021

      • ruaok
        mockup

        • 2021-03-05 06451, 2021

      • shivam-kapila
        A sec please

        • 2021-03-05 06427, 2021

      • shivam-kapila
        https://www.figma.com/file/42lt9ldEzRabn4QQxIKDHgā€¦

        • 2021-03-05 06436, 2021

      • ruaok
        I was also wondering about the similar users that would be shown on your own friends page.

        • 2021-03-05 06458, 2021

      • BrainzGit
        [listenbrainz-server] paramsingh opened pull request #1316 (masterā€¦param-recommend-recordings-from-listens-page): [wip] Allow users to recommend recordings from the listens page https://github.com/metabrainz/listenbrainz-serverā€¦

        • 2021-03-05 06413, 2021

      • shivam-kapila
        ruaok: 'You mean people similar to people I follow?

        • 2021-03-05 06424, 2021

      • ruaok
        I think so, yes.

        • 2021-03-05 06441, 2021

      • shivam-kapila
        I added something to the figmas board. something like that works?

        • 2021-03-05 06445, 2021

      • mckean_ has quit

        • 2021-03-05 06447, 2021

      • shivam-kapila
        *figma

        • 2021-03-05 06437, 2021

      • ruaok
        hmm. tightly packed sidebar, empty main space.

        • 2021-03-05 06444, 2021

      • ruaok
        what else goes into the main space here?

        • 2021-03-05 06438, 2021

      • shivam-kapila
        we were not yet sure. I think we were gonna ask users to pin something (like their fav. recordings) to their profile?

        • 2021-03-05 06415, 2021

      • ruaok
        how about putting the "similar users" into the main area for now?

        • 2021-03-05 06418, 2021

      • shivam-kapila
        I have a few questions

        • 2021-03-05 06428, 2021

      • _lucifer
        mapping import done

        • 2021-03-05 06437, 2021

      • iliekcomputers
        does the similarity need to be a dial?

        • 2021-03-05 06445, 2021

      • iliekcomputers
        how does it fit on the listens page?

        • 2021-03-05 06406, 2021

      • shivam-kapila
        we can use linear scale if thats preferable

        • 2021-03-05 06423, 2021

      • iliekcomputers
        last.fm does a simple progress bar line which fills up and changes color with different levels of compatability.

        • 2021-03-05 06453, 2021

      • shivam-kapila
        (I mean thats an easier react component :D)

        • 2021-03-05 06417, 2021

      • ruaok
        _lucifer: dataframes requested, with 365 days to start with.

        • 2021-03-05 06424, 2021

      • shivam-kapila
        > I have a few questions

        • 2021-03-05 06424, 2021

      • shivam-kapila
        Do we want to make the profile page for those whom I follow and whom I dont exactly to be same

        • 2021-03-05 06426, 2021

      • _lucifer
        šŸ‘

        • 2021-03-05 06452, 2021

      • shivam-kapila
        2. Do we show similarity score on profile of people whom I already follow?

        • 2021-03-05 06405, 2021

      • _lucifer
        ruaok: did you send job_type for similarity?

        • 2021-03-05 06423, 2021

      • ruaok
        python3 manage.py spark request_dataframes --days=365 --job-type=user_similarity

        • 2021-03-05 06445, 2021

      • shivam-kapila
        3. Do we show people similar to John Doe(whom I dont follow yet) when I visit their profile

        • 2021-03-05 06446, 2021

      • ruaok
        we should rename the job type to "similar_users" which is what we call this feature in other places.

        • 2021-03-05 06458, 2021

      • _lucifer
        i have renamed it and pushed the change

        • 2021-03-05 06414, 2021

      • _lucifer
        but its not updated in the container yet

        • 2021-03-05 06435, 2021

      • test1651 joined the channel

        • 2021-03-05 06445, 2021

      • ruaok
        thanks.

        • 2021-03-05 06450, 2021

      • mckean joined the channel

        • 2021-03-05 06420, 2021

      • shivam-kapila
        iliekcomputers: How about these: https://getbootstrap.com/docs/3.3/components/#proā€¦

        • 2021-03-05 06459, 2021

      • iliekcomputers
        shivam-kapila: yep, something exactly like that

        • 2021-03-05 06405, 2021

      • test1651 has left the channel

        • 2021-03-05 06433, 2021

      • shivam-kapila
        okay

        • 2021-03-05 06420, 2021

      • shivam-kapila
        1. Do we want to make the profile page for those whom I follow and whom I dont exactly to be same

        • 2021-03-05 06436, 2021

      • shivam-kapila
        2. Do we show similarity score on profile of people whom I already follow?

        • 2021-03-05 06449, 2021

      • shivam-kapila
        3. Do we show people similar to John Doe(whom I dont follow yet) when I visit their profile

        • 2021-03-05 06415, 2021

      • shivam-kapila
        ruaok: Mr_Monkey: iliekcomputers: _lucifer: alastairp Can you share your preferences for these

        • 2021-03-05 06443, 2021

      • ruaok
        my take: 1. not sure. 2. sure. 3. yes, I think so.

        • 2021-03-05 06406, 2021

      • _lucifer
        are we going to followers while fetching similarity? if not, might as well display it for all users

        • 2021-03-05 06423, 2021

      • Mr_Monkey
        1. probably? What would be the differences?

        • 2021-03-05 06423, 2021

      • Mr_Monkey
        2. yes

        • 2021-03-05 06423, 2021

      • Mr_Monkey
        3. I think that's a good way for users to explore more music tastes without being too invasive. Definite yes for me.

        • 2021-03-05 06416, 2021

      • iliekcomputers
        Mr_Monkey: can you review https://github.com/metabrainz/listenbrainz-serverā€¦ Once this is done, I can get started on the changes we need to make to feed API endpoint.

        • 2021-03-05 06421, 2021

      • shivam-kapila
        I am also not quite sure about the differences yet

        • 2021-03-05 06435, 2021

      • Mr_Monkey
        iliekcomputers: OK!

        • 2021-03-05 06452, 2021

      • shivam-kapila
        iliekcomputers: Where should we place the similarity score component on listens page

        • 2021-03-05 06456, 2021

      • Mr_Monkey
        Food first, but top of my list.

        • 2021-03-05 06459, 2021

      • shivam-kapila
        Above BP?

        • 2021-03-05 06413, 2021

      • BrainzGit
        [mbsssss] reosarevok opened pull request #60 (masterā€¦SEARCH-646): SEARCH-646: Prioritize exact matches for tag search https://github.com/metabrainz/mbsssss/pull/60

        • 2021-03-05 06413, 2021

      • Mr_Monkey
        Under the user name?

        • 2021-03-05 06416, 2021

      • iliekcomputers
        shivam-kapila: i'd say we can put it somewhere just above the listens or something.

        • 2021-03-05 06433, 2021

      • v6lur joined the channel

        • 2021-03-05 06435, 2021

      • Mr_Monkey
        We should probably refactor in the future and have a UserName component we can drop in anywhere that has the user name (duh!), compatibility score and follow button.

        • 2021-03-05 06435, 2021

      • reosarevok
        yvanzo: ^ can you see if that PR seems legit? If so, how do we test it? :)

        • 2021-03-05 06401, 2021

      • ruaok
        _lucifer: cron snook an incremental import into the mix. lets see how that goes. :)

        • 2021-03-05 06402, 2021

      • iliekcomputers
        Mr_Monkey: yep, and the supporter badge idea we had.

        • 2021-03-05 06409, 2021

      • Mr_Monkey
        That too !

        • 2021-03-05 06432, 2021

      • Mr_Monkey
        And anything else we end up with that we want to associate with a short presentation of the user

        • 2021-03-05 06440, 2021

      • D4RK-PH0_ joined the channel

        • 2021-03-05 06432, 2021

      • D4RK-PH0_ has quit

        • 2021-03-05 06406, 2021

      • D4RK-PH0_ joined the channel

        • 2021-03-05 06456, 2021

      • d4rkie has quit

        • 2021-03-05 06400, 2021

      • shivam-kapila
        iliekcomputers: Reg. the failing build numpy has dropped support for python 3.6.x and we are using 3.6.9

        • 2021-03-05 06421, 2021

      • iliekcomputers
        :/

        • 2021-03-05 06429, 2021

      • iliekcomputers
        did it happen like yesterday?

        • 2021-03-05 06400, 2021

      • _lucifer
        i think numpy 1.20.1 and a pr was recently merged to update numpy

        • 2021-03-05 06452, 2021

      • _lucifer
        we need to revert that for spark, i'll open a PR

        • 2021-03-05 06417, 2021

      • alastairp
        shivam-kapila: oops, good catch. that was my fault

        • 2021-03-05 06425, 2021

      • alastairp
        let's downgrade again

        • 2021-03-05 06454, 2021

      • shivam-kapila
        what version were we previously using?

        • 2021-03-05 06416, 2021

      • alastairp
        https://github.com/metabrainz/listenbrainz-serverā€¦

        • 2021-03-05 06445, 2021

      • alastairp
        could you revert that and also add a comment that explains why we use this version?

        • 2021-03-05 06454, 2021

      • shivam-kapila
        sure thing

        • 2021-03-05 06429, 2021

      • alastairp
        I think that _lucifer was looking at upgrading the java version of the spark images, so once we do that we can also upgrade python (see https://github.com/metabrainz/listenbrainz-serverā€¦)

        • 2021-03-05 06441, 2021

      • _lucifer
        ruaok: this incremental dump import is taking a long time ;)

        • 2021-03-05 06458, 2021

      • ruaok
        ` __init__: Copying old listens into '/temp'`

        • 2021-03-05 06410, 2021

      • ruaok
        if this is moving a whole dump out of the way... yeah.

        • 2021-03-05 06414, 2021

      • ruaok
        not the best timing, cron.

        • 2021-03-05 06426, 2021

      • iliekcomputers
        yeah, that's a thing we need to fix.

        • 2021-03-05 06426, 2021

      • ruaok
        sheesh, cron. timing is supposed to be your one job.

        • 2021-03-05 06446, 2021

      • iliekcomputers
        it wouldn't be a problem if our data in spark was partitioned by day instead of month.

        • 2021-03-05 06409, 2021

      • ruaok
        _lucifer: gives me a moment to debug my code. :)

        • 2021-03-05 06424, 2021

      • v6lur has quit

        • 2021-03-05 06426, 2021

      • iliekcomputers
        Mr_Monkey: once you're back from lunch, let's chat about the things i should return from the feed endpoint once before i get started.

        • 2021-03-05 06422, 2021

      • BrainzGit
        [listenbrainz-server] shivam-kapila opened pull request #1317 (masterā€¦revert-numpy-version): Revert numpy version to 1.19.4 https://github.com/metabrainz/listenbrainz-serverā€¦

        • 2021-03-05 06428, 2021

      • shivam-kapila
        alastairp: done

        • 2021-03-05 06417, 2021

      • alastairp
        shivam-kapila: there is a 1.19.5 released too

        • 2021-03-05 06409, 2021

      • shivam-kapila
        oh yes it also supports 3.6. Updating

        • 2021-03-05 06439, 2021

      • iliekcomputers
        shivam-kapila: can you add a comment explaining why we can't upgrade in the requirements.txt

        • 2021-03-05 06417, 2021

      • shivam-kapila
        done

        • 2021-03-05 06427, 2021

      • alastairp
        shivam-kapila: sorry, one more thing. can you put the comment one line above, instead of on the same line?

        • 2021-03-05 06431, 2021

      • BrainzGit
        [listenbrainz-server] amCap1712 merged pull request #1295 (masterā€¦spotify): Check if user has revoked authorization if refreshing token fails https://github.com/metabrainz/listenbrainz-serverā€¦

        • 2021-03-05 06449, 2021

      • _lucifer
        ruaok: alastairp: if its fine i'd like to make a release to beta to test the above PR

        • 2021-03-05 06415, 2021

      • shivam-kapila
        alastairp: done

        • 2021-03-05 06437, 2021

      • alastairp
        _lucifer: what service does this affect? spotify reader?

        • 2021-03-05 06447, 2021

      • iliekcomputers
        _lucifer: we don't run spotify reader in beta, so it'll go in prod.

        • 2021-03-05 06450, 2021

      • alastairp
        it's only present on prod

        • 2021-03-05 06457, 2021

      • _lucifer
        oh ok!

        • 2021-03-05 06402, 2021

      • alastairp
        maybe we should take a bit more time and set up a full stack on test?

        • 2021-03-05 06443, 2021

      • _lucifer
        https://www.irccloud.com/pastebin/4bgntSzp/

        • 2021-03-05 06450, 2021

      • _lucifer
        oh come on another one :(

        • 2021-03-05 06403, 2021

      • shivam-kapila
        get some popcorn _lucifer

        • 2021-03-05 06447, 2021

      • _lucifer
        ruaok, can you please check the queue and cancel in case there are more?

        • 2021-03-05 06422, 2021

      • ruaok
        only our request.

        • 2021-03-05 06436, 2021

      • _lucifer
        šŸ˜„

        • 2021-03-05 06436, 2021

      • ruaok
        I just finished debugging my code. let me push a new image and get that deployed.

        • 2021-03-05 06456, 2021

      • Mr_Monkey
        iliekcomputers: I'm back !

        • 2021-03-05 06457, 2021

      • ruaok
        but I am happy to terminate the request consumer to allows us to move on.

        • 2021-03-05 06414, 2021

      • iliekcomputers
        Mr_Monkey: hey.

        • 2021-03-05 06424, 2021

      • _lucifer
        sure if it won't end up corrupting the dump

        • 2021-03-05 06445, 2021

      • Mr_Monkey
        So this is what I have so far: https://docs.google.com/document/d/1_rVY55eHksV0Gā€¦

        • 2021-03-05 06450, 2021

      • ruaok
        unsure. let's hope for the best.

        • 2021-03-05 06406, 2021

      • _lucifer
        šŸ¤ž

        • 2021-03-05 06424, 2021

      • iliekcomputers
        yo