#metabrainz

/

      • ishaanshah[m]
        shivam-kapila: Thanks, computer became unusable had to restart it😅
      • 2020-05-15 13615, 2020

      • alastairp
        ah, interesting. OK, I'll use copy_from and put it in a db
      • 2020-05-15 13631, 2020

      • alastairp
        probably faster to query what i need from a database anyway
      • 2020-05-15 13605, 2020

      • shivam-kapila
        ishaanshah[m]: I was thinking that will you be able to bash into the container after your RAM got full.
      • 2020-05-15 13604, 2020

      • iliekcomputers
        alastairp: what are you using it for?
      • 2020-05-15 13620, 2020

      • alastairp
        I just want a dump of user + artist + play counts
      • 2020-05-15 13659, 2020

      • iliekcomputers
        but for what? :D
      • 2020-05-15 13605, 2020

      • iliekcomputers curious
      • 2020-05-15 13612, 2020

      • mzfr has quit
      • 2020-05-15 13655, 2020

      • ishaanshah[m]
        Nope, it was too late, thanks for the command anyways I have a feeling I will be needing in the future😀
      • 2020-05-15 13626, 2020

      • shivam-kapila
        I also googled it ;)
      • 2020-05-15 13635, 2020

      • ishaanshah[m]
        It*
      • 2020-05-15 13654, 2020

      • alastairp
        iliekcomputers: if this import works with no issues, I should have a demo shortly
      • 2020-05-15 13615, 2020

      • alastairp
        it's really fortuitious that you all got this running again and made me remember it
      • 2020-05-15 13628, 2020

      • iliekcomputers
        🎉
      • 2020-05-15 13616, 2020

      • iliekcomputers
      • 2020-05-15 13641, 2020

      • ruaok
        thanks!
      • 2020-05-15 13658, 2020

      • Mr_Monkey
        Useful, thanks !
      • 2020-05-15 13612, 2020

      • ruaok
        I jumped back into working with it and installed the code in a different place and it just magically worked this morning (because you added a module capability)
      • 2020-05-15 13624, 2020

      • ruaok
        that was nice synchronicity!
      • 2020-05-15 13625, 2020

      • ruaok
        thanks!
      • 2020-05-15 13600, 2020

      • rdswift
        Glad to hear it's working for you. I plan to make a small upgrade to the module sometime soon to allow adding lines to the header section if required, and to also add to the message body.
      • 2020-05-15 13653, 2020

      • BrainzGit
        [listenbrainz-server] mayhem opened pull request #853 (master…artist-relations): Add artist/artist_credit relations https://github.com/metabrainz/listenbrainz-server…
      • 2020-05-15 13640, 2020

      • ruaok
        yep, this weekend I'll continue working on the recommendation stuff and then hopefully more people will get to use it as well.
      • 2020-05-15 13647, 2020

      • ruaok
        thanks for making this one pesky part go away!
      • 2020-05-15 13616, 2020

      • Mr_Monkey
        I feel it's got a lot of potential. Soonâ„¢ playable MusicBrainz pages?
      • 2020-05-15 13644, 2020

      • ruaok
        oh, yes, please. I've been waiting for this for... 2 decades now.
      • 2020-05-15 13645, 2020

      • reosarevok
      • 2020-05-15 13646, 2020

      • BrainzBot
        MBS-10828: musicbrainz-docker createdb.sh fails.. out of space
      • 2020-05-15 13608, 2020

      • Mr_Monkey
        Implementing multiple music sources is a huge pain… but it'll be worth it !
      • 2020-05-15 13636, 2020

      • ruaok
        I hope you're making it nice and modular so we can add more sources
      • 2020-05-15 13640, 2020

      • jmp_music has quit
      • 2020-05-15 13645, 2020

      • Mr_Monkey
        That's the idea.
      • 2020-05-15 13653, 2020

      • ruaok
        <3
      • 2020-05-15 13603, 2020

      • Mr_Monkey
        I'm gutting the poor spotify player out
      • 2020-05-15 13613, 2020

      • Mr_Monkey
        There's lots of back and forth between the components, so lots of handler to pass around, but It'll be modular all right
      • 2020-05-15 13655, 2020

      • Mr_Monkey
        (The main player component has the playback controls, while each music source can update progress, pause/play state, etc.)
      • 2020-05-15 13602, 2020

      • Mr_Monkey
        In the future, it would be nice to implement feedback about incorrect song matching —maybe with an option to provide a better match— to post some info to MessyBrainz
      • 2020-05-15 13610, 2020

      • Mr_Monkey
        Straight in the player
      • 2020-05-15 13637, 2020

      • shivam-kapila
        Sounds interesting
      • 2020-05-15 13613, 2020

      • shivam-kapila
      • 2020-05-15 13613, 2020

      • shivam-kapila
        This is some pretty cool stuff.
      • 2020-05-15 13614, 2020

      • yvanzo
        reosarevok: thanks, on it
      • 2020-05-15 13620, 2020

      • reosarevok
      • 2020-05-15 13648, 2020

      • reosarevok
        Might be relevant :)
      • 2020-05-15 13636, 2020

      • shivam-kapila
        reosarevok: Yeah. I had observed while looking into Chhavi's profile. Thanks :)
      • 2020-05-15 13622, 2020

      • ishaanshah[m]
        iliekcomputers: Hey!
      • 2020-05-15 13605, 2020

      • iliekcomputers
        ishaanshah[m]: hey
      • 2020-05-15 13611, 2020

      • iliekcomputers
        how's the day been?
      • 2020-05-15 13613, 2020

      • ishaanshah[m]
        A bit tough, progress has become a bit slow :(
      • 2020-05-15 13632, 2020

      • ishaanshah[m]
        SQL doesn't agree with me
      • 2020-05-15 13654, 2020

      • v6lur has quit
      • 2020-05-15 13604, 2020

      • iliekcomputers
        if you need any help, feel free to ask here
      • 2020-05-15 13611, 2020

      • ishaanshah[m]
        I got all the time ranges to work
      • 2020-05-15 13618, 2020

      • ruaok
        I'll second that.
      • 2020-05-15 13629, 2020

      • ruaok
        asking good questions is a great skill to have. one day, I'll learn it!
      • 2020-05-15 13604, 2020

      • iliekcomputers
        ishaanshah[m]: that's pretty cool.
      • 2020-05-15 13614, 2020

      • iliekcomputers
        you should open a PR for that first.
      • 2020-05-15 13617, 2020

      • shivam-kapila
        ruaok: you are far better than me 😬
      • 2020-05-15 13629, 2020

      • shivam-kapila
        At asking questions. I am horrible
      • 2020-05-15 13638, 2020

      • ruaok
        I've also been doing this for 20 years. :)
      • 2020-05-15 13643, 2020

      • ishaanshah[m]
        Is there any reason that we don't use orm style queries>
      • 2020-05-15 13657, 2020

      • Freso
        rdswift: I’m considering using your module to replace the file writing stuff in the Add Cluster As Release Picard plugin.
      • 2020-05-15 13601, 2020

      • iliekcomputers
        it's mostly because we're better at sql.
      • 2020-05-15 13614, 2020

      • iliekcomputers
        so sql is easier for us to maintain long term.
      • 2020-05-15 13624, 2020

      • Freso
        (That plugin currently has its own code that does basically what your module is doing.)
      • 2020-05-15 13627, 2020

      • ruaok
        and ORMs hate super complex data like our schemas
      • 2020-05-15 13632, 2020

      • ruaok
        well, MB in particular.
      • 2020-05-15 13647, 2020

      • iliekcomputers
        ORM style queries don't really benefit our projects in any real, impactful way
      • 2020-05-15 13658, 2020

      • ishaanshah[m]
        Oh, I have used the pysparks orm style queries
      • 2020-05-15 13632, 2020

      • ishaanshah[m]
      • 2020-05-15 13618, 2020

      • ishaanshah[m]
        I had some trouble working with date time comparisons in raw sql, but orm styled worked flawlessly
      • 2020-05-15 13626, 2020

      • rdswift
        Freso, cool. I hope it works okay for you.
      • 2020-05-15 13635, 2020

      • iliekcomputers
        i am kinda hesitant to do ORM style queries in one place and SQL in others
      • 2020-05-15 13648, 2020

      • iliekcomputers
        what's the problem you faced with date time comparisons?
      • 2020-05-15 13609, 2020

      • ishaanshah[m]
        Invalid syntax was one,
      • 2020-05-15 13633, 2020

      • ishaanshah[m]
        Incompatible data types
      • 2020-05-15 13607, 2020

      • ishaanshah[m]
        I guess, the ORM one handles the type conversion on its own
      • 2020-05-15 13646, 2020

      • alastairp
        hey,
      • 2020-05-15 13647, 2020

      • alastairp
        so
      • 2020-05-15 13606, 2020

      • alastairp
      • 2020-05-15 13641, 2020

      • iliekcomputers
        ishaanshah[m]: a compromise here would be to do the filter in python and then get the actual stats in sql
      • 2020-05-15 13643, 2020

      • alastairp
        I took the listenbrainz artist stats, and some code that one of my coworkers at UPF had written, using matrix factorisation colaborative filtering
      • 2020-05-15 13615, 2020

      • iliekcomputers
        i'm very hesitant to accept ORM code when all the rest of our code uses SQL
      • 2020-05-15 13625, 2020

      • ishaanshah[m]
        Hmm, that sounds reasonable
      • 2020-05-15 13634, 2020

      • ishaanshah[m]
        I agree that SQL is more readable
      • 2020-05-15 13644, 2020

      • alastairp
        it generated this model. given the "train" data, it made these recommendations. The "test" data was held out of the list of data that was given to the model, but we know that the user listened to it. The True/False in the Recommended line is if the recommendation exists in the test set or no
      • 2020-05-15 13602, 2020

      • alastairp
        this data looks really good. I think I can look up recs for a given username too, with a bit of work
      • 2020-05-15 13605, 2020

      • iliekcomputers
        because eventually one or two years down the line, we will have to migrate to one method because it's too complicated.
      • 2020-05-15 13617, 2020

      • iliekcomputers
        which is something I would like to avoid.
      • 2020-05-15 13621, 2020

      • ruaok
        alastairp: one wrong? not bad.
      • 2020-05-15 13627, 2020

      • alastairp
        in addition to this, it spits out artist similarities. here are similar artists to the beatles, using a few different metrics:
      • 2020-05-15 13633, 2020

      • alastairp
        SIMILARS ['The Beatles', 'The Rolling Stones', 'Electric Light Orchestra', 'Simon & Garfunkel', 'Arcade Fire', 'Bob Marley & The Wailers', 'John Lennon', 'The Who', 'Paul McCartney', 'Oasis']
      • 2020-05-15 13633, 2020

      • alastairp
        SIMILARS ['The Beatles', 'John Lennon', 'Simon & Garfunkel', 'Paul McCartney', 'George Harrison', 'The Rolling Stones', 'The Hollies', 'Electric Light Orchestra', 'The Mamas & the Papas', 'Arcade Fire']
      • 2020-05-15 13646, 2020

      • alastairp
        ruaok: right, and even then "wrong" isn't so much the correct word to use here
      • 2020-05-15 13650, 2020

      • ishaanshah[m]
        Yes, I agree
      • 2020-05-15 13600, 2020

      • alastairp
        it's just saying "the algorithm suggests these artists, but you haven't listened to it before"
      • 2020-05-15 13606, 2020

      • alastairp
        so that's a _good_ recommendation!
      • 2020-05-15 13607, 2020

      • ruaok
        let me generate my similar artists to beatles, hang on.
      • 2020-05-15 13616, 2020

      • ishaanshah[m]
        I will filter using ORM and then revert to the old query
      • 2020-05-15 13625, 2020

      • ruaok
        ahhh, I see!
      • 2020-05-15 13630, 2020

      • iliekcomputers
        thanks!
      • 2020-05-15 13646, 2020

      • ishaanshah[m]
        Another questiom
      • 2020-05-15 13618, 2020

      • ishaanshah[m]
      • 2020-05-15 13648, 2020

      • ishaanshah[m]
        replace {artist_range} with :artist_range
      • 2020-05-15 13608, 2020

      • ruaok
        alastairp: so, let me get this right -- based on our top artists, this algorithm can then sort out related artist that others should listen to?
      • 2020-05-15 13630, 2020

      • ishaanshah[m]
        the text query that is generated has is something like this '{'all_time'}'
      • 2020-05-15 13647, 2020

      • alastairp
        based on user + artist listen count, given a different user's artist list, it recomends new artists to listen to
      • 2020-05-15 13647, 2020

      • ishaanshah[m]
        However it should be '{all_time}'
      • 2020-05-15 13605, 2020

      • ishaanshah[m]
        I am getting a syntax error because of this
      • 2020-05-15 13606, 2020

      • alastairp
        well, not so much "new"
      • 2020-05-15 13607, 2020

      • iliekcomputers
        the range is not user input, right?
      • 2020-05-15 13610, 2020

      • Mr_Monkey
        alastairp: That looks promising
      • 2020-05-15 13614, 2020

      • iliekcomputers
        you can just do .format
      • 2020-05-15 13619, 2020

      • iliekcomputers
        if it's not user input
      • 2020-05-15 13632, 2020

      • mzfr joined the channel
      • 2020-05-15 13655, 2020

      • Mr_Monkey
        I feel like the test set is quite wide to really appreciate the quality of the recommendation
      • 2020-05-15 13605, 2020

      • iliekcomputers
        we pass params to sqlalchemy which sanitizes them to avoid sql injection.
      • 2020-05-15 13610, 2020

      • ruaok
        alastairp: so, two artists lists are the input?
      • 2020-05-15 13627, 2020

      • alastairp
        the input to train the model is all listens
      • 2020-05-15 13629, 2020

      • ruaok
        wait. or two user artist histograms?
      • 2020-05-15 13635, 2020

      • ishaanshah[m]
        Hmm, I will try that again
      • 2020-05-15 13649, 2020

      • iliekcomputers
        in this case, i don't think we need that explicitly here, could just use query.format
      • 2020-05-15 13658, 2020

      • alastairp
        and the input to get a recommendation is the listen histogram of the user you want the recommendations for
      • 2020-05-15 13608, 2020

      • ruaok
        got it.
      • 2020-05-15 13614, 2020

      • ruaok
        wow. sweet.
      • 2020-05-15 13617, 2020

      • ishaanshah[m]
        But what about the artist->artist_range part?
      • 2020-05-15 13623, 2020

      • ruaok
        you just made all the work I just finished obsolete!
      • 2020-05-15 13642, 2020

      • CallerNo6 has quit
      • 2020-05-15 13604, 2020

      • alastairp
        yeah, I mentioned an overview of our plans in my lab meeting
      • 2020-05-15 13605, 2020

      • iliekcomputers
        i don't think i completely understand what is happening in the query.
      • 2020-05-15 13616, 2020

      • alastairp
        and coworker said "oh yeah, I did this based on the lastfm listening history dataset"
      • 2020-05-15 13632, 2020

      • alastairp
        (not G's one, an older one that Oscar Celma released 10-15 years ago)
      • 2020-05-15 13643, 2020

      • iliekcomputers
        right now, if you want, you can just get the json from the column, update it in python and insert it back if the sql is getting too complicated.
      • 2020-05-15 13651, 2020

      • iliekcomputers
        i'll have a look more deeply later.
      • 2020-05-15 13616, 2020

      • alastairp
        ruaok: the compilation album metric is sweet though, we should try and polish that idea
      • 2020-05-15 13622, 2020

      • ishaanshah[m]
        Ok, thanks
      • 2020-05-15 13626, 2020

      • ruaok
      • 2020-05-15 13632, 2020

      • ruaok
        results for beatles.
      • 2020-05-15 13638, 2020

      • ishaanshah[m]
        I don't have much experience with SQL
      • 2020-05-15 13645, 2020

      • ruaok
        alastairp: and there is a PR up for that alg.
      • 2020-05-15 13648, 2020

      • alastairp
        looks like "famous UK musicicians and groups"
      • 2020-05-15 13655, 2020

      • iliekcomputers
        well, you'll have it by the time this project ends
      • 2020-05-15 13607, 2020

      • ishaanshah[m]
        I am learning as I go, so please bear with me if I ask silly doubts
      • 2020-05-15 13607, 2020

      • ruaok
        yes, this breaks down real fast. if music wasn't popular, it won't work.
      • 2020-05-15 13609, 2020

      • Mr_Monkey
        Yeah. The Prodigy in that list sticks out like a sore thumb
      • 2020-05-15 13616, 2020

      • ruaok
        your stuff is much better!
      • 2020-05-15 13627, 2020

      • iliekcomputers
        np, as many questions as possible, that's what we're here for.
      • 2020-05-15 13646, 2020

      • alastairp
        so, this algorithm took about 1h30 to train on the full LB dataset that iliekcomputers linked me just after lunch
      • 2020-05-15 13650, 2020

      • ruaok
        oh sorry. that is not beatles. that was portishead.