#metabrainz

/

      • ruaok
        Gentlecat: yea. there was a misunderstanding how timestamps were handled internally.

        • 2015-09-16 25904, 2015

      • ruaok
        this seems another holdover from that.

        • 2015-09-16 25907, 2015

      • ruaok
        fixing now.

        • 2015-09-16 25935, 2015

      • github joined the channel

        • 2015-09-16 25935, 2015

      • github
        [listenbrainz-server] gentlecat pushed 1 new commit to listens-experiment: https://github.com/metabrainz/listenbrainz-server…

        • 2015-09-16 25935, 2015

      • github
        listenbrainz-server/listens-experiment f78b817 Roman Tsukanov: Merge branch 'master' into listens-experiment

        • 2015-09-16 25935, 2015

      • github has left the channel

        • 2015-09-16 25933, 2015

      • ariscop_ joined the channel

        • 2015-09-16 25935, 2015

      • ariscop has quit

        • 2015-09-16 25935, 2015

      • ariscop_ has quit

        • 2015-09-16 25939, 2015

      • github joined the channel

        • 2015-09-16 25939, 2015

      • github
        [listenbrainz-server] gentlecat opened pull request #16: [WIP] Remember current page (timestamp) when switching between pages (master...listens-experiment) https://github.com/metabrainz/listenbrainz-server…

        • 2015-09-16 25939, 2015

      • github has left the channel

        • 2015-09-16 25911, 2015

      • ariscop joined the channel

        • 2015-09-16 25932, 2015

      • ruaok
        \ø/

        • 2015-09-16 25907, 2015

      • alastairp
        ruaok: coming in? already in?

        • 2015-09-16 25918, 2015

      • ruaok
        committing, coming in.

        • 2015-09-16 25955, 2015

      • github joined the channel

        • 2015-09-16 25955, 2015

      • github
        [listenbrainz-server] mayhem pushed 1 new commit to master: https://github.com/metabrainz/listenbrainz-server…

        • 2015-09-16 25955, 2015

      • github
        listenbrainz-server/master a0b1aa4 Robert Kaye: Fixes the server side of the listens branch

        • 2015-09-16 25955, 2015

      • github has left the channel

        • 2015-09-16 25937, 2015

      • ruaok
        http://listenbrainz.org/user/rob

        • 2015-09-16 25943, 2015

      • ruaok
        thanks Gentlecat!

        • 2015-09-16 25926, 2015

      • Gentlecat
        http://listenbrainz.org/listen/user/rob?max_ts=14…

        • 2015-09-16 25949, 2015

      • ruaok
        js or server issue?

        • 2015-09-16 25907, 2015

      • Gentlecat
        https://github.com/metabrainz/listenbrainz-server…

        • 2015-09-16 25900, 2015

      • Gentlecat
        looks like it's this line https://github.com/metabrainz/listenbrainz-server…

        • 2015-09-16 25905, 2015

      • Gentlecat
        I can't reproduce it

        • 2015-09-16 25933, 2015

      • Gentlecat
        "Previous page" is not working basically

        • 2015-09-16 25958, 2015

      • ruaok
        in general or because of this reverse() problem?

        • 2015-09-16 25928, 2015

      • Gentlecat
        because of this problem

        • 2015-09-16 25936, 2015

      • ruaok
        k

        • 2015-09-16 25955, 2015

      • Gentlecat
        oh, I probably can't reproduce it because my VM is completely outdated

        • 2015-09-16 25902, 2015

      • Gentlecat
        schema changes and stuff

        • 2015-09-16 25925, 2015

      • ruaok
        you'll want to truncate your listens table in csqlsh

        • 2015-09-16 25939, 2015

      • Gentlecat
        I'll just create a new one

        • 2015-09-16 25902, 2015

      • ruaok
        k

        • 2015-09-16 25952, 2015

      • Mineo wonders why max_ts is passed to from_id and min_ts to to_id because he would expect to_id to contain the larger of both values

        • 2015-09-16 25928, 2015

      • The_Freso joined the channel

        • 2015-09-16 25943, 2015

      • Freso has quit

        • 2015-09-16 25943, 2015

      • The_Freso is now known as Freso

        • 2015-09-16 25936, 2015

      • ruaok
        feh, too noisy here. off to UPF.

        • 2015-09-16 25945, 2015

      • ruaok has quit

        • 2015-09-16 25919, 2015

      • The_Freso joined the channel

        • 2015-09-16 25937, 2015

      • Freso has quit

        • 2015-09-16 25937, 2015

      • The_Freso is now known as Freso

        • 2015-09-16 25911, 2015

      • ruaok joined the channel

        • 2015-09-16 25949, 2015

      • zas
        ariscop: ok, is the plugin good enough to be merged ?

        • 2015-09-16 25919, 2015

      • ariscop
        *shrug* i use it

        • 2015-09-16 25929, 2015

      • Gentlecat
        ruaok: https://gist.github.com/gentlecat/c8b140e174d4c6a…

        • 2015-09-16 25955, 2015

      • github joined the channel

        • 2015-09-16 25956, 2015

      • github
        [listenbrainz-server] mayhem pushed 1 new commit to master: https://github.com/metabrainz/listenbrainz-server…

        • 2015-09-16 25956, 2015

      • github
        listenbrainz-server/master e0db73c Robert Kaye: Fix the ascending sort order

        • 2015-09-16 25956, 2015

      • github has left the channel

        • 2015-09-16 25901, 2015

      • Mineo
        zas, ariscop: are you talking about https://github.com/musicbrainz/picard-plugins/pul… if so, does it now prevent http://tickets.musicbrainz.org/browse/PICARD-751 from happening?

        • 2015-09-16 25918, 2015

      • ruaok
        Gentlecat: looks like there is a package missing for postgres.

        • 2015-09-16 25925, 2015

      • ruaok
        can you log into the VM and see which one it is?

        • 2015-09-16 25932, 2015

      • ruaok
        then I can try and tweak the chef config.

        • 2015-09-16 25943, 2015

      • Gentlecat
        what do I need to do?

        • 2015-09-16 25959, 2015

      • ruaok
        prev works on the main server now. :)

        • 2015-09-16 25919, 2015

      • ruaok
        there is a package missing that postgres needs.

        • 2015-09-16 25929, 2015

      • ruaok
        see if you can figure out which package needs installing

        • 2015-09-16 25948, 2015

      • ariscop
        Mineo, should work correctly either way

        • 2015-09-16 25929, 2015

      • zas
        Mineo: i merged a fix for PICARD-751 , should be fine now

        • 2015-09-16 25901, 2015

      • Mineo
        zas: except for all the people who're using the stable version of picard

        • 2015-09-16 25911, 2015

      • Gentlecat
        ruaok: https://gist.github.com/gentlecat/0b9a72bfc756d5f…

        • 2015-09-16 25909, 2015

      • ruaok
        does it converge now?

        • 2015-09-16 25934, 2015

      • ariscop
        i haven't tried it against upstream picard, fedora has 1.3.2 in its repos

        • 2015-09-16 25925, 2015

      • Mineo
        well, I don't think a plugin should leave picard in a state where it looks like it's doing something, but is not

        • 2015-09-16 25957, 2015

      • Gentlecat
        ruaok: https://gist.github.com/gentlecat/389883786166837…

        • 2015-09-16 25923, 2015

      • ariscop
        it shouldn't, acousticbrainz plugin is using try/catc/finally, same as the upstream code seems to be

        • 2015-09-16 25933, 2015

      • ruaok
        how did you get two postgres installs?

        • 2015-09-16 25959, 2015

      • ruaok
        was this a fresh VM?

        • 2015-09-16 25902, 2015

      • Gentlecat
        yes

        • 2015-09-16 25955, 2015

      • Gentlecat
        I have no idea where 9.3 came from

        • 2015-09-16 25932, 2015

      • ruaok
        our chef stuff specifcially installs 9.4

        • 2015-09-16 25955, 2015

      • ruaok
        ah, I see basebox installing some postgres stuff

        • 2015-09-16 25958, 2015

      • ruaok investigates

        • 2015-09-16 25915, 2015

      • ruaok
        russss, RJ2: Any idea why postgres 9.3 gets installed here?

        • 2015-09-16 25907, 2015

      • ruaok
        this is what we're using to install right now:

        • 2015-09-16 25907, 2015

      • ruaok
        https://github.com/metabrainz/listenbrainz-server…

        • 2015-09-16 25942, 2015

      • RJ2
        i think i've had this issue with chef attribute ordering before

        • 2015-09-16 25914, 2015

      • RJ2
        doing a converge to a fresh vm now to test

        • 2015-09-16 25945, 2015

      • alastairp
        kepstin: you're lucky you're logged into lastfm developer site. Those who aren't can't...

        • 2015-09-16 25956, 2015

      • alastairp
        I recommend that you never sign out

        • 2015-09-16 25944, 2015

      • ruaok
        RJ2: thanks

        • 2015-09-16 25908, 2015

      • russss joined the channel

        • 2015-09-16 25956, 2015

      • _5moufl joined the channel

        • 2015-09-16 25936, 2015

      • ruaok
        russss, RJ2: quick question about cassandra stuffs.

        • 2015-09-16 25945, 2015

      • RJ2
        sure

        • 2015-09-16 25956, 2015

      • ruaok
        we think it will be useful to store a boolean in the DB if a listen has been imported or if it is a new one.

        • 2015-09-16 25911, 2015

      • ruaok
        the idea is to query for the last imported scrobble from last.fm.

        • 2015-09-16 25922, 2015

      • ruaok
        so that we can do incremental imports.

        • 2015-09-16 25937, 2015

      • ruaok
        and only so the massive import of a whole history once.

        • 2015-09-16 25957, 2015

      • RJ2
        fetching the most recent listen from cassandra would be pretty cheap, that would give you the last date?

        • 2015-09-16 25958, 2015

      • ruaok
        so, if we add this boolean to the cassandra listens table, will be we able to find it easily?

        • 2015-09-16 25912, 2015

      • ruaok
        not always.

        • 2015-09-16 25915, 2015

      • RJ2
        oh sorry i see what you mean

        • 2015-09-16 25929, 2015

      • ruaok
        i fear that this is not a proper usage of cassandra, no?

        • 2015-09-16 25932, 2015

      • RJ2
        so you'd never have to update that field, just for scrobble provenance

        • 2015-09-16 25941, 2015

      • ruaok
        correct.

        • 2015-09-16 25944, 2015

      • RJ2
        yeah that's not a good use of cassandra really

        • 2015-09-16 25904, 2015

      • ruaok
        ok, I was thinking that might be the case.

        • 2015-09-16 25911, 2015

      • russss
        just have a "last imported scrobble" table, userid->timestamp mapping

        • 2015-09-16 25925, 2015

      • RJ2
        yup

        • 2015-09-16 25931, 2015

      • russss
        that saves storing a boolean for every row as well

        • 2015-09-16 25931, 2015

      • ruaok
        in cassandra, or in postgres?

        • 2015-09-16 25934, 2015

      • ruaok
        or redis?

        • 2015-09-16 25947, 2015

      • RJ2
        pg or cass really

        • 2015-09-16 25958, 2015

      • RJ2
        maybe cass so the python inserter can update that table as needed

        • 2015-09-16 25921, 2015

      • RJ2
        look at a "from_lastfm" field in the scrobble json, and have the python issue an insert to the last_imported_scrobble table

        • 2015-09-16 25927, 2015

      • ruaok
        and really it would only have one row per user, right?

        • 2015-09-16 25941, 2015

      • RJ2
        row key would be uid yeah, and only column would be the timestamp

        • 2015-09-16 25952, 2015

      • ruaok
        ok, got it.

        • 2015-09-16 25912, 2015

      • ruaok
        alastairp and I are presenting listenbrainz at the Future Music Forum tomorrow.

        • 2015-09-16 25916, 2015

      • RJ2
        show me the cql for if you want a sanity check

        • 2015-09-16 25921, 2015

      • RJ2
        ah cool

        • 2015-09-16 25925, 2015

      • ruaok
        I'm wondering what things need to be settled before we do that.

        • 2015-09-16 25938, 2015

      • ruaok
        but this sounds like a bit of a task to try and sneak in today.

        • 2015-09-16 25957, 2015

      • ruaok
        this doesn't sound terrible to add after the fact, no?

        • 2015-09-16 25918, 2015

      • ruaok
        where after the fact is once people have stared importing their listen histories.

        • 2015-09-16 25929, 2015

      • RJ2
        yeah seems fine. will be a small table

        • 2015-09-16 25930, 2015

      • alastairp
        we just need to hope that no one goes away tomorrow night and implements listenbrainz listening in their pet music player

        • 2015-09-16 25942, 2015

      • alastairp
        and starts sending non-lastfm sourced data

        • 2015-09-16 25902, 2015

      • RJ2
        would be lots of updates if you insert imported scrobbles one at a time. might be best to just do the update timestamp thing on the last imported lastfm scrobble in the batch as an optimisation

        • 2015-09-16 25903, 2015

      • ruaok points to http://listenbrainz.org/contribute

        • 2015-09-16 25909, 2015

      • ruaok
        see boldface. :)

        • 2015-09-16 25934, 2015

      • demonimin joined the channel

        • 2015-09-16 25934, 2015

      • ruaok
        RJ2: makes sense.

        • 2015-09-16 25909, 2015

      • Freso
        Eep. I missed a typo earlier! D:

        • 2015-09-16 25914, 2015

      • Freso prepares a new PR

        • 2015-09-16 25955, 2015

      • alastairp
        we have 2 more questions for everyone -

        • 2015-09-16 25904, 2015

      • alastairp
        1) what colour should we paint the bike shed?

        • 2015-09-16 25922, 2015

      • alastairp
        2) should we have a versioned namespace for api queries?

        • 2015-09-16 25936, 2015

      • ruaok
        mauve!

        • 2015-09-16 25941, 2015

      • Freso
        1) Metallic green.

        • 2015-09-16 25941, 2015

      • ruaok
        wait, what?

        • 2015-09-16 25943, 2015

      • Freso
        2) Yes.

        • 2015-09-16 25943, 2015

      • ruaok
        orange!

        • 2015-09-16 25905, 2015

      • RJ2
        i like api.example.com/1.0/blah for api paths

        • 2015-09-16 25908, 2015

      • alastairp
        3) what should the namespace look like

        • 2015-09-16 25915, 2015

      • RJ2
        since it's really obvious what version you're on

        • 2015-09-16 25939, 2015

      • RJ2
        and easy to send requests to various backends from the load balancer, when the version is in the url

        • 2015-09-16 25956, 2015

      • ruaok
        http://tickets.musicbrainz.org/browse/LB-31

        • 2015-09-16 25910, 2015

      • ruaok
        +1 to that RJ

        • 2015-09-16 25934, 2015

      • ruaok
        right now the API lives at /listen/user/<username> GET and POST

        • 2015-09-16 25954, 2015

      • ruaok
        /api/v1/listen/user/<user> ?

        • 2015-09-16 25901, 2015

      • ruaok
        as a new version?

        • 2015-09-16 25916, 2015

      • RJ2
        well, listens of a user would be under /1.0/user/RJ/listens imo

        • 2015-09-16 25939, 2015

      • RJ2
        also: /1.0/artist/$artid/listens etc