#metabrainz

/

      • D4RK-PH0ENiX has quit

        • 2017-07-14 19534, 2017

      • D4RK-PH0ENiX joined the channel

        • 2017-07-14 19512, 2017

      • D4RK-PH0ENiX has quit

        • 2017-07-14 19526, 2017

      • Mineo has quit

        • 2017-07-14 19549, 2017

      • D4RK-PH0ENiX joined the channel

        • 2017-07-14 19505, 2017

      • Major_Lurker has quit

        • 2017-07-14 19518, 2017

      • D4RK-PH0ENiX has quit

        • 2017-07-14 19540, 2017

      • D4RK-PH0ENiX joined the channel

        • 2017-07-14 19524, 2017

      • D4RK-PH0ENiX has quit

        • 2017-07-14 19530, 2017

      • D4RK-PH0ENiX joined the channel

        • 2017-07-14 19533, 2017

      • D4RK-PH0ENiX has quit

        • 2017-07-14 19559, 2017

      • D4RK-PH0ENiX joined the channel

        • 2017-07-14 19507, 2017

      • samj1912 has quit

        • 2017-07-14 19511, 2017

      • cjnolet_work
        Is ListenBrainz in a state where we might be able to get a dump of the data?

        • 2017-07-14 19503, 2017

      • MajorLurker joined the channel

        • 2017-07-14 19506, 2017

      • github joined the channel

        • 2017-07-14 19506, 2017

      • github
        [critiquebrainz] gentlecat closed pull request #129: Add support for running PyLint checks in Jenkins + some fixes (master...pylint) https://git.io/vQQlA

        • 2017-07-14 19506, 2017

      • github has left the channel

        • 2017-07-14 19504, 2017

      • Nyanko-sensei joined the channel

        • 2017-07-14 19549, 2017

      • D4RK-PH0ENiX has quit

        • 2017-07-14 19528, 2017

      • d4rkie has quit

        • 2017-07-14 19515, 2017

      • D4RK-PH0ENiX joined the channel

        • 2017-07-14 19549, 2017

      • cjnolet`office has quit

        • 2017-07-14 19538, 2017

      • D4RK-PH0ENiX has quit

        • 2017-07-14 19539, 2017

      • D4RK-PH0ENiX joined the channel

        • 2017-07-14 19531, 2017

      • D4RK-PH0ENiX has quit

        • 2017-07-14 19533, 2017

      • Slurpee has quit

        • 2017-07-14 19522, 2017

      • D4RK-PH0ENiX joined the channel

        • 2017-07-14 19525, 2017

      • d4rkie joined the channel

        • 2017-07-14 19524, 2017

      • D4RK-PH0ENiX has quit

        • 2017-07-14 19544, 2017

      • Mineo joined the channel

        • 2017-07-14 19546, 2017

      • samj1912 joined the channel

        • 2017-07-14 19500, 2017

      • D4RK-PH0ENiX joined the channel

        • 2017-07-14 19548, 2017

      • Nyanko-sensei has quit

        • 2017-07-14 19507, 2017

      • Nyanko-sensei joined the channel

        • 2017-07-14 19509, 2017

      • agentsim has quit

        • 2017-07-14 19543, 2017

      • agentsim joined the channel

        • 2017-07-14 19528, 2017

      • D4RK-PH0ENiX has quit

        • 2017-07-14 19540, 2017

      • kyan joined the channel

        • 2017-07-14 19537, 2017

      • D4RK-PH0ENiX joined the channel

        • 2017-07-14 19516, 2017

      • Nyanko-sensei has quit

        • 2017-07-14 19528, 2017

      • Mineo has quit

        • 2017-07-14 19508, 2017

      • samj1912 has quit

        • 2017-07-14 19530, 2017

      • 18WAAMDJ2 has quit

        • 2017-07-14 19549, 2017

      • ruaok
        cjnolet_work: the dump wouldn't be worth it -- maybe in a month or so once loads of people import their data.

        • 2017-07-14 19528, 2017

      • UmkaDK joined the channel

        • 2017-07-14 19548, 2017

      • CatQuest
        yea, it'd no good just to be based on like 5 people. with idk mostly my data in both me and robs account? :D

        • 2017-07-14 19557, 2017

      • CatQuest
        ruaok: I'm curious: last time (not yesterday) i *did* get the same amount of listens. but not this time? (it said so) do you nkow why it "worked" before?

        • 2017-07-14 19516, 2017

      • CatQuest
        i like how the updated dialogue (I did to make it up to date with the listens doen since i started) now say how many listens there where :D

        • 2017-07-14 19521, 2017

      • CatQuest
        500 exactly :O

        • 2017-07-14 19547, 2017

      • ruaok
        yeah, we have an update to the dialog I haven't merged and pushed live yet for incremental imports

        • 2017-07-14 19515, 2017

      • CatQuest
        ruaok: the listenbrainz dialogue box could have a "close" button once the import it over: in case i don't just want ot go to the profile just want to close it (and importagain f.ex)

        • 2017-07-14 19555, 2017

      • ruaok
        yeah, I think there is a bug for that.

        • 2017-07-14 19504, 2017

      • CatQuest
        huh. it said "Successfully submitted 50" after doing it *again* just after the next one.. but it's only one.. so the part about "Please note that some of these listens might be duplicates " are those 49?

        • 2017-07-14 19509, 2017

      • CatQuest
        hmm but...

        • 2017-07-14 19519, 2017

      • CatQuest
        they *did* get done right that other time...

        • 2017-07-14 19548, 2017

      • ruaok
        iliekcomputers: ^ ?

        • 2017-07-14 19524, 2017

      • CatQuest
        you'll remember i posted the results in here and went all \o/ fireworks

        • 2017-07-14 19540, 2017

      • iliekcomputers
        It sent a page of 50 listens

        • 2017-07-14 19503, 2017

      • iliekcomputers
        Of which 49 might have been dups

        • 2017-07-14 19545, 2017

      • CatQuest
        https://chatlogs.metabrainz.org/brainzbot/metabra…

        • 2017-07-14 19548, 2017

      • CatQuest
        I mean it's gone up to 1,164,064 since then, but I've only been scrobbling from the same box and nothing else fishy for ages now

        • 2017-07-14 19542, 2017

      • ruaok
        LB: 2,858,361 BQ: 2,856,008 :(

        • 2017-07-14 19503, 2017

      • CatQuest
        BQ?

        • 2017-07-14 19518, 2017

      • CatQuest
        also who's that with twice as many listens as me :O

        • 2017-07-14 19533, 2017

      • ruaok
        BigQuery

        • 2017-07-14 19559, 2017

      • CatQuest
        oh this is what you've sent to them?

        • 2017-07-14 19505, 2017

      • ruaok
        yeah

        • 2017-07-14 19506, 2017

      • CatQuest
        hmm so seems there is a problem there too :/

        • 2017-07-14 19512, 2017

      • ruaok
        they should be exactly the same.

        • 2017-07-14 19513, 2017

      • ruaok
        yes. :(

        • 2017-07-14 19519, 2017

      • CatQuest
        :(

        • 2017-07-14 19523, 2017

      • CatQuest
        well lets hope the sun sticks today and not goes away beind a cloud and stays there like yesterday :/

        • 2017-07-14 19523, 2017

      • CatQuest
        and that the analogy carries over to you work with this

        • 2017-07-14 19529, 2017

      • CatQuest
        🙇

        • 2017-07-14 19533, 2017

      • ruaok
        no tenemos sol en cataluyna hoy tampoco. :(

        • 2017-07-14 19514, 2017

      • iliekcomputers
        ruaok: this didn't happen before the new deduplication stuff right?

        • 2017-07-14 19537, 2017

      • iliekcomputers
        Does BigQuery have a defined primary key or similar?

        • 2017-07-14 19540, 2017

      • ruaok
        correct, this is new.

        • 2017-07-14 19501, 2017

      • ruaok
        it does not have a primary key, nor can it do deduplication. which is why we have influx doing it.

        • 2017-07-14 19551, 2017

      • iliekcomputers
        https://github.com/metabrainz/listenbrainz-server…

        • 2017-07-14 19522, 2017

      • ruaok
        the insertId is only used to ensure that inserts work and are not duplicated.

        • 2017-07-14 19540, 2017

      • ruaok
        it is only used for the submission process to BQ

        • 2017-07-14 19520, 2017

      • iliekcomputers
        seems like rows in bigquery with the same insertId can get overwritten sometimes https://stackoverflow.com/questions/38496899/big-…

        • 2017-07-14 19516, 2017

      • ruaok
        hmmm. I see no errors in the bigquery log.

        • 2017-07-14 19531, 2017

      • ruaok
        I doubt that is what is happening. :(

        • 2017-07-14 19531, 2017

      • iliekcomputers
        :(

        • 2017-07-14 19534, 2017

      • iliekcomputers
        insertId should still be unique for each row according to google's documentation though https://cloud.google.com/bigquery/docs/reference/…

        • 2017-07-14 19546, 2017

      • iliekcomputers
        even if that is not what causes the bug

        • 2017-07-14 19508, 2017

      • ruaok
        oh. I see what you're saying. yes, that is likely the problem.

        • 2017-07-14 19527, 2017

      • ruaok
        it should be user + ts + recording_msid. right?

        • 2017-07-14 19545, 2017

      • iliekcomputers
        yeah

        • 2017-07-14 19500, 2017

      • ruaok
        another reset. sigh.

        • 2017-07-14 19505, 2017

      • ruaok
        I'll make it happen.

        • 2017-07-14 19514, 2017

      • ruaok
        can you look at the nitpick comment on the PR?

        • 2017-07-14 19524, 2017

      • iliekcomputers
        yeah sure

        • 2017-07-14 19525, 2017

      • ruaok
        maybe we can knock out another run post haste.

        • 2017-07-14 19559, 2017

      • iliekcomputers
        ruaok: I fixed https://github.com/metabrainz/listenbrainz-server…

        • 2017-07-14 19530, 2017

      • ruaok
        woo!

        • 2017-07-14 19536, 2017

      • github joined the channel

        • 2017-07-14 19536, 2017

      • github
        [listenbrainz-server] mayhem closed pull request #225: Don't show api inconsistency message on incremental imports (master...inc-msg) https://git.io/vQHdd

        • 2017-07-14 19536, 2017

      • github has left the channel

        • 2017-07-14 19507, 2017

      • iliekcomputers
        question: how would we write tests for bigquery-writer for stuff slike this ideally?

        • 2017-07-14 19534, 2017

      • ruaok
        i have no idea. :(

        • 2017-07-14 19522, 2017

      • github joined the channel

        • 2017-07-14 19522, 2017

      • github
        [listenbrainz-server] mayhem opened pull request #226: Improve the BQ insert id and increase font size on scraper dialog message (master...bq-insertid-fix) https://git.io/vQ73H

        • 2017-07-14 19522, 2017

      • github has left the channel

        • 2017-07-14 19529, 2017

      • ruaok
        iliekcomputers: ^^

        • 2017-07-14 19520, 2017

      • iliekcomputers
        Nice, looking :)

        • 2017-07-14 19515, 2017

      • saifulbkhan has quit

        • 2017-07-14 19501, 2017

      • github joined the channel

        • 2017-07-14 19501, 2017

      • github
        [listenbrainz-server] mayhem closed pull request #226: Improve the BQ insert id and increase font size on scraper dialog message (master...bq-insertid-fix) https://git.io/vQ73H

        • 2017-07-14 19501, 2017

      • github has left the channel

        • 2017-07-14 19506, 2017

      • ruaok
        merged. pushed. starting deploy.

        • 2017-07-14 19511, 2017

      • iliekcomputers pinged alastairp about the tests :)

        • 2017-07-14 19537, 2017

      • zas
        Moin

        • 2017-07-14 19549, 2017

      • ruaok
        moin

        • 2017-07-14 19559, 2017

      • zas
        i just did a switch kiki->herb->kiki to upgrade openresty-gateways (https://github.com/metabrainz/docker-openresty/co…)

        • 2017-07-14 19551, 2017

      • ruaok
        iliekcomputers: deployed.

        • 2017-07-14 19502, 2017

      • ruaok
        the dialog now only says "<gear> working". :(

        • 2017-07-14 19512, 2017

      • ruaok
        which is true, but still.

        • 2017-07-14 19551, 2017

      • ruaok
        yeah, happens on my test account as well.

        • 2017-07-14 19548, 2017

      • iliekcomputers
        i might have messed up somewhere

        • 2017-07-14 19512, 2017

      • iliekcomputers
        https://github.com/metabrainz/listenbrainz-server… :(

        • 2017-07-14 19518, 2017

      • iliekcomputers
        welp, let me fix

        • 2017-07-14 19520, 2017

      • iliekcomputers
        sorry!

        • 2017-07-14 19525, 2017

      • ruaok
        no paso nada

        • 2017-07-14 19552, 2017

      • ruaok wonders if it will stop correctly

        • 2017-07-14 19521, 2017

      • github joined the channel

        • 2017-07-14 19521, 2017

      • github
        [listenbrainz-server] paramsingh opened pull request #227: Fix reference to an undefined variable (master...missed-something) https://git.io/vQ7GX

        • 2017-07-14 19521, 2017

      • github has left the channel

        • 2017-07-14 19519, 2017

      • psolanki
        Gentlecat: Hey

        • 2017-07-14 19520, 2017

      • psolanki
        I changed the term 'rating' to 'popularity' except at two places

        • 2017-07-14 19521, 2017

      • psolanki
        ws/review/views - It would be better if I do that when I change ws code in a seperate PR

        • 2017-07-14 19536, 2017

      • github joined the channel

        • 2017-07-14 19536, 2017

      • github
        [listenbrainz-server] mayhem closed pull request #227: Fix reference to an undefined variable (master...missed-something) https://git.io/vQ7GX

        • 2017-07-14 19536, 2017

      • github has left the channel

        • 2017-07-14 19521, 2017

      • psolanki
        and db/users.py - It is used for gravatar url

        • 2017-07-14 19527, 2017

      • agentsim has quit

        • 2017-07-14 19542, 2017

      • ruaok
        restarted. importing again. message appears.

        • 2017-07-14 19545, 2017

      • ruaok
        fingers crossed.

        • 2017-07-14 19512, 2017

      • iliekcomputers
        41,549 listens on both last.fm and LB

        • 2017-07-14 19540, 2017

      • ruaok
        BQ: 264,547 LB: 264,547

        • 2017-07-14 19541, 2017

      • ruaok
        phew.

        • 2017-07-14 19543, 2017

      • ruaok
        !m iliekcomputers

        • 2017-07-14 19543, 2017

      • BrainzBot
        You're doing good work, iliekcomputers!

        • 2017-07-14 19536, 2017

      • Nyanko-sensei joined the channel

        • 2017-07-14 19556, 2017

      • D4RK-PH0ENiX has quit

        • 2017-07-14 19548, 2017

      • iliekcomputers is relieved

        • 2017-07-14 19507, 2017

      • ruaok
        comments, love, hate, whatever?