#metabrainz

/

      • CatQuest
        weird, and a pity about not spain though :(

        • 2017-01-23 02321, 2017

      • CatQuest
        but ticektmaster using mb data is hella great :D

        • 2017-01-23 02300, 2017

      • reosarevok
        Leo_Verto: also, you mean Finns. "fins" are the ones that move a fish through the water. "Finns" are the ones that move vodka through themselves

        • 2017-01-23 02320, 2017

      • CatQuest
        lmao

        • 2017-01-23 02302, 2017

      • iliekcomputers joined the channel

        • 2017-01-23 02326, 2017

      • iliekcomputers
        mayhem: ping, I was hoping you could tell me what work you've done in your big-query branch on listenbrainz and what remains, so maybe I could help out with the branch a little.

        • 2017-01-23 02359, 2017

      • mayhem
        sure.

        • 2017-01-23 02303, 2017

      • iliekcomputers
        kinda bored of fixing minor bugs :P

        • 2017-01-23 02316, 2017

      • mayhem
        the main point of work was to shovel the data to bigquery.

        • 2017-01-23 02329, 2017

      • mayhem
        but, bigquery cannot dedup the incoming listens.

        • 2017-01-23 02344, 2017

      • mayhem
        and postgres isn't a long term viable data storage.

        • 2017-01-23 02358, 2017

      • reosarevok
        Heh, I don't think I want to go there, but nice poster. http://www.nummirock.fi/

        • 2017-01-23 02330, 2017

      • mayhem
        based on positive feedback on influxfb, I decided to use influxdb as a way to keep the master archival copy of the data and de-dep it at the same time.

        • 2017-01-23 02300, 2017

      • mayhem
        and kafka sucks ass. anything java related ends up being a pain in the ass.

        • 2017-01-23 02305, 2017

      • mayhem
        so I tossed that out as well.

        • 2017-01-23 02328, 2017

      • mayhem
        so the bigquery branch has a misleading name, overall. it should be "big-ass-overhaul" really

        • 2017-01-23 02308, 2017

      • mayhem
        I replaced kafka with a custom redis construction.

        • 2017-01-23 02319, 2017

      • mayhem draws a diagram

        • 2017-01-23 02343, 2017

      • Leo_Verto
        in mspaint?

        • 2017-01-23 02347, 2017

      • iliekcomputers
        mayhem: if I understand correctly: user -> listenbrainz -> influx for dedup -> bigquery is the plan? please correct me if i'm wrong

        • 2017-01-23 02343, 2017

      • mayhem
        https://usercontent.irccloud-cdn.com/file/CU33ZXL…

        • 2017-01-23 02357, 2017

      • mayhem
        pretty much.

        • 2017-01-23 02306, 2017

      • mayhem
        the influx bubble is the master archive.

        • 2017-01-23 02345, 2017

      • iliekcomputers
        mayhem: okay, makes sense.

        • 2017-01-23 02348, 2017

      • mayhem
        great.

        • 2017-01-23 02351, 2017

      • mayhem
        now, what is left?

        • 2017-01-23 02302, 2017

      • mayhem
        the test system pre-dates the docker files.

        • 2017-01-23 02314, 2017

      • mayhem
        so the test system needs to be moved to docker, which is where I am now.

        • 2017-01-23 02318, 2017

      • mayhem
        are you docker versed?

        • 2017-01-23 02346, 2017

      • iliekcomputers
        mayhem: don't have that much experience but if you point me to what to do, I'm sure I could do it.

        • 2017-01-23 02315, 2017

      • mayhem
        ok, that is one option. but lets explore some more.

        • 2017-01-23 02335, 2017

      • mayhem
        the other thing that needs doing is import from listenbrainz (alpha).

        • 2017-01-23 02349, 2017

      • mayhem
        there are some people who have been scrobbling to LB, even in its alpha state.

        • 2017-01-23 02310, 2017

      • iliekcomputers
        yeah i've been scrobbling too

        • 2017-01-23 02312, 2017

      • mayhem
        the current DB is inaccurate due to the second resolution bug from the last.fm web pages vs the last.fm API

        • 2017-01-23 02333, 2017

      • mayhem
        so, migrating the DB is not a good idea.

        • 2017-01-23 02351, 2017

      • mayhem
        instead what we need to do is offer an option for someone to import from the LB alpha version into LB beta version.

        • 2017-01-23 02303, 2017

      • mayhem
        which entails reading all of the listens from a given user and importing them into LB.

        • 2017-01-23 02321, 2017

      • mayhem
        see what I mean?

        • 2017-01-23 02303, 2017

      • iliekcomputers
        mayhem: reading from where? the current listenbrainz db?

        • 2017-01-23 02318, 2017

      • mayhem
        correct.

        • 2017-01-23 02339, 2017

      • iliekcomputers
        okay, that seems doable

        • 2017-01-23 02342, 2017

      • mayhem
        at some point when we release this, I'll move listebrainz.org to alpha.listenbrainz.org and release the beta version under listenbrainz.org

        • 2017-01-23 02309, 2017

      • mayhem
        so, if you would like to contribute new code, writing this import function would be a really great asset.

        • 2017-01-23 02303, 2017

      • iliekcomputers
        mayhem: just so I'm completely clear, what we need is an option in the beta that reads from the current postgres db and then sends it to influx, yes?

        • 2017-01-23 02322, 2017

      • mayhem
        I would set it up on a higher level.

        • 2017-01-23 02343, 2017

      • mayhem
        meaning that I would do it on an HTTP basis.

        • 2017-01-23 02350, 2017

      • mayhem
        it could even be a separate web app, really.

        • 2017-01-23 02306, 2017

      • mayhem
        fetch all the listens from one and submit to the new one.

        • 2017-01-23 02322, 2017

      • iliekcomputers
        a seperate web app that fetches from the listenbrainz api and then submits to the beta?

        • 2017-01-23 02330, 2017

      • mayhem
        yes.

        • 2017-01-23 02337, 2017

      • iliekcomputers
        okay

        • 2017-01-23 02347, 2017

      • mayhem
        or we could include it into the LB app -- doesn't really matter too much.

        • 2017-01-23 02358, 2017

      • iliekcomputers
        mayhem: but how do I submit listens to the beta right now?

        • 2017-01-23 02301, 2017

      • mayhem
        I guess including it is slightly easier for users to understand.

        • 2017-01-23 02315, 2017

      • mayhem
        you can't, since that service isn't up.

        • 2017-01-23 02328, 2017

      • mayhem
        but, you can start your own copy.

        • 2017-01-23 02339, 2017

      • mayhem
        checkout the bigquery branch and run it.

        • 2017-01-23 02357, 2017

      • iliekcomputers
        right, indeed.

        • 2017-01-23 02358, 2017

      • iliekcomputers
        mayhem: I think I get it, I think I'll start with a new app and then we'll see about moving it into listenbrainz

        • 2017-01-23 02321, 2017

      • SothoTalKer
        time to up the ante a bit.

        • 2017-01-23 02330, 2017

      • mayhem
        as long as they are both flask apps, that should work.

        • 2017-01-23 02337, 2017

      • mayhem
        and as I said earlier, I am out this week.

        • 2017-01-23 02355, 2017

      • mayhem
        but hopefully next week I can put my head down for a few days and get the BQ branch ready for review.

        • 2017-01-23 02301, 2017

      • SothoTalKer
        enjoy your holidays =)

        • 2017-01-23 02305, 2017

      • iliekcomputers
        mayhem: that's okay, I'll probably need a week to get it done anyways

        • 2017-01-23 02314, 2017

      • iliekcomputers
        Have fun!

        • 2017-01-23 02316, 2017

      • mayhem
        iliekcomputers: perfect.

        • 2017-01-23 02320, 2017

      • iliekcomputers
        mayhem: thanks for the help.

        • 2017-01-23 02330, 2017

      • mayhem
        holidays. :) I'm going to a 1.5 day conference in Chicago.

        • 2017-01-23 02345, 2017

      • mayhem
        what a stooopdily long way to go for 1.5 days. :(

        • 2017-01-23 02302, 2017

      • iliekcomputers
        what conference is it?

        • 2017-01-23 02310, 2017

      • mayhem
        ordcamp.com

        • 2017-01-23 02343, 2017

      • iliekcomputers
        > 250 hand-picked attendees

        • 2017-01-23 02348, 2017

      • iliekcomputers
        seems exclusive :)

        • 2017-01-23 02310, 2017

      • gcilou
        buut chicago!

        • 2017-01-23 02310, 2017

      • mayhem
        hard to get into, yes.

        • 2017-01-23 02326, 2017

      • mayhem
        but worth the trip. even in jan-fucking-uary!

        • 2017-01-23 02340, 2017

      • dboys_ has quit

        • 2017-01-23 02326, 2017

      • SothoTalKer
        Musicbrainz: would it be possible to make a checkbox for "video" on recordings when you edit a release?

        • 2017-01-23 02303, 2017

      • iliekcomputers
        mayhem: just a small question, asking for the listenbrainz user token in external apps isn't a security no-no, right? the token is made to be shared with apps?

        • 2017-01-23 02342, 2017

      • mayhem
        totally fine.

        • 2017-01-23 02315, 2017

      • mayhem
        and dealing with this is probably a good reason to not have a separate app. just keep it in one... makes it easier for the user too...

        • 2017-01-23 02327, 2017

      • reosarevok
        SothoTalKer: IIRC there's already a ticket

        • 2017-01-23 02352, 2017

      • iliekcomputers
        yeah, you're right.

        • 2017-01-23 02308, 2017

      • mayhem
        just base your work off the bigquery branch.

        • 2017-01-23 02312, 2017

      • iliekcomputers
        okay

        • 2017-01-23 02334, 2017

      • dboys_ joined the channel

        • 2017-01-23 02330, 2017

      • r_dunn joined the channel

        • 2017-01-23 02359, 2017

      • mihaitish has quit

        • 2017-01-23 02332, 2017

      • Gentlecat
        anyone knows what https://github.com/mineo/mb-rngpy is?

        • 2017-01-23 02351, 2017

      • mayhem
        musicbrainz relax ng python something your mom?

        • 2017-01-23 02314, 2017

      • SothoTalKer
        it's a random number generator, of course :D

        • 2017-01-23 02321, 2017

      • alastairp
        mayhem: UK, if there are any

        • 2017-01-23 02347, 2017

      • alastairp
        looks like there's one more spare

        • 2017-01-23 02312, 2017

      • Wildwatz joined the channel

        • 2017-01-23 02323, 2017

      • hibiscuskazeneko joined the channel

        • 2017-01-23 02343, 2017

      • ferbncode has quit

        • 2017-01-23 02315, 2017

      • iliekcomputers has quit

        • 2017-01-23 02327, 2017

      • iliekcomputers joined the channel

        • 2017-01-23 02329, 2017

      • lazka joined the channel

        • 2017-01-23 02322, 2017

      • iliekcomputers has quit

        • 2017-01-23 02348, 2017

      • samj1912 has quit

        • 2017-01-23 02310, 2017

      • samj1912 joined the channel

        • 2017-01-23 02301, 2017

      • Mineo joined the channel

        • 2017-01-23 02338, 2017

      • neeraj_ joined the channel

        • 2017-01-23 02314, 2017

      • Quesito giggling at

        • 2017-01-23 02330, 2017

      • Quesito giggling at your mom....

        • 2017-01-23 02357, 2017

      • mayhem
        ohh, you dig yer mom jokes?

        • 2017-01-23 02311, 2017

      • mayhem
        I need someone to enjoy those with.

        • 2017-01-23 02315, 2017

      • Quesito
        of course!

        • 2017-01-23 02325, 2017

      • Quesito
        who doesn't?

        • 2017-01-23 02337, 2017

      • mayhem
        loads of people here only know them because of me.

        • 2017-01-23 02344, 2017

      • mayhem
        but they don't really *know* them.

        • 2017-01-23 02356, 2017

      • mayhem
        except reosarevok's mom. she seems to get it.

        • 2017-01-23 02316, 2017

      • Quesito
        LOL

        • 2017-01-23 02345, 2017

      • mayhem
        check your sent mail folder and see if a convo matches?

        • 2017-01-23 02305, 2017

      • mayhem
        and yes, this is not an exact science where you turn over product after payment.

        • 2017-01-23 02307, 2017

      • Quesito
        come on.....your mom jokes are a solid portion of important history that should be referenced as often as possible....

        • 2017-01-23 02316, 2017

      • mayhem
        Quesito: <3

        • 2017-01-23 02326, 2017

      • mayhem
        I <3 and your mom.

        • 2017-01-23 02312, 2017

      • Quesito
        I <3 your mom too.

        • 2017-01-23 02324, 2017

      • Quesito
        I got nothing--nothing is adding up with this cat.

        • 2017-01-23 02354, 2017

      • Quesito
        I'm just digging as I don't buy what he's selling.

        • 2017-01-23 02309, 2017

      • Quesito
        but as he used my name....it's recent?

        • 2017-01-23 02311, 2017

      • mayhem
        ask him what his product is

        • 2017-01-23 02330, 2017

      • mayhem
        must be. I was kinda surprised to see it him mention you.

        • 2017-01-23 02351, 2017

      • Quesito
        yeah--no matching names, websites, email addresses....nothing.

        • 2017-01-23 02323, 2017

      • mayhem
        email him and start a convo. see if we're being paranoid or if he remains cagey.

        • 2017-01-23 02334, 2017

      • Quesito
        I put a message on the communtiy thread--but will email as well.

        • 2017-01-23 02314, 2017

      • agentsim_ joined the channel

        • 2017-01-23 02329, 2017

      • agentsim has quit

        • 2017-01-23 02357, 2017

      • kyan joined the channel

        • 2017-01-23 02305, 2017

      • Mineo
        someone should give Quesito "MetaBrainz something something" titles on community.meb

        • 2017-01-23 02330, 2017

      • Mineo
        +a *title

        • 2017-01-23 02346, 2017

      • SothoTalKer
        arrrr

        • 2017-01-23 02331, 2017

      • reosarevok
        Mineo (and Quesito): done!

        • 2017-01-23 02301, 2017

      • Quesito
        I have a title now? I just got my first badge...things are moving so quickly! :)

        • 2017-01-23 02327, 2017

      • Gentlecat
        I see it's one of the dependencies for SIR

        • 2017-01-23 02356, 2017

      • Gentlecat
        *ugh, got disconnected*

        • 2017-01-23 02305, 2017

      • Gentlecat
        Mineo: https://github.com/mineo/mb-rngpy ^

        • 2017-01-23 02321, 2017

      • Gentlecat
        can you clarify what this does?

        • 2017-01-23 02318, 2017

      • Mineo
        it's basically the mmd-schema package for python

        • 2017-01-23 02351, 2017

      • Mineo
        it's a bunch of classes for all the things defined in the xml schema that can be serialized to xml

        • 2017-01-23 02357, 2017

      • Gentlecat
        ah

        • 2017-01-23 02340, 2017

      • Gentlecat
        so it needs an update after the last schema change, right?

        • 2017-01-23 02329, 2017

      • Gentlecat
        and can we move to metabrainz org or is that something you built on your own?

        • 2017-01-23 02336, 2017

      • Mineo
        we can move it

        • 2017-01-23 02348, 2017

      • Mineo
        Gentlecat: can you create repositories in the meb organization? I need to transfer it to someone who can do that first

        • 2017-01-23 02359, 2017

      • Gentlecat
        yes