#metabrainz

/

      • alastairp
        but "version" in the highlevel has both ll and hl version data
      • 2016-10-31 30549, 2016

      • alastairp
        OK, super
      • 2016-10-31 30507, 2016

      • alastairp
        OK. those were the 3 main questions I had (which I emailed about)
      • 2016-10-31 30552, 2016

      • ruaok
        yeah, I think the answer for #3 comes out of everthing we've already said.
      • 2016-10-31 30507, 2016

      • ruaok
        load, make new tables based on query patterns, go.
      • 2016-10-31 30516, 2016

      • ruaok
        kinda like what we were expecting with cassandra.
      • 2016-10-31 30531, 2016

      • ruaok
        duplication and adding more columns is the solution to a lot of problems.
      • 2016-10-31 30536, 2016

      • ruaok
        storage is cheap/free.
      • 2016-10-31 30536, 2016

      • alastairp
        yeah
      • 2016-10-31 30537, 2016

      • alastairp
        oh
      • 2016-10-31 30540, 2016

      • ruaok
        yes?
      • 2016-10-31 30550, 2016

      • alastairp
        one more thing, though I think that "storage is cheap" answers it
      • 2016-10-31 30501, 2016

      • Freso
        CatQuest CatCat, the meeting's in an hour and 6 minutes.
      • 2016-10-31 30502, 2016

      • alastairp
        currently in the schema I have 1 column per tag name
      • 2016-10-31 30521, 2016

      • alastairp
        e.g. title, artist mbid, release date
      • 2016-10-31 30525, 2016

      • Freso
        ruaok: Pono player?
      • 2016-10-31 30537, 2016

      • alastairp
        so they are specified one by one in the schema
      • 2016-10-31 30513, 2016

      • ruaok
        ingest all the data, then make a new table with each tag being in one column.
      • 2016-10-31 30516, 2016

      • alastairp
        we have a whitelist of tags in the extractor, so we are pretty confident that we won't add many more
      • 2016-10-31 30541, 2016

      • alastairp
        we have a small number of tags with spaces in them
      • 2016-10-31 30550, 2016

      • alastairp
        but bq doesn't allow spaces in column names
      • 2016-10-31 30500, 2016

      • alastairp
        I guess that falls under "massage data until it fits"
      • 2016-10-31 30539, 2016

      • ruaok
        f says: Don't like spaces, use _
      • 2016-10-31 30501, 2016

      • alastairp
        OK. we can change to _. this is just because this is what we got straight out of taglib
      • 2016-10-31 30504, 2016

      • alastairp
        can you clarify
      • 2016-10-31 30516, 2016

      • alastairp
        7:55 PM <ruaok> ingest all the data, then make a new table with each tag being in one column.
      • 2016-10-31 30525, 2016

      • alastairp
        why a new table?
      • 2016-10-31 30533, 2016

      • alastairp
        one second. let me upload my schema
      • 2016-10-31 30549, 2016

      • alastairp
      • 2016-10-31 30533, 2016

      • alastairp
        in here, I have tags as part of a subfield (highlevel->metadata->tags)
      • 2016-10-31 30544, 2016

      • alastairp
        is it better to flatten this data into multiple tables??
      • 2016-10-31 30501, 2016

      • ruaok
        possibly,
      • 2016-10-31 30505, 2016

      • ruaok
        depends on the use case.
      • 2016-10-31 30507, 2016

      • alastairp
        e.g. have a tags table and index it using the same postgres rowid?
      • 2016-10-31 30528, 2016

      • ruaok
        more columns or more tables. depending on the query needs.
      • 2016-10-31 30511, 2016

      • ruaok
        again, possibly.
      • 2016-10-31 30534, 2016

      • ruaok
        the biggest thing that you and I were missing is the uploadable JS snippets and the transformational aspects of how to work with BQ.
      • 2016-10-31 30552, 2016

      • ruaok
        now that I get this pattern, everything else makes more sense to me.
      • 2016-10-31 30505, 2016

      • ruaok
        lets have a chat when I get home and see if we can sort out our schemas.
      • 2016-10-31 30510, 2016

      • alastairp
        cool
      • 2016-10-31 30521, 2016

      • alastairp
        gracias rob. gracias felipe!
      • 2016-10-31 30542, 2016

      • ruaok
        de nada!
      • 2016-10-31 30555, 2016

      • ruaok
        ok, time for food.
      • 2016-10-31 30531, 2016

      • demonimin has quit
      • 2016-10-31 30508, 2016

      • Freso
        ruaok: GCI task: Create a formal MetaBrainz letterhead?
      • 2016-10-31 30552, 2016

      • demonimin joined the channel
      • 2016-10-31 30552, 2016

      • demonimin has quit
      • 2016-10-31 30552, 2016

      • demonimin joined the channel
      • 2016-10-31 30518, 2016

      • CatQuest
        ooh printed offical stationary? ooohhh
      • 2016-10-31 30532, 2016

      • CatQuest
        I'd want a letter from metabrainz the :O
      • 2016-10-31 30535, 2016

      • CatQuest
        then*
      • 2016-10-31 30523, 2016

      • Freso
        CatQuest: Naw, not printed. But just something for when writing letter-type documents that are not e-mails.
      • 2016-10-31 30542, 2016

      • CatQuest
        yes? or aw
      • 2016-10-31 30512, 2016

      • CatQuest
        it's easy to make pre-printed letters though, (with the logo on and thne you could write on it with a printer or hand)
      • 2016-10-31 30529, 2016

      • CatQuest
        anyway I'd still want that
      • 2016-10-31 30534, 2016

      • Freso
        (E.g., I just submitted a college recommendation for gcilou which wanted me to attach an external document describing her a bit more in-depth. Would have been neat to have used an "MeB letterhead" for that. Just to make it a little more official, y'know.
      • 2016-10-31 30544, 2016

      • Freso
        )
      • 2016-10-31 30557, 2016

      • gcilou
        😉
      • 2016-10-31 30524, 2016

      • CatQuest thinks he doesn't actually quite know what "letterhead" means really, but doesn't care to be elaborated further. it cn wit, in any case I aprove of the gci task!
      • 2016-10-31 30533, 2016

      • Freso
        :)
      • 2016-10-31 30534, 2016

      • CatQuest
        it can wait*
      • 2016-10-31 30507, 2016

      • kyan joined the channel
      • 2016-10-31 30551, 2016

      • Zastai joined the channel
      • 2016-10-31 30559, 2016

      • Freso has snacks
      • 2016-10-31 30501, 2016

      • Freso
        <BANG>
      • 2016-10-31 30509, 2016

      • Freso
        Hi!
      • 2016-10-31 30520, 2016

      • Freso
        CatCat, starting now. :)
      • 2016-10-31 30539, 2016

      • Freso
        So. Let's start off with zas today!
      • 2016-10-31 30500, 2016

      • zas
        hmm k
      • 2016-10-31 30555, 2016

      • zas
        i improved the openresty configuration, so ssl certs and autossl works
      • 2016-10-31 30515, 2016

      • zas
        i also prepared a rate limiter for ws, using redis as backend
      • 2016-10-31 30537, 2016

      • zas
        anonymous ftp and rsyncd are ready too
      • 2016-10-31 30538, 2016

      • Freso
        (Other people to go: Freso ruaok? bitmap(?) Gentlecat reosarevok(-whenever-he's-around…) CatCat/CatQuest Leftmost? LordSputnik? Leo_Verto? chirlu? ...)
      • 2016-10-31 30549, 2016

      • Leftmost is present.
      • 2016-10-31 30556, 2016

      • zas
        plus tons of minor things related to our new setup
      • 2016-10-31 30558, 2016

      • reosarevok
        I'm around at normal times again :)
      • 2016-10-31 30512, 2016

      • Freso
        reosarevok: 🙌
      • 2016-10-31 30516, 2016

      • CatQuest
        (freso you don't have to ping catcat anymore)
      • 2016-10-31 30524, 2016

      • CatQuest
        thnak you
      • 2016-10-31 30526, 2016

      • zas
        i'm also preparing monitoring stuff
      • 2016-10-31 30547, 2016

      • zas
        i'll work on this tomorrow so we can actually have a better view of new architecture
      • 2016-10-31 30552, 2016

      • Freso
        zas: Do you feel like you're on or behind/ahead of track?
      • 2016-10-31 30553, 2016

      • bitmap
        I'm on my phone waiting for a train
      • 2016-10-31 30505, 2016

      • zas
        go bitmap
      • 2016-10-31 30513, 2016

      • zas
        Freso: behind
      • 2016-10-31 30521, 2016

      • Freso
        zas: :(
      • 2016-10-31 30553, 2016

      • zas
        bitmap: ?
      • 2016-10-31 30510, 2016

      • zas
        reosarevok: ?
      • 2016-10-31 30515, 2016

      • Freso
        Maybe let's go reosarevok instead and bitmap can go whenever he has a connection.
      • 2016-10-31 30522, 2016

      • reosarevok
        Support. Basically nothing else this week
      • 2016-10-31 30535, 2016

      • reosarevok
        Because it's end-of-period aka all my homework is due :D
      • 2016-10-31 30554, 2016

      • reosarevok
        (well, that and keeping up with subscriptions, that keeps taking at least an hour if not more every day...)
      • 2016-10-31 30508, 2016

      • reosarevok
        (I need to subscribe to less things and/or people need to get better at classical)
      • 2016-10-31 30520, 2016

      • reosarevok
        But fin for now. bitmap: around now? if not, CatQuest
      • 2016-10-31 30529, 2016

      • CatQuest
        :O
      • 2016-10-31 30541, 2016

      • CatQuest
        hi
      • 2016-10-31 30542, 2016

      • CatQuest
        ok so
      • 2016-10-31 30543, 2016

      • CatQuest
        with my schoolwork down the drain, (infact I had a meeting about that today) I now have more time to do instruments! I plan on going to the library on tuesdays, wednesdays and thursdays, so on those 3 days I should be able to get done a few instruments. I've also lent music instrumentbooks from the library which I plan on reading and incorporating into new (and already added) instruments.
      • 2016-10-31 30515, 2016

      • CatQuest
        de' e' bærre lækkert.
      • 2016-10-31 30524, 2016

      • CatQuest
        (do we do "fin" anymore?)
      • 2016-10-31 30536, 2016

      • Freso
        CatQuest: you say the next one to go :)
      • 2016-10-31 30541, 2016

      • CatQuest
        oh
      • 2016-10-31 30546, 2016

      • CatQuest
        uhm.. freso? oh wait...
      • 2016-10-31 30553, 2016

      • Freso
        Sure!
      • 2016-10-31 30535, 2016

      • Freso
        I got sick at the beginning of last week.
      • 2016-10-31 30551, 2016

      • Freso
        I finally was alright on Friday... and then I got hit with the flu on Saturday.
      • 2016-10-31 30502, 2016

      • reosarevok
        :(
      • 2016-10-31 30505, 2016

      • CatQuest
        :(
      • 2016-10-31 30509, 2016

      • Freso
        So I haven't done as much as I wanted to this week - mostly been hanging around IRC and forums.
      • 2016-10-31 30515, 2016

      • reosarevok
        Spain is supposed to have nice weather, what are you getting sick from! :p
      • 2016-10-31 30531, 2016

      • Freso
        And I battled a bit with upgrading Discourse, which eventually resolved itself.
      • 2016-10-31 30512, 2016

      • Freso
        I've also written a college recommendation letter for gcilou, so it'll clearly be me that's the reason for her getting in anywhere now.
      • 2016-10-31 30525, 2016

      • Freso
        (Obviously I'm not to blame at all if she doesn't.)
      • 2016-10-31 30525, 2016

      • gcilou
        🙌 of course
      • 2016-10-31 30556, 2016

      • Freso
        And I had lunch with alastairp earlier today. Was nice to see him in the office for once, even if we sat in separate rooms. ;)
      • 2016-10-31 30525, 2016

      • Freso
        I think that's it.
      • 2016-10-31 30530, 2016

      • alastairp
        I'm not sure if you're implying I should be there more often... it's not like you pay me :D
      • 2016-10-31 30535, 2016

      • Freso
        Leftmost: Everybody go right!
      • 2016-10-31 30540, 2016

      • Leftmost
        Woo!
      • 2016-10-31 30550, 2016

      • CatQuest goes left
      • 2016-10-31 30502, 2016

      • CatQuest
        LeftmostCatQuest?
      • 2016-10-31 30506, 2016

      • chirlu has quit
      • 2016-10-31 30525, 2016

      • CatQuest
        chirlu went a bit too much left and/or right
      • 2016-10-31 30535, 2016

      • Leftmost
        I attended the GSoC Mentor Summit. There were a lot of great sessions, and I went to some solid ones on UX/design and on documentation, which both gave me some things to think about for the near future of BB.
      • 2016-10-31 30538, 2016

      • reosarevok
        He went too right and he left? :p
      • 2016-10-31 30542, 2016

      • reosarevok shuts up now
      • 2016-10-31 30546, 2016

      • chirlu joined the channel
      • 2016-10-31 30552, 2016

      • CatQuest hahahaHA
      • 2016-10-31 30510, 2016

      • Leftmost
        I also put together a document on the minimally-viable product for BB and what our focuses ought to be.
      • 2016-10-31 30517, 2016

      • CatQuest
        hmm
      • 2016-10-31 30520, 2016

      • Leftmost
        A few cleanup PRs as well and some review.
      • 2016-10-31 30526, 2016

      • Freso
        (Other people to go: ruaok? bitmap(?) Gentlecat LordSputnik? Leo_Verto? chirlu? ...)
      • 2016-10-31 30530, 2016

      • LordSputnik is here
      • 2016-10-31 30530, 2016

      • Leftmost
        Fin. Gentlecat?
      • 2016-10-31 30549, 2016

      • Freso
        Leftmost: No GCI things? :/
      • 2016-10-31 30504, 2016

      • CatQuest
        Freso: ruaok said he probably couldn't make the meeting
      • 2016-10-31 30513, 2016

      • Gentlecat
        I fixed issues with dockerized version of picard website and deployed it https://picard.musicbrainz.org/
      • 2016-10-31 30553, 2016

      • Leftmost
        Freso, as in, I didn't do any GCI things or didn't attend any GCI things?
      • 2016-10-31 30554, 2016

      • Gentlecat
        worked on CAA redirect a bit more, trying to fix tests
      • 2016-10-31 30501, 2016

      • Freso
        CatQuest: I know. Hence the question mark.
      • 2016-10-31 30537, 2016

      • Gentlecat
        wrote startup scripts for all services that I've been working on
      • 2016-10-31 30508, 2016

      • Gentlecat
        moved CB to new hosting https://critiquebrainz.org/
      • 2016-10-31 30520, 2016

      • Freso
        🙌
      • 2016-10-31 30530, 2016

      • Freso
        How many things are still on OldHost?
      • 2016-10-31 30534, 2016

      • Gentlecat
        and fixed Sentry integration with bitmap's help
      • 2016-10-31 30547, 2016

      • Gentlecat
      • 2016-10-31 30552, 2016

      • Gentlecat
        fin
      • 2016-10-31 30557, 2016

      • Gentlecat
        chirlu?
      • 2016-10-31 30523, 2016

      • Freso
        bitmap briefly made an appearance, but no one else has spoken up, so let's move on.
      • 2016-10-31 30539, 2016

      • Freso
        Freso: Cancel next week's meeting!