#metabrainz

/

      • samj1912 would've had to rethink his PR otherwise
      • 2017-12-22 35614, 2017

      • gcilou joined the channel
      • 2017-12-22 35641, 2017

      • heyoni joined the channel
      • 2017-12-22 35625, 2017

      • yvanzo
        samj1912: confirmed
      • 2017-12-22 35628, 2017

      • LordSputnik
        Leftmost: what do you mean?
      • 2017-12-22 35614, 2017

      • Leftmost
        There's no state passed down to the IdentifierRow component. It just creates a row for every identifier with generic props.
      • 2017-12-22 35641, 2017

      • S_Pradhan joined the channel
      • 2017-12-22 35649, 2017

      • S_Pradhan
        Hi
      • 2017-12-22 35602, 2017

      • S_Pradhan
        Can you please help me out
      • 2017-12-22 35618, 2017

      • reosarevok
        Maybe. What with_
      • 2017-12-22 35619, 2017

      • reosarevok
        ?
      • 2017-12-22 35644, 2017

      • S_Pradhan has quit
      • 2017-12-22 35623, 2017

      • Lotheric has quit
      • 2017-12-22 35623, 2017

      • xps2 joined the channel
      • 2017-12-22 35606, 2017

      • MusicbrainzB0T1 joined the channel
      • 2017-12-22 35642, 2017

      • Lotheric joined the channel
      • 2017-12-22 35648, 2017

      • MusicbrainzB0T has quit
      • 2017-12-22 35605, 2017

      • xps2 has quit
      • 2017-12-22 35643, 2017

      • xps2 joined the channel
      • 2017-12-22 35602, 2017

      • drsaunde joined the channel
      • 2017-12-22 35605, 2017

      • heyoni has quit
      • 2017-12-22 35628, 2017

      • heyoni joined the channel
      • 2017-12-22 35646, 2017

      • naiveai
        yvanzo: https://pastebin.com/GesFHu4Z first query gets the potentially confusing artists, and second query gets a list of recordings with their artist names in an array
      • 2017-12-22 35637, 2017

      • djwhitey has quit
      • 2017-12-22 35638, 2017

      • djwhitey_ joined the channel
      • 2017-12-22 35644, 2017

      • KassOtsimine has quit
      • 2017-12-22 35611, 2017

      • KassOtsimine joined the channel
      • 2017-12-22 35611, 2017

      • KassOtsimine has quit
      • 2017-12-22 35611, 2017

      • KassOtsimine joined the channel
      • 2017-12-22 35608, 2017

      • samj1912
        zas: can you make a sir dashboard with https://stats.metabrainz.org/dashboard/db/hetzner… and the previous rmq queues that you linked?
      • 2017-12-22 35649, 2017

      • rgunkar has quit
      • 2017-12-22 35609, 2017

      • KassOtsimine
        wait. what is this "sleep" thing?
      • 2017-12-22 35607, 2017

      • theremin joined the channel
      • 2017-12-22 35645, 2017

      • drsaunde has quit
      • 2017-12-22 35603, 2017

      • gcilou has quit
      • 2017-12-22 35640, 2017

      • ferbncode_ has quit
      • 2017-12-22 35616, 2017

      • Sophist-UK joined the channel
      • 2017-12-22 35650, 2017

      • ferbncode_ joined the channel
      • 2017-12-22 35640, 2017

      • github joined the channel
      • 2017-12-22 35640, 2017

      • github
        [listenbrainz-server] paramsingh opened pull request #326: Restructure listen dumps (master...restructure-listen-dumps) https://git.io/vbHG2
      • 2017-12-22 35640, 2017

      • github has left the channel
      • 2017-12-22 35630, 2017

      • MusicbrainzB0T joined the channel
      • 2017-12-22 35638, 2017

      • annebelleo joined the channel
      • 2017-12-22 35640, 2017

      • MusicbrainzB0T1 has quit
      • 2017-12-22 35638, 2017

      • naiveai
        yvanzo: for a detailed explanation, will "This report shows releases (official and promotional only) that have mediums with formats that support Disc ID, but do not have them"
      • 2017-12-22 35631, 2017

      • annebelleo has quit
      • 2017-12-22 35603, 2017

      • naiveai
        what does "NOTICE: table "releases_missing_disc_ids" does not exist, skipping" when running a report mean?
      • 2017-12-22 35622, 2017

      • SothoTalKer
        moin moin
      • 2017-12-22 35602, 2017

      • theremin
        I've been talking to naiveai about psql not working on the pre-built VM. They said in #musicbrainz to maybe ask in here?
      • 2017-12-22 35647, 2017

      • theremin
        AND/OR, my main goal is to export a huge list of tracks in alpha order.
      • 2017-12-22 35605, 2017

      • samj1912
        theremin: you can use our json dumps
      • 2017-12-22 35624, 2017

      • theremin
        Can you guide an amateur through the process?
      • 2017-12-22 35653, 2017

      • theremin
        amateur = 1/2 idiot.
      • 2017-12-22 35610, 2017

      • samj1912
        theremin: iirc you just wanted works right?
      • 2017-12-22 35619, 2017

      • samj1912
      • 2017-12-22 35625, 2017

      • samj1912
        its a json dump of all works
      • 2017-12-22 35632, 2017

      • theremin
        I'll take it, but then I realized I think I want tracks
      • 2017-12-22 35645, 2017

      • samj1912
        we have recordings
      • 2017-12-22 35622, 2017

      • samj1912
      • 2017-12-22 35626, 2017

      • samj1912
        hmm, this is weird
      • 2017-12-22 35647, 2017

      • samj1912
        ruaok: bitmap is the recording dump supposed to be only 5.7 mb?
      • 2017-12-22 35650, 2017

      • theremin
        what do I do with these?
      • 2017-12-22 35614, 2017

      • samj1912
        theremin: download the work dump for now and untar it
      • 2017-12-22 35610, 2017

      • bitmap
        samj1912: yes, because it only include standalone recordings
      • 2017-12-22 35620, 2017

      • samj1912
        oh okay
      • 2017-12-22 35629, 2017

      • naiveai
        are reports time-limited or something?
      • 2017-12-22 35629, 2017

      • samj1912
        so release is the correct one then?
      • 2017-12-22 35635, 2017

      • bitmap
        the release dump includes the rest of them
      • 2017-12-22 35625, 2017

      • samj1912
        bitmap: I made some new triggers, I think now it doesn't miss any paths
      • 2017-12-22 35616, 2017

      • bitmap
        cool, I will look at the prs again today
      • 2017-12-22 35644, 2017

      • samj1912
        thanks :D
      • 2017-12-22 35645, 2017

      • naiveai
        yvanzo: the line AND status NOT IN (3, 4) is making my query much, much slower
      • 2017-12-22 35651, 2017

      • naiveai
        should I just remove it
      • 2017-12-22 35601, 2017

      • naiveai
        and by much much slower I mean like REALLY slow
      • 2017-12-22 35606, 2017

      • naiveai
        nearly 150s
      • 2017-12-22 35640, 2017

      • theremin
        oh, recordings is 18mil. that would be great.
      • 2017-12-22 35646, 2017

      • theremin
        still extracting
      • 2017-12-22 35652, 2017

      • samj1912
        theremin: ftp://ftp.musicbrainz.org/pub/musicbrainz/data/js… you want the release dump from here
      • 2017-12-22 35624, 2017

      • samj1912
      • 2017-12-22 35651, 2017

      • samj1912 has never been happier seeing so many zeroes :P
      • 2017-12-22 35636, 2017

      • theremin
        it's extracted
      • 2017-12-22 35608, 2017

      • samj1912
        theremin: now use any scripting language to extract the parts you want
      • 2017-12-22 35624, 2017

      • samj1912
        if you know some python, it should be a couple of lines of code
      • 2017-12-22 35621, 2017

      • heyoni has quit
      • 2017-12-22 35634, 2017

      • naiveai
        this is very od
      • 2017-12-22 35605, 2017

      • gcilou joined the channel
      • 2017-12-22 35633, 2017

      • naiveai
        yvanzo: done
      • 2017-12-22 35625, 2017

      • zas
        samj1912: I'll make the dashboard you need tomorrow
      • 2017-12-22 35615, 2017

      • Leftmost
        LordSputnik, would you say data migrations should go into bookbrainz-sql or only schema migrations?
      • 2017-12-22 35623, 2017

      • samj1912
        zas: Thanks!
      • 2017-12-22 35634, 2017

      • LordSputnik
        Leftmost: what sort of data migrations?
      • 2017-12-22 35645, 2017

      • Leftmost
        I just put together a PR that filters on identifier types. It sets the proper type on existing identifier types, creates new Wikidata identifier types for each entity type, and then migrates old Wikidata identifiers to the proper type.
      • 2017-12-22 35612, 2017

      • LordSputnik
        Hmm
      • 2017-12-22 35625, 2017

      • LordSputnik
        Yeah put it somewhere in -sql
      • 2017-12-22 35635, 2017

      • Leftmost
        Okay.
      • 2017-12-22 35638, 2017

      • LordSputnik
        Leftmost: speaking of data, I've been thinking of moving static lists like Creator Types, Work Types, Edition Format/Status into the code, rather than the database.
      • 2017-12-22 35600, 2017

      • LordSputnik
        Benefits are: adding types doesn't require a schema change; probably easier to use throughout the code
      • 2017-12-22 35636, 2017

      • Leftmost
        Adding types wouldn't require a schema change as-is. They're tables, not enums.
      • 2017-12-22 35602, 2017

      • LordSputnik
        Oh yeah, I meant database access, wasn't concentrating
      • 2017-12-22 35618, 2017

      • LordSputnik
        So someone can just submit a PR to add more types
      • 2017-12-22 35641, 2017

      • Leftmost
        It'd also make it impossible to tell what the type of a Creator is if they only had the database.
      • 2017-12-22 35643, 2017

      • LordSputnik
        And it reduces the amount of work needed to set up a new BB database from scratch, rather than from a dump
      • 2017-12-22 35603, 2017

      • LordSputnik
        True
      • 2017-12-22 35625, 2017

      • travis-ci joined the channel
      • 2017-12-22 35626, 2017

      • travis-ci
        Project bookbrainz-site build #1576: passed in 5 min 29 sec: https://travis-ci.org/bookbrainz/bookbrainz-site/…
      • 2017-12-22 35626, 2017

      • travis-ci has left the channel
      • 2017-12-22 35612, 2017

      • LordSputnik
        Perhaps we can achieve the same thing by loading the data into the application when it starts
      • 2017-12-22 35627, 2017

      • naiveai
        but then most of the advantages are lost
      • 2017-12-22 35642, 2017

      • Leftmost
        That's on my radar for the rewritten data layer.
      • 2017-12-22 35644, 2017

      • naiveai
        becomes pretty complicated to orchestrate
      • 2017-12-22 35610, 2017

      • LordSputnik
        I'll think about this some more
      • 2017-12-22 35635, 2017

      • heyoni joined the channel
      • 2017-12-22 35624, 2017

      • Leftmost
      • 2017-12-22 35605, 2017

      • djwhitey joined the channel
      • 2017-12-22 35646, 2017

      • djwhitey_ has quit
      • 2017-12-22 35640, 2017

      • Leftmost
        The former doesn't do much for us now, but the latter might.
      • 2017-12-22 35651, 2017

      • madmouser1 has quit
      • 2017-12-22 35600, 2017

      • zas
        samj1912: https://stats.metabrainz.org/dashboard/db/sir-and… (needs more work, but that's a start)
      • 2017-12-22 35650, 2017

      • samj1912
        good enough for me really :P
      • 2017-12-22 35600, 2017

      • samj1912
        can I also make dashboards?
      • 2017-12-22 35645, 2017

      • naiveai
        wow, I got a query! yay!!
      • 2017-12-22 35602, 2017

      • naiveai
        it also takes literally 1 full minute to execute on my small test db :(
      • 2017-12-22 35610, 2017

      • naiveai
      • 2017-12-22 35614, 2017

      • zas
        samj1912: yes, i set you as editor, so you can update/create dashboards, but please, only work on your dashboards, grafana is sometimes very picky, and you can easily break existing stuff. That's less an issue now because of the new feature https://github.com/grafana/grafana/pull/8472 but still, better not break things
      • 2017-12-22 35654, 2017

      • yokel joined the channel
      • 2017-12-22 35655, 2017

      • samj1912
        okay, yup, will work on my own ones
      • 2017-12-22 35601, 2017

      • samj1912
        thanks
      • 2017-12-22 35630, 2017

      • zas
        note there are a lot of metrics stored in influxdb, ask me if you arent sure about smt, also you need to use derivatives to get bytes/s etc..
      • 2017-12-22 35651, 2017

      • alastairp
        hmm, I found a bug in the bcn metro ticket system
      • 2017-12-22 35651, 2017

      • zas
        check existing graphs queries
      • 2017-12-22 35654, 2017

      • TehTotalPwnage has quit
      • 2017-12-22 35601, 2017

      • samj1912
        okay
      • 2017-12-22 35604, 2017

      • naiveai
        do we have indexes on any columns in the db?
      • 2017-12-22 35648, 2017

      • zas
        important: we collect metrics every minute, but queries are limited in number of results, so you can't draw detailed graphs over a long period, those are truncated silently, it is a bit disapointing sometimes. In any case, just ask me (and read docs ofc). I still have a lot of things to learn concerning this stack.
      • 2017-12-22 35612, 2017

      • TehTotalPwnage joined the channel
      • 2017-12-22 35652, 2017

      • gcilou has quit
      • 2017-12-22 35630, 2017

      • Dalek_ has quit
      • 2017-12-22 35616, 2017

      • Dalek_ joined the channel
      • 2017-12-22 35626, 2017

      • samj1912
        zas: how do I group these by containers? https://stats.metabrainz.org/dashboard/db/sir-and…
      • 2017-12-22 35644, 2017

      • samj1912
        oh wait, it didnt save
      • 2017-12-22 35645, 2017

      • samj1912
        1 sec
      • 2017-12-22 35630, 2017

      • samj1912
      • 2017-12-22 35641, 2017

      • samj1912
        this ^
      • 2017-12-22 35632, 2017

      • samj1912
        wow, this is really cool
      • 2017-12-22 35602, 2017

      • samj1912 reads about grafana and telegraf
      • 2017-12-22 35625, 2017

      • naiveai is splitting his head open
      • 2017-12-22 35632, 2017

      • SothoTalKer
        naiveai: how's the task going? :D
      • 2017-12-22 35602, 2017

      • naiveai
        very, very badly
      • 2017-12-22 35610, 2017

      • SothoTalKer
        why is that?
      • 2017-12-22 35626, 2017

      • naiveai
        1. i've got a query that's taking waaaaaaaaayyyyyyyy to long already