#metabrainz

/

      • nav2002 has quit
      • chaban has quit
      • c1e0 joined the channel
      • c1e0_ joined the channel
      • c1e0 has quit
      • nav2002 joined the channel
      • Tanish joined the channel
      • Tanish has quit
      • Tanish joined the channel
      • Tanish has quit
      • abhishekpanwar joined the channel
      • abhishekpanwar has quit
      • Tanish joined the channel
      • Tanish has quit
      • abhishekpanwar joined the channel
      • KassOtsimine joined the channel
      • abhishekpanwar has quit
      • zas
        bitmap: around?
      • ruaok
        mooooin!
      • alastairp: aidanlw17 : have you seen this? https://github.com/deezer/spleeter
      • nav2002 has quit
      • alastairp
        yeah, I saw it announced on our maling lists
      • (hi)
      • bitmap: zas: I'm trying to debug https://github.com/alastair/python-musicbrainzn..., I tried the url and once got a notification (in json) "unable to connect to db server" or similar - I've since been unable to reproduce it
      • is it known that there is some kind of issue with webservice workers unable to connect to the db sometimes?
      • zas
        do you have urls queried? and time it failed?
      • alastairp
      • about 10 minutes ago
      • zas
      • this drop on 403 usually shows network issues (not under our control), but we also had a floyd issue (high temp, temporary), I'll check logs
      • alastairp
        I'd like to reproduce it, at least to be able to report the issue
      • it's annoying that the ws is returning json by default sometimes, also it seems like it was returned with HTTP200
      • D4RK-PH0ENiX has quit
      • Gazooo has quit
      • Gazooo joined the channel
      • D4RK-PH0ENiX joined the channel
      • zas
        alastairp: btw, 503s are almost always generated by gateways, not backends
      • we have code to manage format thing (which is a bit messy, depending on parameters and headers)
      • if you ask for json and get xml error, that's a bug, but if I remember correctly there are few corner cases that couldn't be easily fixed
      • ruaok
        ohh, this is quite interesting: https://postgrest.org/en/v6.0/
      • should be quite useful for our new data sets.
      • reosarevok hears "new data sets", gets interested
      • reosarevok
        Or is this just LB / MsB stuff?
      • (in which case also interested, but I'll leave the hard work to y'all :p)
      • ruaok
        the first thing I would try would be the artist-artist data set...
      • zas
        interesting, though nothing really new to me, we hardly have any TTL under 5-10 minutes, and only for records that may/have change. I'll review this and increasing TTLs if there's no reason for them to be low (most of the times we decreased those to help with hardware/network changes). And gandi doesn't let one set a ttl to values under 300.
      • ruaok
        yeah, I wonder if some of our 5 minute TTLs should be moved up to 1hr.
      • c1e0_ is now known as c1e0
      • antlarr has quit
      • a cool looking project that just signed up with us: http://hifiberry.com/hifiberryos
      • SothoTalKer
        moin (:
      • Mr_Monkey
        ruaok: looks nice !
      • SothoTalKer
        ruaok, zas: i'm not too well informed but is it not possible to override the TTL manually?
      • ruaok
        probably, but what impact would that have?
      • alastairp
        SothoTalKer: we as owners of the domain set our own ttl, but occasionally we lower it (if for example we want to change it in the near future)
      • SothoTalKer
        i mean you set a default long TTL, but in case there's a change you can issue a manual update or something?
      • alastairp
        sure, you release a new version of the dns record with a new ttl
      • ruaok
        if you need to changes it, you drop the TTL really low, wait for the previously high duration to expire, then move your services, wait for the shorter window to expire and then you can raise the TTL again.
      • alastairp
        (but then you have to wait for the old ttl to expire before clients use the new one)
      • ruaok
        in theory no one should notice the move.
      • alastairp
        ^ what he said :)
      • btw, hello from ismir 2019
      • ruaok
        in the netherlands, yes?
      • SothoTalKer
        ok, one could automate that then. :)
      • alastairp
        no chocolate table, no sticker table. these conferences could learn a few things from open source confs
      • yes, in Delft
      • ruaok
        ahh <3 delft.
      • yes, why not start a chocolate table at ISMIR?
      • alastairp
        old city is really cute
      • ruaok
        barfingly cute, even.
      • I have a good friend who lives literally on the edge of the market.
      • abhishekpanwar joined the channel
      • The HiFiBerryOS folks just offered to ship us a sound card to try out -- I would love for this to replace my sonos system. :)
      • alastairp
        reproduced the error with WS:
      • this page returns http 200, so we try and parse it (as xml)
      • ruaok
        I keep seeing this error being talked about, when it should be an easy fix -- our DB server isn't close to its limitations, why not add more connections. bitmap, can you clue me in as to why we're not doing this?
      • alastairp
        regardless of the fix, it'd be great to see this returning xml and an error status code if possible (if the error is raised by the workers), I'll open a ticket for this
      • zas
        ruaok: the real question is why do we even reach limits, it happens since we move to floyd.
      • ruaok
        I think that problems has existed longer than that. LB hits it occasionally.
      • zas
        can it be related to LB? we had issues on lemmy & floyd at the same time, and load on floyd temporarly attained very high values last weeks, it usually solves by itself after few minutes. But for sure it wasn't happening when master db was still on bowie. AFAIK, a lot of things changed on db side with the migration, mainly to have easier master/slave switch
      • ruaok
        not sure -- could be. when I looked at that case, I noticed that the load on lemmy was high because of consul config changes being pushed.
      • that causes all the containers to restart at the same time, causing a load spike.
      • but the random out of connection errors have been around for a quite a long time.
      • zas
        alastairp: at which UTC time did you get this error?
      • alastairp
        zas: just before I pasted to you in PM, or 1-2 minutes earlier
      • Lotheric has quit
      • Lotheric joined the channel
      • abhishekpanwar has quit
      • abhishekpanwar joined the channel
      • abhishekpanwar
        Mr_Monkey: BookBrainz will be part of GSOC 2020, right?
      • Mr_Monkey
        Hi abhishekpanwar !
      • We're not there yet, but it was a success this year so I don't see why not !
      • abhishekpanwar
        Okay great!
      • abhishekpanwar has quit
      • CatQuest
        Mr_Monkey: what about for gci? i would like to have some data entry tasks
      • maybe not a huge amount to start with but we have some "it's a lot harder to do duplicate artists" now
      • Mr_Monkey
        CatQuest: I was going to do a lot less entry tasks myself compared to last year, but if you're up for doing *some*, yay ! :)
      • I haven't reviewed the BB tasks yet but I can let you know as soon as that's done
      • Freso just realised we’re in a new month and should send it invoice… - may not get to that until after meeting :x
      • bitmap
        ruaok: zas: we should bump max_client_conn at least (maybe from 400 to 500 to start?) though it'd also be great if we had some pgbouncer pool stats in graphana to see where/how we're hitting those limits, then we can tune them better
      • ruaok
        moin bitmap. yes, that sounds like a great plan. how can zas collect stats?
      • (zas is AFK for the rest of the day, so I am asking the questions that he would naturally ask :) )
      • bitmap
        haha okay
      • the way I see them is to connect to the 'pgbouncer' database on floyd and run 'show stats;' / 'show databases;'
      • sorry, 'show pools;' not stats
      • antlarr joined the channel
      • ruaok
        can you please do me a favor and make an MBH ticket for zas that summarizes this info and maybe add a couple of links for more info as well as noting which data values you'd like to see graphed?
      • In particular, providing SQL queries that he can run on the command line would be super helpful, methinks.
      • bitmap
        yup, I can certainly do so
      • ruaok
        fabu, thanks.
      • travis-ci joined the channel
      • travis-ci
        [picard:master@e835277 - build #5206] CI errored! (https://travis-ci.org/metabrainz/picard/builds/...)
      • travis-ci has left the channel
      • alastairp
        bitmap: would it be possible to return this error in xml for the ws? (or at the very least return a non-200 status code?)
      • bitmap
        yeah, the 200 status code is definitely a bug. I can look into having it return xml too
      • can you add a ticket when you have time?
      • antlarr has quit
      • alastairp
        yes, will do
      • thanks
      • antlarr joined the channel
      • SothoTalKer
        i do love the positiveness in here :-)
      • Mr_Monkey
        ruaok: Gah. After some more digging, it might just be that the Spotify player will fail if we're not on https (localhost is considered secured so it'll work there too). That could also explain the inconcistent behaviour I've observe depending on where I was testing.
      • Is it possible to put Odyssey on https, and if so could I ask you to do that (no rush ofc) please?
      • ruaok
        Yeah, I can look at that tomorrow.
      • Mr_Monkey
        (Spotify's player uses EME [Encrypted Media Extensions], which won't work on http)
      • Thanks !
      • BrainzGit
        [musicbrainz-server] mwiencek opened pull request #1255 (master…flow-0.111): Flow 0.111 https://github.com/metabrainz/musicbrainz-serve...
      • bitmap
        no yvanzo today either?
      • aidanlw17
        Freso: alastairp and I were thinking about doing a classification task for AB for GCI. We would do something like ask them to choose 5 genres or classes from a set, and choose 10 or so recordings for each class. We can then use the classification later to build datasets. Does that seem like an appropriate task for GCI?
      • (by
      • (by "them" I mean each student)
      • alastairp
        I'm away at a conference this week, so am not around much to discuss this, but should be able to reply to messages when I have spare time if you have any questions :)
      • --> gone
      • aidanlw17
        alastairp: Freso said we still aren't quite ready to add tasks anyways so I think that we can discuss when you get back. Just wanted to open the question while it's on my mind!
      • rdswift
        aidanlw17, the only thing that I see that *might* be an issue is that assigning recordings to genres can be somewhat subjective.
      • CatQuest
        true
      • but genres are genreally (:D) subjective anyway
      • rdswift
        My point exactly. If the mentor has a different *opinion* of a genre, will they be asking the student to redo all or a portion of the task before marking it as complete?
      • CatQuest
        if the opinnion is in what genre i'd say no
      • (unless it's obviously trolling)
      • ie beiber being death metal etc
      • rdswift
        But to someone, perhaps Bieber *is* death metal. ;-)
      • aidanlw17
        rdswift: alastairp can confirm, but I think the idea is that we get more assurance with multiple opinions. If we have multiple students placing tracks from the same artist in a genre, then we get a good idea that the artist fits there
      • CatQuest
        nja. thne they gotta explain why i'd say
      • rdswift
        aidanlw17, I see your point. If you're looking for data to analyze, then that's probably a good task, understanding that an individual piece of data could be "incorrect".
      • yvanzo joined the channel
      • SothoTalKer
        look who's there (:
      • aidanlw17
        rdswift: cool, thanks for the feedback! That's just the general idea of it, we'll have to make it more specific and clear
      • Freso
        <BANG>