#metabrainz

bitmap: around?

mooooin!

alastairp: aidanlw17 : have you seen this? https://github.com/deezer/spleeter

yeah, I saw it announced on our maling lists

(hi)

bitmap: zas: I'm trying to debug https://github.com/alastair/python-musicbrainzn..., I tried the url and once got a notification (in json) "unable to connect to db server" or similar - I've since been unable to reproduce it

is it known that there is some kind of issue with webservice workers unable to connect to the db sometimes?

do you have urls queried? and time it failed?

about 10 minutes ago

we had weird issues : https://stats.metabrainz.org/d/000000061/mbstat...

this drop on 403 usually shows network issues (not under our control), but we also had a floyd issue (high temp, temporary), I'll check logs

I'd like to reproduce it, at least to be able to report the issue

it's annoying that the ws is returning json by default sometimes, also it seems like it was returned with HTTP200

alastairp: btw, 503s are almost always generated by gateways, not backends

we have code to manage format thing (which is a bit messy, depending on parameters and headers)

if you ask for json and get xml error, that's a bug, but if I remember correctly there are few corner cases that couldn't be easily fixed

ohh, this is quite interesting: https://postgrest.org/en/v6.0/

should be quite useful for our new data sets.

Or is this just LB / MsB stuff?

(in which case also interested, but I'll leave the hard work to y'all :p)

the first thing I would try would be the artist-artist data set...

zas: https://00f.net/2019/11/03/stop-using-low-dns-t...

interesting, though nothing really new to me, we hardly have any TTL under 5-10 minutes, and only for records that may/have change. I'll review this and increasing TTLs if there's no reason for them to be low (most of the times we decreased those to help with hardware/network changes). And gandi doesn't let one set a ttl to values under 300.

yeah, I wonder if some of our 5 minute TTLs should be moved up to 1hr.

a cool looking project that just signed up with us: http://hifiberry.com/hifiberryos

moin (:

ruaok: looks nice !

ruaok, zas: i'm not too well informed but is it not possible to override the TTL manually?

probably, but what impact would that have?

SothoTalKer: we as owners of the domain set our own ttl, but occasionally we lower it (if for example we want to change it in the near future)

i mean you set a default long TTL, but in case there's a change you can issue a manual update or something?

sure, you release a new version of the dns record with a new ttl

if you need to changes it, you drop the TTL really low, wait for the previously high duration to expire, then move your services, wait for the shorter window to expire and then you can raise the TTL again.

(but then you have to wait for the old ttl to expire before clients use the new one)

in theory no one should notice the move.

^ what he said :)

btw, hello from ismir 2019

in the netherlands, yes?

ok, one could automate that then. :)

no chocolate table, no sticker table. these conferences could learn a few things from open source confs

yes, in Delft

ahh <3 delft.

yes, why not start a chocolate table at ISMIR?

old city is really cute

barfingly cute, even.

I have a good friend who lives literally on the edge of the market.

The HiFiBerryOS folks just offered to ship us a sound card to try out -- I would love for this to replace my sonos system. :)

reproduced the error with WS:

this page returns http 200, so we try and parse it (as xml)

I keep seeing this error being talked about, when it should be an easy fix -- our DB server isn't close to its limitations, why not add more connections. bitmap, can you clue me in as to why we're not doing this?

regardless of the fix, it'd be great to see this returning xml and an error status code if possible (if the error is raised by the workers), I'll open a ticket for this

ruaok: the real question is why do we even reach limits, it happens since we move to floyd.

I think that problems has existed longer than that. LB hits it occasionally.

can it be related to LB? we had issues on lemmy & floyd at the same time, and load on floyd temporarly attained very high values last weeks, it usually solves by itself after few minutes. But for sure it wasn't happening when master db was still on bowie. AFAIK, a lot of things changed on db side with the migration, mainly to have easier master/slave switch

not sure -- could be. when I looked at that case, I noticed that the load on lemmy was high because of consul config changes being pushed.

that causes all the containers to restart at the same time, causing a load spike.

but the random out of connection errors have been around for a quite a long time.

alastairp: at which UTC time did you get this error?

zas: just before I pasted to you in PM, or 1-2 minutes earlier

Mr_Monkey: BookBrainz will be part of GSOC 2020, right?

Hi abhishekpanwar !

We're not there yet, but it was a success this year so I don't see why not !

Okay great!

Mr_Monkey: what about for gci? i would like to have some data entry tasks

maybe not a huge amount to start with but we have some "it's a lot harder to do duplicate artists" now

CatQuest: I was going to do a lot less entry tasks myself compared to last year, but if you're up for doing *some*, yay ! :)

I haven't reviewed the BB tasks yet but I can let you know as soon as that's done

ruaok: zas: we should bump max_client_conn at least (maybe from 400 to 500 to start?) though it'd also be great if we had some pgbouncer pool stats in graphana to see where/how we're hitting those limits, then we can tune them better

moin bitmap. yes, that sounds like a great plan. how can zas collect stats?

(zas is AFK for the rest of the day, so I am asking the questions that he would naturally ask :) )

haha okay

the way I see them is to connect to the 'pgbouncer' database on floyd and run 'show stats;' / 'show databases;'

sorry, 'show pools;' not stats

can you please do me a favor and make an MBH ticket for zas that summarizes this info and maybe add a couple of links for more info as well as noting which data values you'd like to see graphed?

In particular, providing SQL queries that he can run on the command line would be super helpful, methinks.

yup, I can certainly do so

fabu, thanks.

[picard:master@e835277 - build #5206] CI errored! (https://travis-ci.org/metabrainz/picard/builds/...)

bitmap: would it be possible to return this error in xml for the ws? (or at the very least return a non-200 status code?)

yeah, the 200 status code is definitely a bug. I can look into having it return xml too

can you add a ticket when you have time?

yes, will do

thanks

i do love the positiveness in here :-)

ruaok: Gah. After some more digging, it might just be that the Spotify player will fail if we're not on https (localhost is considered secured so it'll work there too). That could also explain the inconcistent behaviour I've observe depending on where I was testing.

Is it possible to put Odyssey on https, and if so could I ask you to do that (no rush ofc) please?

Yeah, I can look at that tomorrow.

(Spotify's player uses EME [Encrypted Media Extensions], which won't work on http)

Thanks !

[musicbrainz-server] mwiencek opened pull request #1255 (master…flow-0.111): Flow 0.111 https://github.com/metabrainz/musicbrainz-serve...

no yvanzo today either?

Freso: alastairp and I were thinking about doing a classification task for AB for GCI. We would do something like ask them to choose 5 genres or classes from a set, and choose 10 or so recordings for each class. We can then use the classification later to build datasets. Does that seem like an appropriate task for GCI?

(by

(by "them" I mean each student)

I'm away at a conference this week, so am not around much to discuss this, but should be able to reply to messages when I have spare time if you have any questions :)

--> gone

alastairp: Freso said we still aren't quite ready to add tasks anyways so I think that we can discuss when you get back. Just wanted to open the question while it's on my mind!

aidanlw17, the only thing that I see that *might* be an issue is that assigning recordings to genres can be somewhat subjective.

true

but genres are genreally (:D) subjective anyway

My point exactly. If the mentor has a different *opinion* of a genre, will they be asking the student to redo all or a portion of the task before marking it as complete?

if the opinnion is in what genre i'd say no

(unless it's obviously trolling)

ie beiber being death metal etc

But to someone, perhaps Bieber *is* death metal. ;-)

rdswift: alastairp can confirm, but I think the idea is that we get more assurance with multiple opinions. If we have multiple students placing tracks from the same artist in a genre, then we get a good idea that the artist fits there

nja. thne they gotta explain why i'd say

aidanlw17, I see your point. If you're looking for data to analyze, then that's probably a good task, understanding that an individual piece of data could be "incorrect".

look who's there (:

rdswift: cool, thanks for the feedback! That's just the general idea of it, we'll have to make it more specific and clear

<BANG>