(There were a number of diversity discussions about music festivals in Denmark last year. MB would be a perfect fit to codify this data and make reports from.)
2017-04-25 11529, 2017
Hobbyboy joined the channel
2017-04-25 11548, 2017
hugo
@fresco: here is an overview of the styles https://www.youtube.com/watch?v=I0zEAQTnUrU not the music i listen to daily, it is just folk but this radiostation just plays those sounds
2017-04-25 11505, 2017
drsaunders joined the channel
2017-04-25 11545, 2017
Sophist-UK
samj1912: Ok - I give up. I have a test_metadata.pyc file and cannot find a matching .py file anywhere.
2017-04-25 11557, 2017
samj1912
lol :P
2017-04-25 11506, 2017
samj1912
I think you made it sometime and it stayed
2017-04-25 11519, 2017
samj1912
we have a test_scripts and a test_formats
2017-04-25 11529, 2017
samj1912
that we use for similar tests
2017-04-25 11536, 2017
samj1912
you can add it to test_scripts maybe?
2017-04-25 11515, 2017
Sophist-UK
Well the .pyc file would not get deleted by git checkout, only if I do it manually. So it will hang around. But at some point I must have had a matching .py file on my PC and run tests.
2017-04-25 11501, 2017
Sophist-UK
Perhaps I attempted to create a set of tests for metadata and then decided it was too much effort. Who knows. It was 3 years ago and I can't remember what I had for breakfast.
2017-04-25 11552, 2017
Mineo
there's a module called decompyle (I think) that can turn a .pyc back into a .py file if you want to have a look at it :)
ruaok: will do my initial project pitch to a couple of profs tomorrow
2017-04-25 11525, 2017
samj1912
I was thinking about extending MsB to a NLP based metadata extractor (I think I have a very very vague idea how it currently works on last fm data)
2017-04-25 11511, 2017
samj1912
couldnt find much on MsB-server repo
2017-04-25 11539, 2017
samj1912
I don't even know much about what the scope and future plans for MsB are
2017-04-25 11557, 2017
SothoTalker_
MusicBrainz? :D
2017-04-25 11503, 2017
samj1912
messybrainz
2017-04-25 11520, 2017
SothoTalker_
:-)
2017-04-25 11533, 2017
Quesito
hey gcilou! I added a few things to the UX survey doc, about to call it a night, any q's or thoughts before I go?
2017-04-25 11515, 2017
gcilou
Hey Quesito :) I looked over it, looks nice!
2017-04-25 11557, 2017
Quesito
hopefully getting somewhere....:P let me know your thoughts / comments, etc!
2017-04-25 11510, 2017
gcilou
Yeah, definitely! I'll look more thoroughly soon
2017-04-25 11517, 2017
Quesito
thank you! any advice, thoughts, etc are needed :)
2017-04-25 11503, 2017
Sophist-UK
sam1912: I decompiled the test_metadata.py file and it doesn't look to me like code I would have written. But relatively short - I will try to build a comprehensive test_metadata.py.
2017-04-25 11520, 2017
regagain has quit
2017-04-25 11535, 2017
samj1912
okay
2017-04-25 11526, 2017
samj1912 sleeps
2017-04-25 11519, 2017
SothoTalker_
nighty
2017-04-25 11548, 2017
SothoTalker_
is there any way to circumvent 503 errors, except being a paying supporter? ^^
2017-04-25 11515, 2017
kepstin
SothoTalker_: what are you trying to do?
2017-04-25 11551, 2017
SothoTalker_
just fetch stuff from the webservice
2017-04-25 11508, 2017
kepstin
sure, but what kind of stuff, and how much?
2017-04-25 11521, 2017
SothoTalker_
all releases of a label
2017-04-25 11557, 2017
SothoTalker_
which are 16 requests in total, to get everything.
2017-04-25 11503, 2017
kepstin
so, the way to not get 503 errors is to follow the rate limiting requirements...
2017-04-25 11518, 2017
SothoTalker_
i currently make one request every 5 seconds
2017-04-25 11542, 2017
kepstin
hmm, that should be ok; the actual limit is ~1/second
2017-04-25 11507, 2017
kepstin
per ip address, for properly configured apps (set user agent, etc.)
2017-04-25 11532, 2017
SothoTalker_
it's just a simple script :-)
2017-04-25 11559, 2017
kepstin
if you're using some generic user agent rather than specifying one that identifies your tool, you hit much harsher rate limits
2017-04-25 11557, 2017
kepstin
so that's the first thing to do - set a user agent, call it "SothoTalker's Label Scraper" or whatever, doesn't matter - then you should be able to use the webservice at 1 request/second
if you want faster than 1 req/second, then you basically have two options - look into getting an api key with raised rate limit for the musicbrainz.org ws, or run your own replicated server locally which you can query as fast as you like (or even just read info direct from the postgres db)
2017-04-25 11556, 2017
SothoTalker_
well, i dont want faster access, i just dont want that 503 error happen to me :D
2017-04-25 11545, 2017
SothoTalker_
i suspect the WS is just hit heavily, since this did not occur earlier like 2 weeks ago.
2017-04-25 11554, 2017
kepstin
right now I assume you're being thrown into the "anonymous" group because of the user agent on your webservice requests, and due to the global 50/s limit, that's unpredictable.
2017-04-25 11515, 2017
kepstin
so fix the user agent first, then see if that helps :/
2017-04-25 11533, 2017
SothoTalker_
i tried a few ua strings, and it didnt help :)
2017-04-25 11559, 2017
SothoTalker_
i even made my script wait 10 seconds between each request
2017-04-25 11555, 2017
kepstin
hmm. I guess the webservice might just be overloaded overall, then.
2017-04-25 11512, 2017
SothoTalker_
that's my guess, too.
2017-04-25 11524, 2017
kepstin
I don't know what the current global limit is, but I thought it was raised with the switch to newhost
2017-04-25 11556, 2017
SothoTalker_
well, that problem does happen since about 2 weeks ago. before i had no problems.
2017-04-25 11521, 2017
kepstin
well, the solution is to "try again later" - If you want to encode that into your script, look at having a backoff algorithm so it's not making the overload worse.
2017-04-25 11536, 2017
SothoTalker_
and it's not like my script permanently hammers the server. i just run it once or twice a day to update my spreadsheet
2017-04-25 11524, 2017
SothoTalker_
i might have to parse the 503 error message and put a 'try again after a few seconds' code :x
2017-04-25 11530, 2017
kepstin
if you're doing any sort of query like that, try to run it at a random time each day rather than the same time every day
2017-04-25 11552, 2017
kepstin
well, you shouldn't retry the entire series of requests - just have it wait for a while, then continue where it left off...
2017-04-25 11502, 2017
SothoTalker_
that's what i meant
2017-04-25 11516, 2017
kepstin
you probably want to do an exponential backoff - so if you're e.g. waiting 1s between requests, then if one fails, wait 2s before trying again, if it fails again, wait 4s, and so on.
2017-04-25 11554, 2017
kepstin
and if the wait gets too long, maybe decide to give up with an error message ;)
2017-04-25 11516, 2017
SothoTalker_
heh
2017-04-25 11526, 2017
SothoTalker_
well, now it went through, luckily
2017-04-25 11534, 2017
alastairp
gcilou: I don't think alpha import should have as much focus. Ideally we want people to ignore their alpha account
2017-04-25 11526, 2017
gcilou
Ok! I can definitely adjust to that. Rob sent me a comment about his ideal layout for the page, so the whiteboard was following that.
2017-04-25 11547, 2017
alastairp
Ah, I didn't see that
2017-04-25 11502, 2017
alastairp
Anyway, that's my opinion
2017-04-25 11534, 2017
gcilou
yeah that's cool. I'm trying a few things out to get a feel for it
