#metabrainz

\o heisan gcilou

sorry that of all the scandinavian countries you had to go to sweden :D

our usual gelateria makes the cut of the best in barcelona: http://www.lavanguardia.com/comer/sitios/201706...

Lol the sun is finally peeking out

But man, these trains are difficult to figure out

the t-banen?

or trying to figure out why the swedish trains all look fugly?

All of them. Especially with my non-existent swedish

I'm sure swedes speak english

Their signs don't

these trains are never going to mate: https://www.seat61.com/images/Sweden-oresund-tr... (partly because they are actually danish)

do you have a working mobile phone? google translate is a life saver.

rob: SothoTalKer linked something interesting earlier: https://www.bleepingcomputer.com/news/security/...

but you can stop any swede and ask. not only will they speak english, but they will be glad to help.

or take you to ikea, which may not be what you want.

stay wy from ikea! :O

(although the meatballs are jum)

True. The train and bus buying system is weird

CatQuest: good think we don't operate a hadoop cluster

well I read the word "elasticsearch" in there so I figured.. better

share it just in case?

/it might be insteresting anyway

hmm gcilou I don't know if swedish systems are that different from norwegian ones.. whats weird about them? :D

they run on time? there are so many trains in a day?

hmm, trains running on time, I wonder how that works

:)

just for info, ever since musicbrainz-mirror went down last month. The main MB web service has also been unusable. I think a few big users switched back and its overloading again

it even works in spain. :)

zag: we

're aware and working on it.

not yet, no

k thx

Lol I guess just buying tickets for a ride rather than a certain number of days is different than in the US

Hi, all. We have an issue where replication is permanently stuck in 'Continuing a previously aborted load' but no statements are executed and dbmirror_pending does not get cleared. This started happening at sometime between 1600 and 1700 BST yesterday. Do you have any advice as to how we might 'reset' replication? Can I safely truncate the dbmirror_pending table for example?

bitmap, ruaok ^

hmmm, I don't know what might causes that.

but truncating the table might cause the replication to surely fail. which is may already have.

do you have logs going back that you could look at and see what happened?

wow, sweden looks ugly '-'

It happened across all of our replication instances, production and pre-production databases - there was no user input. What I did see was that the previous packet did not complete applying all of its statements.

I saw someone filed MBS-9366 just now, which sounds the same. but I don't know what could've caused this yet

MBS-9366: duplicate key value violates unique constraint "artist_alias_idx_primary" https://tickets.metabrainz.org/browse/MBS-9366

Thanks. That certainly looks like it could be the same problem as it matches the time the error started.

(for us)

especially in the artist_alias table

104949 would be the problematic packet

Yes

"artist_alias_idx_primary" UNIQUE, btree (artist, locale) WHERE primary_for_locale = true AND locale IS NOT NULL

So it seems like the artist/locale is being modified but it's trying to modify it to an existing artist/locale

that's in the dbmirror_pendingdata for that packet, the first and last sequences there are certain to conflict

Certainly looks that way

what's bizarre is that the artist IDs are different in the 'key' rows

never seen anything like that

fun. :(

yeah

there was an artist merge with those other ids https://musicbrainz.org/edit/45678247

jesus2099, you broke everything!

Got another supporter @ support asking about the same issue. Let me know as soon as we have a fix so I can tell them :)

but how the hell did dbmirror see both ids for the same sequence

nvm, those artist IDs are different because they're update operations, sorry, I was thinking they were inserts

still alarming, but a lot less so :P

thanks coffee

lol

more IPs blocked, if someone complains, ask for IP first, we may have false positives on this batch

wow, alomst 6000 hits to /collection/create

and ~250k useragents from yahoo and bing

Gah. I knew crawlers would be a problem for us back in the day.

I didn't thin they'd still be a problem.

well, craw-delay is now set to 2 seconds for any bot except IA. that should make the hits per day to go to ~43200

depending how many bots are run simultaneously

someone is working at hetzner today: > We have received your order and shall inform you once we have activated your request.

yes, they are called "auto response bot" :D

one that takes 45 minutes to respond?

German efficiency! :p

I can see the thinking.

we'll have one bot respond right away. and another that responds in 45 minutes.

yeah, people will think we work on holidays.

45 mins is actually a very short response time.

Devious!

SothoTalKer: for a human yes, for a bot probably not :)

their automatic response system is slightly overloaded because they got so many requests

What is this, our ws?

:p

i bet it is just some poor person who has to work on holidays but can't actually do anything else than route your request to the right persons.

haters are gonna hate, sheesh.

new servers?

moar!!

i think you should use a server farm and not try to run websites like this from home! :X

What if you live on a farm?

animals don't count ^-^

too many spam users @_@

you could report them

no, see yesterday's conversation

hmph, why is crazy webcrawler still in this list :/

because it takes up to 2 weeks until they read the new robots.txt

hmmm

same with all the other disallowed bots

http://crazywebcrawler.com/ "click here if oyu don't want us to crawl your website"

indeed an admin could write them a mail and request an instant stop

hmmm

"Blocking our web crawler by IP address will not work. "

nice, that last set of blockings increased throughput by another ~1,000 requests/min

if you write them a mail, they put your site on a blacklist the bot ignores.

hey maybe we should?

they tal kabout "if we crawl your siteto quickly"

we blocked them totally via robots.txt so hits should gradually decline

i already know that, i read the robots.txt

I wonder if we should only allow: Google, IA and then common crawl. if someone wants our data for their search engine, use common crawl

if we still see the same amount of hits after 2 weeks, we could ask them nicely :)

what is common crawl?

also I womder what the "-" useragent/referrer is

also top request "/"

"/" is the main site

basiclaly "no user agent/referrer" ?

yes

ruaok: I don't think we should block stuff like yandex completely (which seems has most of the market share in Russia)

common crawl is a data set of crawled web pages. http://commoncrawl.org/

Slowing them should be good enough

ah so / is internal links (someone going from some link inside mb to another page)

some people might use referrer blockers or come from sites that link through those or come directly to MB

reosarevok: I'm not denying them access to our data. I'm suggesting they get it via common crawl

Sure, but is there any chance they're going to use that?

CatQuest: "/" is someone going from the front page of MB to somewhere else on MB.

reosarevok: depends on how much they care to have our data.

CatQuest: "-" is when there's no referrer info.