#metabrainz

mo’’in’

moin

[sir] yvanzo merged pull request #105 (master…bump-req): Update requirements mbdata and pysolr https://github.com/metabrainz/sir/pull/105

Moin !

Mr_Monkey: Hi!

Hi akhilesh

Mr_Monkey: How can I update bookbrainz_test database schema?

Why are these tests failing https://gist.github.com/akhilesh26/95803febadc3...

akhilesh: You can run the sql script at sql/migrations/2019-04/up.sql against bookbrainz_test

Yes, tests are failing because of the schema change.

Ok :)

If you have psql on your local machine, yo can run `psql -h localhost -p 5432 -U bookbrainz -d bookbrainz_test < ./sql/migrations/2019-04/up.sq`

If not, you'll have to run a similar command from within the website docker container, using postgres instead of localhost

You also need to run this script afterwards:

`psql -h localhost -p 5432 -U bookbrainz -d bookbrainz_test < ./sql/scripts/create_triggers.sql`

Hmm, in the commands I posted, you'll probably want to replace the user (-U flag) with postgres instead of bookbrainz

Yes, It require, how to replace user with postgres instead of bookbrainz?

`-U postgres`

instread of `-U bookbrainz`

ok

Mr_Monkey: Running now, Thanks!

Great :)

Mr_Monkey: In which issue, You are working ?

in BB

Currently on ListenBrainz, but about to go back to tackling entity merging

I am working with date validation tests, then I think we should move to author_credit

also, it is imp

Mr_Monkey ^

Agreed

That gives me a day or two to think of a plan and what needs to be done

By that time, I will try to finish date related work.

Hi. I'm struggling to get full indexing working properly on Solr. I note a lot of errors in logs about missing fields when running reindex from mb-sir. It seems that the submodules for mb-solr are very old and aren't aware of some schema 25 fields. Does anyone know what revisions I should be using to reindex against a schema 25 database? Or should I be using a < schema 25 version of mb-sir to do the indexing? Thanks!

Hi Matthew_: are errors about label/isni only?

Hi yvanzo. I'm mostly seeing issues with artists - haven't got any further than the 'a's ;) I'm seeing 'unknown field 'primary_alias'' a lot.

schema 25 should still work with previous versions of mb-solr/sir

Matthew_: Ok, which versions of mb-solr and sir do you run?

sir master is ready to work with schema 25, but mb-solr is still work-in-progress

We still run previous versions of mb-solr/sir in production.

For mb-solr, I've tried 'master' should I be using v3.0? For sir, I don't see any branch/tag that leaps out as stable so I'm also on master

OK. Thanks, yvanzo. What versions/revisions would you recommend for those components?

mb-solr v3.0

and sir commit 5109d2606eb84a444b1c15dd6efe70571e67afd7

Great. Thanks, yvanzo!

Matthew_: you will have to patch sir requirements.txt because mbdata 2017.6.2 is no more available from PyPI

Thanks yvanzo. What version should I be replacing it with? most recent? 25.0.3?

Same version, but it should retrieved from github instead as follows:

OK. Thanks. Will suck it and see

something very wrong with solr search server

Matthew_: I pushed that fix to metabrainz/sir branch schema-24

ah it's back, 2 nodes were down... not sure why & how

Brilliant. Thanks yvanzo!

[listenbrainz-server] MonkeyDo opened pull request #572 (master…search-song-spotify): Search song spotify https://github.com/metabrainz/listenbrainz-serv...

ignore telegraf alerts, that's me, just reboot grafana machine for upgrades

i'll reboot jira machine for upgrades and new cert deployment

yvanzo: you around ? in case of...

zas: yup

i'll reboot jira now

voilà, jira is back, with new certs, system & kernel were upgraded too

Mineo: sir reindex never stops, it cannot be interrupted using Ctrl+c either, should 'queue_to_solr' have a 'live' parameter we could disable?

ruaok: *.meb certs deployed

yay, thanks!

!m zas

You're doing good work, zas!

Project bookbrainz-site build #2183: passed in 3 min 42 sec: https://travis-ci.org/bookbrainz/bookbrainz-sit...

ferbncode: Hi

ferbncode: Finals could always be better, but it wasn't too bad.

the icon is grayed out

alastairp, and you've hit the `scan` button I assume

no, it wasn't clear that you have to scan before you can submit (in this case I know that the album has no fingerprints, so I assumed that scan wasn't necessary)

also, scan only works if there are files in the left, but since they all have mbids, they all go straight to the right

anyway, I found the scan button now, doing that.

no, it's not very clear. it trips me up every time I've done it.

You can select and drag a set of files back to the left to do further operations on them at any time.

thanks

looks like acoustid is down anyway, I'll finish tomorrow

argh. anyway, iirc the submit button will also be ghosted if there's already a match (same mbid and acoustid).

spellew: Cool, just let me know whenever you're done fixing the timelines. :) Also, hit me with any questions that you encounter o/

ferbncode: Sure thing

alastairp: re comments on PR, if the rows aren't ordered, suppose the current max is 3, the query then returns rows 8, 9, 10 without returning 4, 5, 6, that'd be a bug.

iliekcomputers: mmm, sure. but it only has to return 4, 5, 6 eventually. if it computes 8 9 10 then the left join against highlevel will have non-null rows, so they won't be selected, and maybe 4 5 6 will appear

but once 10's hl data is calculated, the new max would become 10, so the query would only return ids >= 10.

that's the way I wrote it, not sure if there's something I'm missing.

mmm, yes. this is the bit that I was missing

you're right. if we want to have a limit we need to do it in order

OK, I think we should try it splitting into 2 queries too then - as ruaok pointed out last week, doing order by with the json table in the picture makes queries quite slow

do you understand the queryplans that I asked for?

yep, i'm working on it.

doing it on frank to get actual values.

yeah, good(ish) idea

don't break it :)

* no order by, no limit -- https://explain.depesz.com/s/8fsx

* new query -- ll.id >= 10**6, with order by -- https://explain.depesz.com/s/Svet

* only join ll and hl, ll.id >= 10** 6, with order by -- https://explain.depesz.com/s/tlqV

the query joining ll, llj and hl is faster?