#metabrainz

hey samj1912

Hey o/

not much issues with new search, it performs well enough, but we lack of cpu power on peak times

that's more an issue with hetzner cloud performance

It's still on 4 machines?

apart that, i need you to upgrade solr, because it's blocking the next step to the drop of ws/1 (current solr version has security vulnerabilites, so we can't open to public yet)

yes, 4 nodes + 2 haproxy

Upgrade solr as in?

performance is still better than old search though ;)

solr version

Ah to 7.5?

to the most recent one, afaik we didn't because of some issues, prolly with mbsssss, dunno

I'll look into it

solr search peak around ~160 req/s with response time <300ms

Nice!

we are cpu-bounded, and hetzner cloud machines have variable cpu power... so performance varies, it works good enough for now though

in best cases we go down to 60ms response time when load is low

zas: what was the name of the test vm that we had for solr tests?

nvm

found it

Wait you can have profile pictures on IRC? 🤔

yeah :P

If you use irccloud

zas: you around?

yep

I synced the deps properly

pushing out the new solr ver of mb-solr

Ooo cool

the next time you want to do so

simply download the latest solr form the website

do a find for all the jars

and update the relevant versions in pom.xml inside mb-solr

but the deployment does'nt rely on mb-solr (do i miss something?) https://github.com/metabrainz/mb-solr-cloud

it does right?

it gets the jar?

ah yes

https://github.com/metabrainz/mb-solr-cloud/blo... is implied too

yes

I updated both to 7.5.0

see the test branch on mb-solr

building out a new jar

and testing on mb-solr-test

can we deploy nodes by nodes ? i mean if we upgrade one node, is the solr cloud supposed to handle it ?

hmm

not sure

it should

ah yes it should

i hope so; but i'll be ready to shut down all if it goes wrong, so we need to proceed on low activity times (in european morning)

cool

at worse you can just bring a node down

and revert

yes

i'll be busy this weekend, but i'll give it a try monday morning

about picard, we are about to release 2.1, outsidecontext and i think we are almost done, still few minor issues to solve before release, but overall it looks good, it should happen next week

nice :D

george: You can also have profile pictures with Matrix. :-) https://opensource.com/article/17/5/introducing...

ruaok++

ruaok: Annual report looks great. I left a comment with one question I had

Freso i was planning to make a presentation on MB

i need your help with that

Nice work and efforts by the MeB team in 2017. Even with the hiccups, it looks like it was an eventful year and I got a better picture to how things are ticking

I appreciate the transparency

zas: http://35.193.119.46:8983/solr/#/

it works

good :) thanks !

can you provide a PR for mb-solr-cloud ? i'll merge and attempt a deployment on monday

yeah

in case you ever want to test it on queen again

pushed a new release https://github.com/metabrainz/mb-solr/releases/...

zas https://github.com/metabrainz/mb-solr-cloud/pull/3

have fun ;)

Also, I might be giving a talk on solr this fosdem if it gets accepted :D

Let me know if any of you peeps are there

jwf: bad copypasta. that column is the user privs column from the DB. I need to remove it.

fxied.

reosarevok: take a look, please?

Oh no

Ok, I have a Wikimedia event starting soon (well 20 min ago, but :p) but I should find some time to read that while there

no rush.

ruaok or zas: the wikidata bot still hasn't made it past areas :-( can one of you do a `SELECT count(*) FROM bot_wikidata_area_processed;` on the `wikidata_bot_db`? I'm curious what the difference between that number and the number of area-wiki* relationships (~110k) is

k

21914

That's... not a lot

given how long its been running, it smells like a bug

Think so, yeah

My bots go through that many entries in a few days tops

(different sources etc, but I assume the delay is connected to WD not MB)

reosarevok: I just gave you write access to the doc.

reload and just make the changes.

ouch, that number is unexpectedly low. if there's time (doesn't have to be today), could someone send me a snippet of the logs? the bot is usually very chatty

very chatty == will take a moment.

getting logs out of docker is a pain.

hmm.

1.5GB uncompressed so far.

yeah, I really want to fix docker logging on our setup too

I kind of thing the best way is to have a centralised logging server, but that seems annoying to set up :(

so annoying that we've not managed it in 2 years.

ruaok: ok

I'd like to see the logs too if possible

I'll publish it once the xz finishes.

could be a while still.

huh?

651k. Down from 1.5g.

emailed to reo and Mineo

thanks

ruaok: Ah. All good! :+1:

ruaok: "The silver and gold supporter categories picked up in 2016; also we gave up on the mobile tier in 2016" 2017 I assume but confirm

yeah

ruaok: seems pretty good in general, changed a few things

thank you!

[picard-plugins] phw closed pull request #189: Add format_performer_tags plugin (2.0...2.0) https://git.io/fpbto

[picard-plugins] rdswift opened pull request #190: Add UI changes by outsidecontext / phw (2.0...2.0) https://git.io/fpbGG

howdy, I have a local Musicbrainz server setup and trying to run an import from a dump

e.g. ./admin/InitDb.pl --createdb --import /tmp/dumps/mbdump*.tar.bz2 --echo

but that command gives me an error: Can't locate JSON.pm in @INC (you may need to install the JSON module)

is this expected? I followed instructions from https://musicbrainz.org/doc/MusicBrainz_Server/... to setup and don't think I did anything weird

hi wagu, did you install Perl dependencies by following https://github.com/metabrainz/musicbrainz-serve...

@yvanzo: I did not, but assumed that was done as part of https://musicbrainz.org/doc/MusicBrainz_Server/...

am I wrong? I thought this was a prebuilt server

wagu: oh, you mean you installed the VM?

yes

and I do see perl dependencies I think in ~/perl5/lib/perl5 so pretty sure they exist

Ok, that might be an environment issue.

what do you mean?

You need to set environment for running Perl command with: eval $( perl -Mlocal::lib )

@yvanzo: ah thanks so much! that works