#metabrainz

3 failures? how will you live with that samj1912?

:D

peacefully :P

good plan. be buddha.

wow, a peak of 60 result cache hits?

thats a lot higher than I expected.

that's gotta help a lot.

zas: https://stats.metabrainz.org/d/T4MODrIiz/solr-c...

zas, samj1912: one test I would be interested in doing is to see if we can get a smaller cloud instance to handle the traffic.

how come it stopped after 12:30?

because three nearly idle instances that are required to run solr cloud seems less than ideal.

hmm, looks like the replayer stopped?

the disk usage stat seems broken

and during peak the one node could't keep up -- surprising it only got 3 errors.

!m samj1912

You're doing good work, samj1912!

ruaok: its not?

its the disk space usage?

oh, perhaps so.

but yeah, the replayer stopped at 12:30

I see disk io.

weird

no traceback on the replay?

nope

log rotation.

oh maybe during the log rotation

yeah

input files rotated from under it

was gonna say the same :P

catch the exception and restart.

no exception though

i'm off for lunch, haproxy machines are untuned yet, but should work

I made a symbolic link to the access log

probably that's why

I'm deeply impressed that that it nearly handled the whole load.

yeah, that would do it too.

a restart *should* work once the new log appears

ruaok: pong -- will submit eval this evening

ok, thanks!

ruaok: peak avg. hit rate was 81r/s

so the actual peak reqs/s might have been higher

ok, keep the replay going and then also turn beta to the new solr cluster once the haproxy stuff is working.

okay, what do you wanna do about the indexing?

have you tried indexing on queen since the memory leak fix?

nope

let me get all the nodes up and ready

try it and watch it carefully. at the first sign on the apocalypse, stop it.

yup

of

it would be ideal to get it updating from production.

zas: how do you update the solr cloud repos on the machine?

it always gives me an error on pull

public key denied

do you connect with ssh -A ?

ah no

samj1912: ping

zas: pong

did you enable solr2 & 3 already?

yup

setting solr cloud up for beta

can you do a bunch of requests to the floating IP 195.201.47.148 ?

i'm setting up telegraf for haproxy, i need few numbers

requests like normal solr ones?

yes

okay

zas: fyi I have deleted all the indexes

so it wont show any results

but it will respond

zas: which port?

80

http

zas: done

around 20k reqs

let me know if you need more

how long did it take on your side ?

18.5 seconds?

but its not accurate since there is no data

ok, it seems to work, but we'll need to improve that with much more data

doing it right now

at least haproxy is reporting stats

cool

and failover seems to work

where can I see them?

but that's just a proof it works, i don't know yet about useful metriics

cool

I'll put some data in there

btw, i took the smallest instances for load balancer, i'm curious on how it performs

okay

1 vcpu, 2gb ram

zas: where rmq currently?

trille i think

cool, thanks

zas: dont start sir on queen

k

zas: I am reindexing off of queen

load might get high, but ram should be stable

recordings should finish in 4 hrs acc. to my calculations

actually, we might be looking at a record 3hr for recording reindex

Hello there!

hey MrNIce

Hi Samj1912, can you help me to find out a bad setting or a bug in Picard?

or do you have something in mind?

I have an issue using Picard and would like to fix it

cool, what's the bug?

MB Picard version 1.4.2 Opensuse Leap 15.0 I have folders sorted by CD with music files inside. I drag 1 folder to middle window (I don't know the name of the window) and I click "Cluster" then I select the title of this CD and click "Lookup in Browser". Some time I get the page in my browser, it's OK but sometime I get in my browser a page with text only "Index of file:///home/username" and the list of my home folder files includ

Address bar has: file:///home/username/ When I say sometime, I mean with I CD folder it is always OK but with another it is never OK What does that mean? I did an upgrade from Leap 42.3 to 15.0 recently. I never had that with the previous version, I don't know what MB version was in 42.3.

MrNIce: we have stopped dev. for 1.4.x

You should switch to 2.0 beta

As said, I run Linux Opensuse leap 15.0, in the repo I have 1.4.2 version. I'll have to install manualy...

2.0 is a beta version, when stable will be released?

MrNIce: all it should take is `pip install https://github.com/metabrainz/picard/archive/ma...

yay, serge is back

MrNIce: the beta is pretty stable, we are looking at getting a 2.0 final version out sometime by the end of this month