#metabrainz

huh. the VM hail mary script produced a 17GB ova file.

solr-2 runs on meltdown-patched hardware https://chatlogs.metabrainz.org/brainzbot/metab...

17GB is the size of the last image from almost a year ago, shouldn't this one be a bit bigger?

164.GB. what gets produces is basically black magic - I don't really understand what it does. the virtualbox docs hurt my head.

ftp://ftp.eu.metabrainz.org/pub/musicbrainz-vm/... will be done soon

someone please try it -- I haven't looked. that is literally the output of 1 script.

mind boggling really.

I would but downloading 16GB on my current connection takes more than two hours

make a virtual computer. install tons of shit, load data, index it. zip it all up and ship one country over. one script.

that is what nights are for.

my first linux install was downloaded onto floppies. it took more than a week of nights to download that.

aight, bed. nn all.

ha, night!

Good morning o/

zas, solr lb1 is using up a lot of its disk space

Logs?

morning!

ruaok, Für die virtuelle Maschine musicbrainz-vm konnte keine neue Sitzung eröffnet werden.

RawFile#0 failed to create the raw output file /Users/robert/metabrainz/musicbrainz-vm/ubuntu-xenial-16.04-cloudimg-console.log (VERR_FILE_NOT_FOUND).

Fehlercode:NS_ERROR_FAILURE (0x80004005)

Komponente:ConsoleWrap

Interface:IConsole {872da645-4a9b-1727-bee2-5585105b9eed}

sorry, probably should've pastebinned that

deactivating COM1 fixed that, VM is currently stuck in the boot process but using a whole CPU core so I'll let it run for a while

zas, when you get up, please look into the space problem with lb1

It's rising at almost 10% per 12 hrs

LordSputnik: Ok, Tuesday will be fine by me.

I'll have a look

samj1912: ping

is SIR near-realtime updates defeat caching ?

See https://stats.metabrainz.org/d/T4MODrIiz/solr-c...

samj1912: about logs, i'll keep an eye on that, but i think we have enough disk space for now, i'd like to keep logs at least during first days, they'll help to debug issues. We only have 20Gb on those instances

samj1912: also i'm pondering about upgrading to haproxy 1.8 mainly for the seamless reload feature (currently connections error when reloading the config)

Hey zas, yup, cache is invalidated once new docs are updated

as a whole it seems

Hmm, I'll see what I can do about that

yes, please, not sure if there's a solution, but invalidating too much caches on small changes is defeating the cache purpose"

I am not even sure if the entire cache is invalidated or not

But I will see if I can implement a write through cache

i'm not either, but graphs shows "nulls" which looks like "no cache"

Yeah

But immediately after the cache hits are at normal level

Which seems weird to say the least

Another problem with the graphs was the values when there are no ops

It was showing 1 post op/s when it should infact be 0

See the graphs yesterday from around 1300 GMT

yup, we still have things to understand ;)

but overall it works

response times decreased a bit on ws side, due to extra hops

but it has no impact, 15ms->25ms

Cool

moooin!

yvanzo: see Leo_Verto's comments above ^^

ruaok: moiiin, currently loading the VM.

samj1912: https://tickets.metabrainz.org/browse/SOLR-89

SOLR-89: Allow partial match of aliases, not just names

That's what you wanted, right? :)

Yup

hi iliekcomputers: replication packets use 2 different ids in dbmirror_pending file, to figure that out I was searching about the packets online and I found a script which is making use of these ids.

later i found that the repo contains the whole code to apply the packets

Nice, link? Maybe we can reuse the code.

the person is lalinsky, i think the same person who wrote mbdata models

Woah, nice. 😮

We could easily use this with a few changes, I guess.

There's no license in the repo. :/

You can ping luks

I thought that repo was known and used a lot?

And yes, he wrote mbdata

rsh7: you should contact him and ask for permission before using it, mention that we'll probably add it to AB which is GPL2.

samj1912: what would the best way to contact him be?

His oxygene.sk mail id, he also hangs around irc at times under the nick luks

His Gmail ID is his github nick

samj1912: thanks a lot!

rsh7: https://oxygene.sk/about/

So gh_nick@gmail.com should also work

zas, was that storage problem fixed?

Lb1 is upto 40% usage now

Very verbose logs or simply low storage?

Alright, I'll contact him and inform you. Thanks, samj1912 & iliekcomputers.

rsh7: great, keep me updated. :)

reosarevok: you around?

Ish

any idea what this person is talking about? https://community.metabrainz.org/t/the-new-solr...

all of those searches work for me

yvanzo, did the VM work for you?

samj1912: as tag lookup?

That's what they mean, the second part of the page :)

ah

I think the alias_query sub is friggin things there

no need for search modifiers anymore I think

hmm, fixed it

reosarevok: he is talking about indexed search

god knows what he's talking about (since he is saying indexed search is not working but direct search is), I am typing the exact queries and its working, as for tag lookup I am fixing it

Hmm, well let's still get the broken bit fixed and then see :)

[musicbrainz-server] samj1912 opened pull request #689: Fix tag lookups for artist (master...taglookup) https://git.io/fwl4v

done

ah, he was confused

fixed now :)

updated the banner with the blog link

ruaok: SOLR-89 will require a re-index

SOLR-89: Allow partial match of aliases, not just names https://tickets.metabrainz.org/browse/SOLR-89

what do you want to do?

also, we need to figure out someway to do re-indexes without downtime

samj1912: For future reference, if possible, it's better to do the blog first and then set the message. Anyone who's clicked the message away will have it pop up again every time you update it, which could potentially lead to some annoyance. :)

(As long as it just happens once, it's probably fine, but in case you ever feel the urge to tweak the message every few hours or some such… :))

Sure

iliekcomputers: ping!

Pong

Lukas says """ You can assume the project is licensed under the MIT license.

Additionally, if the MetaBrainz people would like to take over the mbslave project, I'd be happy to move it over. I got too disconnected from MB to maintain it. Can you please ask about this on IRC? """

rsh7: great, then. I think we should add a copyright notice with the MIT License over the files you reuse/modify with copyright Lukas and you.

Other than that, I'm not sure about MeB supporting mbslave, I guess bitmap and yvanzo would like to chime in on that.

hm, if Lukas is too disconnecte from MB, does that mean he won't continue updating mbdata either?

If that happens, we'd have to start maintaining mbdata once so many projects of ours start using it.

iliekcomputers: seems good