#metabrainz

if someone deployed something that may use the db on sunday, please tell ;,)

apart that, i tried to setup an alert for packet replication issues, but i need to start over, nagios was reluctant to help me

CatQuest: There

🙇 thank you bukwurm

plus usual PRs reviews, edits, etc...

fin. ruaok ?

k

normal week for me.

accountants, a meeting with the lawyers to cleanup a mess that c created.

bank account still not unfrozen, starting to doubt it will ever be.

CatQuest: It's not, I created the gif online, conversion made the video slow.

helped with replication packet issues, realized that our docs are woefully out of date. we need to work on that.

did some tiny metabrainz navbar cleanup and deployed a link to our shop which promptly generated one sale.

thanks Major_Lurker!

:D

I'm going to be AFK wednesday afternoon - sunday.

if you don't get me your invoice by tomorrow afternoon, you'll need to wait until monday.

fin.

iliekcomputers: go

CatQuest: Oh sorry, I thought you were asking why it's slow 😅

nah :D

I'll go in a sec, I'm deploying CB rn

(iliekcomputers is last one left on the list. If anyone else wants to go, now's the last time to speak up.)

It's WIP, not live. Let's talk about it after the meeting.

oh

okay

Freso: hey!

Can i go after next topic then?

Oh, right. CatQuest is left too! Sorry!

CatQuest: Go!

bukwurm: yup

iliekcomputers is next afaik?

CatQuest: go, I'll go next

👍

just been working a little bit on instruments and releases.

had a very interesting discussion with LordSputnik and bukwurm

in which bukwurm made a document summarising what we talked about

(and has been working on it too ass is the review he made :D)

she?

but mostly preparing seeds and stuff for spring took most of my time this week, should be back on form next week.

it's valborgsnatten this night so I'll be celebrating >:D

🧹

Finn!

iliekcomputers go!

this week was much more productive than the last few, I'll do updates project wise

(iliekcomputers is really the last one left on the list now. :) If anyone else wants to go, *now*'s the time to speak up.)

LB: worked on moving data to the new BQ schema, ruaok pointed out an interesting problem, we discussed solutions to that, gonna code up something today.

CB: worked on comments on reviews, and a bunch of open PRs. proud that the only PRs left now are schema changes. thanks ferbncode!

\o/

yayy. Param on fire. Much parties this week? :D

\o/

AB: met with rsh7 and ferbncode over lunch to discuss how to approach her project in the best way, this is a first for me as a mentor. (was kinda productive)

chhavi_: indeed. :)

rn, working on changes to CB to reduce load on postgres.

fin!

!m iliekcomputers

You're doing good work, iliekcomputers!

Alright! Moving on!

zas: pg spof & scalability

basically we need to improve pg stuff

currently we have bowie (master postgresql) and queen (slave, underused)

while it provides some redundancy, we have any load balancing yet

moving CB (beta and prod) to queen, although I'm not sure how much load CB causes anyways.

so that's a single point of failure (spof): if bowie goes down, we have no automatic switchover

:D

discussing possible solutions is too long here and now, but i'd like people with ideas to tell, we'll try to address those problems asap

some interesting comments about pgpool there https://www.reddit.com/r/PostgreSQL/comments/6k...

our lack of experience in this field is the core issue, if anyone knows someone that could help on this stuff, please make the link ;)

but I think pgpool or looking closer at citus are the best options I see at present...

for sure we have to dig those options

but from what i read none are exactly easy to set up

certainly not pgpool

that's why i think we need to find people willing to help that have experience with whichever tool we go for

we can start by postgresql channels

do a blog post about it. asking for recommendations.

that has worked really well in the past.

it has

<3 our community

yup i agree

Anything else you need to discuss about this?

zas: ?

not really, but that's something we need to put in priority before it becomes a very serious issue, the db is the core of our ecosystem...

Absolutely.

and regarding the current situation i'd say it's urgent ;)

So: make a blog post as well as reach out to people in pgsql channels?

👍

zas: Not saying it isn't urgent, just wondering whether it's time to close up the meeting. :)

it is, i'll do the blog post during this week

Alright.

Thanks for your time everyone!

</BANG>

bitmap, yvanzo: any idea about what started yesterday around 16:00 UTC (which is the start of the load issue) ? i looked at many things and still cannot explain the load on bowie. We know restart/reboot changed nothing in this regard and it isn't due to higher than normal number of requests.

Leo_Verto: Let me know what resources you need for SpamBrainz.

I'd really like to get my hands on some sort of preliminary spam data set

zas: me neither

Leo_Verto: Ok, do you have an MBS setup at hand?

run some basic data analysis on it, get a feel for what exactly we're dealing with here

yvanzo: nope, I'll get one set up asap

CatQuest: I am planning on having integrated search include all types of entities, but raise warning only when a same entity exists with same name (and then making addition of disambiguation compulsory).

yvanzo: did you restart all mb website & ws containers ?

bitmap, zas: it doesn’t seem to be related to latest MB release either.

there are extra disk activity (especially swap) on many nodes when it started : https://stats.metabrainz.org/d/000000052/hetzne...

zas: on Saturday

but you didn't today ?

CatQuest: I plan on modifying the name matching function to add links to entities with same name, exactly as it is in MB.

no

nothing cron-related or deployment-related I'm aware of

zas: I restarted beta containers

(today)

ok, let's restart all prod containers to be sure

bukwurm: that does sound good

(and very inline with what mb currently does)

bitmap: can you do it ?

And maybe make the warning disappear if the user enters a disambiguation?

all mbs prod containers?

yes, website & ws

ok

CatQuest: Of course, it'll all be inline.

yep, since then it indicates "yes this was the problem, you solved it no more a prolem"

yes!

bukwurm: yep. That should be. I

CatQuest: is keeping track of you, so I need not be worried. Couldn't help but be curious. :) :p

ok, CB prod now using postgres-slave

I still think that the popus of the other bit should be inline and expandalbe sections . but LordSputnik said not "for now"

now, CB beta.

Making warning disappear - that'll be 2 minute job max. Let me do it right now.

chhavi_: All views are more than welcome.

:D

one thing to keep i nmind as that mb is having a redesign. we should eventually work on getting parity between bb and mb apearance wize

iliekcomputers: thanks!

take the best of both worlds so to speak

bukwurm, chhavi_ definitly

zas, bitmap: restarting mb prod containers

[critiquebrainz] paramsingh opened pull request #207: Merge production into master (master...merge-branch) https://git.io/vpRQf

ruaok: approve and I'll deploy?

CatQuest: I think as long as functionality remains same, it's all good no matter how it looks.

I'm working for man "user" prespective. a "design perspective" is chhavi's and als oimportant

yvanzo: ok, I just did bootsy and burnside

you can do the rest if you'd like :)

ok, we had the memory issue on ws containers at the time it started

bukwurm: the thing with that is that look does finger into functionality. somewhat

it doesn't explain why it didn't stop after it came back to normal though

it needs to "look" i na way that makes it easy to use ()ie funtion is obvious