#metabrainz

I guess it should be relationships 🤔

Yes, but even then it won't retunr anything.

I think I found the way

Mr_Monkey: I am continuing with PR #408 making myself comfortable with end to end tests. I have even written some basic tests which I would show you once they are done

👍

I think I should continue with tests only. Thanks

iliekcomputers: how does one deal with merge conflicts in package-lock.json in LB? its a generated file and I've got a massive conflict while trying to rebase the timescale branch

what's the url for granfa again?

ruaok: if you installed a new package in the branch, then use the branch's package-lock file. If not, then use the master file

alastairp: https://stats.metabrainz.org/login

stats. that's it. I tried statistics, logs, granfa

Mr_Monkey: ok. thanks.

Cyna: So you could do `const relationshipSet = await author.relationshipSet()` followed by `const relationships = relationshipSet.relationships()`, or…

zas: hi. can you explain to me what kind of http log stats we can do on granfa?

Ideally I'd like to be able to see how many POSTs we get, as well as how many GETs to a specific URL prefix

`const author = await Author({bbid: myBBID}).fetch({withRelated: 'relationshipSet.relationships'})`

in addition to the status code reports that we have

is that possible?

[acousticbrainz-server] alastair opened pull request #368 (master…maintenance): Infrastructure maintenance tasks https://github.com/metabrainz/acousticbrainz-se...

alastairp: if you're talking about mbstats, it doesn't collect this info right now, but I guess I could modify it so it will gather this too

zas: right, I was looking at mbstats, but I saw a number of other dashboards too. Is mbstats the one that captures http logs?

it was meant to do simple & quick mbs stats, almost in real time, so I did some cuts in features

yup, it parses pipelined web logs

it would be great to get method (GET/POST) and counts for path prefixes too. how can we make that happen?

well, mbstats is parsing a simplified log file, which (at the moment) doesn't have this field

if it isn't too urgent, I can look into this

not urgent at all, it's just the next thing on my list that I was looking at

iliekcomputers: Hi. Are you here?

would you like an issue on mbstats, a ticket on jira....?

create an MBH ticket on Jira

ruaok: Hi. There is still some timing issue between timescale_writer and RabbitMQ. When we run `develop.sh up` by the time it starts the timescale writer, the rabbitMQ pipeline is still not up. So the writer exits. Did you get me?

basically that's adding the method field to special log format, and adding parsing code so we can output this to influx

yes, I've been trying to figure out what is going on there, shivam-kapila.

it breaks when do I an import,but not when i record a single listen.

Freso: MBH-292 looks fixed to me. can you confirm?

MBH-292: Jira markup fails to mark quotes beginning with e.g. MBS-4115 https://tickets.metabrainz.org/browse/MBH-292

let me take a closer look at the timescale_writer exiting

Freso: and MBH-312

MBH-312: Support for ECDSA SSH keys (OpenSSH >= 5.7) https://tickets.metabrainz.org/browse/MBH-312

ruaok: Might be possible that just after writing 1st listen it exits.

using the submit.py script that is in the docs, I can enter hundreds and the pipeline works.

I've been working to rebase the timescale branch on master. I stopped trying to use rebase and I've made a new branch.

the merge was much cleaner, but now there are two tests failing that need to be resolved.

but they are minor. let me look at timescale_writer exiting

Oh. I can have a look at the tests. I haven't tested much as that stupid brainzutils error took my whole weekend.

ok, look. at `timescale-rebased`

Okay

iliekcomputers: Please ping me when you got some time. The brainzutils error is finally done. I need to know your views on my approach.

shivam-kapila: hey

can you open a PR and I'll take a look

?

I will

thanks

base off master, because from what I understand the issue should be in master too

(so as to not complicate the timescale branch with an unrelated issue)

Actually I just used pip install --upgrade instead of pip install in requirements.txt. Messybrainz is based on brainzutilsv1.10.0 which didnt have the methods in cache.py

Should I use --upgrade flag with all pip installs. I had this doubt.

I do not think that is a good idea

that would upgrade everything to the latest version which would lead to breakages

I am afraid it might break some code

Yes. Same

let's just upgrade brainzutils in messybrainz

Will that be able to solve issue? Bkz if someone is using the projects already, they will have older versions of brainzutils and the docker build will skip upgrading it in that case. It returns `Requirement already satisfied`

I saw that my LB is based on v1.8.0(I dont know how even when MsB was on v1.10.1) and it didnt upgrade that.

I searched on livegrep from where its getting v1.8.0 but no use

we're trying to fix the problem in production first, and i'll rebuild to fix it

shivam-kapila: docker build doesn't work like that. If you update requirements.txt then the dependencies will be installed again from scratch

someone needs to make messybrainz a package in pip

:D

the github clone stuff is not ideal

+1

MsB and Brainzutils should be in pip.

alastairp: But why didnt I upgrade to 1.10.1 from 1.8.0? Did I do something wrong.

livegrep has been very behind

I don't know the specific steps that you did, but that should be the case

there are no typescript files in there yet

we merged those like a week ago

Yeah. I noticed that too

however, if you're talking about listenbrainz + messybrainz + dependencies, then there might be some additional complications

alastairp: There are so many dependency hierarchies

iliekcomputers: it might be interesting to see how to put messybrainz in lb's requirements.txt, which would remove the git clone stuff

Actually for /code/listenbrainz directory its not git clone in real sense. we use `pip3 install git+https://github.com/metabrainz/messybrainz-s.... We use git clone MsB for /code/messybrainz

The 2 directories use 2 different approaches

here's a radical idea: merge messybrainz into listenbrainz, it isn't used anywhere else, there's lots of duplicated code. there'd be one requirements.txt so no conflicts, no need to do the git clone OR pip install :D

yeah, that's not a crazy idea

originally messybrainz was a separate webservice, but for reasons that was stopped

can't really argue against it.

so, put it as a separate package in lb and remove all the webservice stuff

CatQuest: transclusion seems to be based on a specific table (https://beta.musicbrainz.org/admin/wikidoc) - your page is not in that list, which I think is why it has "This page has not been reviewed by our documentation team" at the top

+1

I'm ok with that.

CatQuest: I'm not sure how update propagation works for page not in that table; it probably requires a "transclusion editor" to push a button

[musicbrainz-server] mwiencek opened pull request #1475 (master…csrf): MBS-10717, MBS-10778: Set SameSite=None on session cookie and add CSRF tokens to user/admin forms https://github.com/metabrainz/musicbrainz-serve...

MBS-10717: Adjust cookie attributes for new behavior in browsers https://tickets.metabrainz.org/browse/MBS-10717

MBS-10778: User/admin forms are prone to CSRF attacks https://tickets.metabrainz.org/browse/MBS-10778

shivam-kapila: huh, I just saw that - I don't know why we install messybrainz both from requiremets.txt and using git clone in the Dockerfile. that's definitely wrong

i haven't looked at that stuff in a long time, but i have a feeling the git clone is needed because we create the messybrainz schema in dev which needs the sql files

ah, that might be the case. however, if that was the reason then an alternative solution would have been to install the sql files into the package when installing from pip

iliekcomputers: have you ever seen anything like this before: https://gist.github.com/mayhem/0bce8b8f1aa08270...

but that's not important now if the solution is to merge the two 👍

this is LB on timescale doing a last.fm import.

everything goes well, but then this error is thrown an the connections to rabbitmq become wonky. lots of closed connections and lots of timeouts.

first time I'm seeing it. https://github.com/pika/pika/issues/112

>It looks like RabbitMQ is closing your connection on you, perhaps because you're redeclaring an exchange that is already declared with different parameters?

could this be happening somewhere

interesting. what is weird about it is that it happens after running fine for a while.

zas: FYI I deleted 50GB of data from boingo, in case you have reporting and noticed something funny

Mr_Monkey: this should be right, right ?

Im getting this and the length is zero

Cyna: You forgot to await authorRelationshipSet.relationships() ;)

Zastai, CatQuest: I think that just updates with cache pretty much. Max couple days anyway

damn.. so many async functions.

Other than that it looks good

Still no change in length

Mr_Monkey: I have a suggestion for entity.js file.. We have to all tests for all the functions in entity.js to ensure they are working properly. It would help avoid errors and provide clarity

For example, getEntityByBbid() could surely use test case to verify cases like if entityBbid exist or not

!m bitmap for adding CSRF tokens

You're doing good work, bitmap for adding CSRF tokens!

If you're suggesting adding more tests where tests are missing, I 100% agree !

As for the length, try seeing what authorRelationships.toJSON() gives you

on it

[musicbrainz-server] mwiencek merged pull request #1470 (master…mbs-10359): MBS-10359: Guess feat. adds bogus trailing join phrases https://github.com/metabrainz/musicbrainz-serve...

MBS-10359: Guess feat. artists from track titles do not give expected result https://tickets.metabrainz.org/browse/MBS-10359

Mr_Monkey: it returns an empty array

Hmm, you might have to call `await author.refresh({withRelated: 'relationshipSet.relationships')`

[musicbrainz-docker] yvanzo merged pull request #131 (mbvm-38-dev…admin-scripts-doc): Document (and lint) admin helper scripts https://github.com/metabrainz/musicbrainz-docke...

apologies. Had to go for dinner without telling.

alastairp: I agree with what iliekcomputers said about using 2 imports for MsB

It is Chinese Language Monday!

People up for reviews: yvanzo, bitmap, pristine__, ruaok, Mr_Monkey, Freso, zas, alastairp, reosarevok, shivam-kapila, _lucifer, Cyna,CatQuest, ishaanshah[m] – anyone else who wants to give review, please let me know ASAP. :)

yvanzo: Go!

Hi!

Last week, I released a new version of MBS and updated beta.

I finally merged, deployed, tested, fixed, and redeployed MBS-10546.

MBS-10546: Dump MB Solr data along with MB DB full export https://tickets.metabrainz.org/browse/MBS-10546

MB search index dumps are finally available at ftp://ftp.eu.metabrainz.org/pub/musicbrainz/dat...

!m yvanzo

You're doing good work, yvanzo!

I updated PR to welcome MBS contributors with iliekcomputers’ review.

I locked New York down, due to bug (loading too much stuff), as well as London Symphony Orchestra (relationships only).

>I locked New York down

nice

sheesh. even more protests now...