#metabrainz

gcrk: I don't know, just wanted to ask you :) my instance also does not log this

I not really familiar with the code there

can you link me the print?

gcrk: the logging is done at https://dev.funkwhale.audio/funkwhale/funkwhale...

gcrk: just this moment found that, was about to paste that link. perfect

I assume we need to set this to debug

oh jeah this gives a lot information

yes, that works. example output on my server: https://gist.github.com/phw/c12271678ab85f33b5a...

i have other issues with my server

lets see if I can get this, too

need a coffee anyways

ruaok: i found the source of the "None", its coming from msb lookups.

!m lucifer

You're doing good work, lucifer!

I don't understand

That None comes from an internal process, gcrk. lucifer is going to fix that now. hopefully that fixes the problem we saw earlier.

alright, so I will stop debugging :)

we *think* that is the problem. we'll have to confirm that .

outsidecontext: gcrk: could you share the json payload from funkwhale debug output for which the mbid was "None" in LB? that would help me debug and validate the fix faster.

lucifer: I can't because I could not reproduce the issue, but maybe gcrk can

gcrk: just listening to the same file would sending its log should do methinks.

gnnngh

heh

it's mostly things like "I can't have the fan on in my office because the noise comes through if I need to talk, so I'm melting"

actually, that's not a bad idea

I carry one with me starting this time of year

esp if I need to wait for a metro

nah, things are going well. I don't need to give much input, and the feedback from the reviewers has been pretty good til now

we have about 4-5 hours to present 3 years of work, so not like we can get into much detail in any of it

lucifer, I would, but currently my storage backend is down and I cannot play any file

yep, I still hate influx.

lucifer: mapping writer is now running with 3 threads, doing 40 listens/s

128 days to complete the mapping.

gcrk: ah ok! no worries.

ruaok: oh nice! :DD that's much better. also, i had taken a look at the query you shared, no idea on how to speed that up but thanks for trying.

bitmap: you said https://tickets.metabrainz.org/browse/MBS-11681 would probably need new indices?

MBS-11681: Edit History pages for discids

outsidecontext: does funkwhale plugin send listening_from field in additional info?

If anyone needs me for anything and it can't wait, today's the day, btw

lucifer: yes, as "Funkwhale"

AFK tomorrow for anything between 4 and 8 days

(still reachable on the phone most of the time, but :) )

lucifer: that's an example request without MBIDs: https://gist.github.com/phw/c12271678ab85f33b5a... (but one that worked for me and not caused the "None" issue)

outsidecontext: strange, i do not see it in those listens. this is what i see in gcrk 's listen with the None mbid `"source": "P"`

param: ^^

They made an 8 million purchase with 75 dollars? Wanna

lucifer: odd. here is another full payload example: https://gist.github.com/phw/6d4c20e202cef29aac3...

(more seriously - neat, every bit helps, probably :) )

outsidecontext: could it possibly be an older version of the plugin? i am viewing those 5000 rows which have "None" mbid and a large number of them have that "source": "P" thing.

unlikely. there is basically only one version, the entire commit history is exactly 2 commits :D

the very initial version did send "recording_mbid" even if it was empty, so you could have received "recording_mbid": null in the JSON

I guess source "P" is something else then

right, that's the issue according to my current hypothesis.

if recording_mbid is None, then it passes as a valid mbid and at a later step gets serialized into json where it become "None".

the screenshot monkey shared earlier shows the payload from what gcrk had listened to, but the source is not visible in that view

or it does not show the payload, but the data you have in the LB database actually

right, that was the frontend view. i am checking the data in db.

can you see if that one also had source P?

yes it has

let me clean up the file and i'll share it with you.

ok, that definitely must be a bug elsewhere. the FW plugin never sent anything else then "Funkwhale" as source

[musicbrainz-server] 14reosarevok opened pull request #2148 (03master…MBS-11342): MBS-11342: Add message about track durations and recording reusing https://github.com/metabrainz/musicbrainz-serve...

gcrk: lucifer and I found something. the issue happens if you configure the Last.fm plugin in FW using the compatibility endpoint in ListenBrainz. If you instead configure the actual ListenBrainz plugin it will work as expected.

It's a bug on LB side, probably with the last.fm compatibility proxy, but lucifer knows better

ruaok: gcrk: outsidecontext: i tried some things with the LB api again and wasn't able to reproduce the issue. The issue only exists when using a Last.fm plugin to submit using the proxy api.

oddly specific.

that too when using the deprecated api compat endpoint.

that's with the audioscrobbler 1.2 API. don't know whether this also is a problem with 2.0

but probably yes

but that also means it is not funkwhale specific but affects all clients using the compat proxy

at least if they send an empty value for MBID, which they probably should not do

ruaok: there's another issue though. if the payload contains recording_mbid: null or "" we retain it even in the LB api. should we do something about it?

I think we should drop the key if empty/null

makes sense. we also have a lot data in the db which already has such mbid fields so we should modify that as well methinks.

yerp

lucifer: https://web.archive.org/web/20090217162831/http...

for the MBID it says "The MusicBrainz Track ID, or an empty string if not known.", so empty value is definitely something to expect in the proxy

[musicbrainz-server] 14mwiencek merged pull request #2147 (03master…MBS-11724): MBS-11724: Fix typo https://github.com/metabrainz/musicbrainz-serve...

are there plans to make acousticbrainz binaries for arm/arm64?

alastairp: ^

outsidecontext: i see, i think we do handle it correctly. the issue is something more subtle. i'll setup a funkwhale instance and try to reproduce the issue with the proxy api.

lucifer: FW actually sent literal "None". I just checked and submitted a fix: https://dev.funkwhale.audio/funkwhale/funkwhale...

oh! thanks for the fix.

but i am still not sure how this went through our checks.

ok! i see that now. we do not validate proxy api listens.

time to dig into the AS spec and see how to propogate errors. 😞

lucifer: or don't fail but ignore the faulty MBID. not sure, but given the many clients out there that's maybe safer?

especially considering this is legacy API

also last.fm itself seems to accept such requests

yeah, i am not sure if much of the clients using legacy apis are even maintained.

this

last.fm probably doesn't verify mbids so it accepts the invalid stuff. we process this during msid calculation so its somewhat of an issue.

considering all this, its probably best to drop invalid mbid fields but still accept the listens.

ruaok: any suggestions?

I think so

tandy[m]: hi, good question. We had a look at building binaries for raspberry pi, but it didn't have sse support at the time we were looking and this had a potential to result in different calculation. we should definitely look at more "mainstream" arm chips, especially M1 but have no timeline for that yet

lucifer: agreed, drop invalid MBIDs, accept listens.

👍

<alastairp "tandy: hi, good question. We had"> fair enough, i was interested in being able to run it on my server which i use to manage my music (an sbc similar to a pi, pine64 rockpro64)

well, "sse" is a specific intel-architecture extension, arm chip floating point vector extensions are something different.

getting bit-exact floating point is a real pain across architectures. Things that really need it often end up using software floating point in order to get exact behaviour :/

(this is about acousticbrainz?)

i think even on x86 platforms, different compiler versions or optimizations were causing variations in the floating point output which is why binaries were provided in the first place.

ruaok: i get your influx hate. https://sentry.metabrainz.org/metabrainz/metric...

exactly

[listenbrainz-server] 14amCap1712 opened pull request #1511 (03master…proxy-fix): Validate mbids of listens submitted through AudioScrobbler 1.2 API https://github.com/metabrainz/listenbrainz-serv...

ruaok: opening a ticket for fixing existing listens, a simple update query would lock the entire table methinks degrading services so we'll have to come up with a better solution.

bitmap, alastairp, ruaok: we lack of disk space on williams, can you check if you have stuff you can remove?

lucifer: how many rows are affected?

zas: I already cleaned up.

ok, thanks

zas: nothing I can see from my end

ruaok: 5000 rows have mbid "None". probably many other have mbid null or "".

bitmap, yvanzo: please have a look at williams

I doubt updating 5000 rows would lock the whole table.

ok

but if you write a script to update row by row, it should be fine.

ruaok: no news from hetzner, right?

nothing.

long 1-2 days wait time we're waiting.

ah row by row makes sense.

I guess our beer swilling friends have a different sense of time than we do

zas: I should probably limit the number of concurrent backups stored to 1 until we have more space

i was wondering why we didn't start migrating williams as boingo was cleared out days ago :)

bitmap: yes :(

we should have new server soon... ;)

"soon"

[listenbrainz-server] 14amCap1712 merged pull request #1511 (03master…proxy-fix): Validate mbids of listens submitted through AudioScrobbler 1.2 API https://github.com/metabrainz/listenbrainz-serv...

alastairp: FYI, that delete query has been running for some hours now. hopefully will finish by tomorrow morning.

reosarevok: I had some questions (though I guess it's not *super* urgent oranything)

CatQuest: is it what you PMd me or something elseM

*?