https://listenbrainz.org/similar-users suggests that the two users listed have identical listening histories, but that doesn’t seem apparent from looking at their profiles, and also their MB profiles don’t seem to hint at them being the same person. Any chance someone with more direct access to the data can give them a look and give me some more insight in what is or might be going on?
lucifer
Freso: that page is currently bugged. i'll look into fixing it soon.
Freso
lucifer: 👍
lucifer: Is there a ticket I can watch? :)
lucifer
Freso: none that i know of. but i'll ping you once its fixed.
ruaok: hi! as you can see from my later comments, it worked but showed another problem. to fix the issue completely we also need to offload the spark invocation to a different thread. i'll probably take the MBID mapping writer route for that.
ruaok: alastairp: let me know when you have some time. need to discuss about artist credit id -> artist mbid in LB stats.
ruaok
I have a few mins. (On mobile)
What is the q?
I suspect that you need a mapping available in spark, yes?
lucifer
lb stats now return artist_credit_id and artist name, but frontend needs a artist mbid so we need to add a translation step somewhere
ruaok
That should be easy to create when you have an mb db connection
lucifer
yeah so doing on LB side is probably better.
ruaok
Look at the mb schema and perhaps get reosarevok to help you with this query, if it isn't obvious.
lucifer
when LB receives stats from spark, replace artist_credit_ids with artist_mbids, right?
ruaok
Then write the data to a pandas dataframe then parquet, then import.
lucifer
ah ok, understood
ruaok
A longer term solution night be to have them included in the mbid mapping.
Export ac id and array[mbids]
It would make the mapping dump a bit larger...
lucifer
makes sense
we are going to need artist mbid almost always though, artist credit are not very useful to expose to users.
*artisy credit ids are not
thanks, i'll try the separate dump for now. we can look into the longer term solution later
peterhil has quit
yvanzo: hi! i am investigating some rabbitmq issues in LB. i see sir is having those too according to rabbitmq-clash logs. mentioning in case its relevant.
annoy fails to compile on the prod build action, I just tried it locally from scratch and it worked fine. Perhaps something related to a cached layer that's incorrect?
I need a way to check the version of gcc in the build container, I'll check
but I've just had a thought - It could be something weird like the the build farm hardware advertising that it has a particular avx extension
which could cause problems if we try and build on one architecture and then deploy on another :(
lucifer
yeah :/, we'll need to configure according to our machine's hardware
alastairp
they have a fix for it which I think will help for now - because we're compiling on a lower version of gcc it'll skip the extension, but this is definitely something to check and fix when we upgrade gcc
OK, I just worked out what mtune=native does. it means "use all of the flags that are available on this specific build machine", which isn't what we want
in newer versions of gcc there are zen flags that we can use
lucifer
nice.
alastairp
oh amazing... I guess you don't know which machine you're going to run on!
let me open another PR to upgrade the version of annoy. at least we'll use the right extensions when using gcc 5. We can address it again when we upgrade to a new baseimage
you can ignore the latest commits, i am trying to come up with a threading solution but haven't reached there yet.
i have written down the issue i found in context of the request consumer but these issues exist elsewhere as well. in api, spark reader so on. i didn't check ts writer yet so don't know about that one.
peterhil has quit
peterhil joined the channel
yvanzo
hi lucifer: thanks, it's likely relevant, btw you would be very welcome to look at sir again after you are done with your current project.
lucifer
sure :D
yvanzo
yyoung: thanks, did not reply the other thread yet, will check your PR as well.
BrainzGit joined the channel
alastairp
lucifer: sure, I'll look at it tomororw when I'm on LB tasks
lucifer
awesome, thanks!
CatQuest
ok yvanzo I'm here now
BrainzGit
[acousticbrainz-server] 14alastair merged pull request #409 (03master…annoy-valid-extensions): Upgrade annoy, force a CPU architecture to avoid unavailable extensions https://github.com/metabrainz/acousticbrainz-se...
Thanks a lot lucifer . I have written a medium article as well recently regarding the GSoC experience. Will share it when we have the meeting in an hour :D
alastairp
zas: I need to rsync 50gb of data between servers. should it be faster through public IP, or private network, or should they be the same? I'm seeing only 3-6MB/sec transfer, which seems low. Are you imposing any speed restrictions?
(frank to clash)
zas
nope no speed restriction, it should be faster over private network
ruaok
Freso: slow week for me, just some bits and bobs, was on semi vacation am on vacation now.
Last week I mostly worked on some improvements for the ListenBrainz player. Some stability improvements and the addition of new features, namely native notifications and media control. Associated PRs are #1539 and #1561 which have more detailed descriptions.
You can test this all out on test.listenbrainz.org <http://test.listenbrainz.org/>;, and feedback would be very appreciated if you find anything that looks broken or unintended (or if you’re happily surprised too).
