#metabrainz

phibb, is this on a windows system?

yeah

I'm not 100% sure, but perhaps Picard is chaning the elipses into three periods before it gets into the renaming.

You might try either moving the replacement to a script in the scripting section rather than the renaming section, or

I tried those too :(

yeah this is from the scripting section

try replacing three periods rather than the elipses.

yeah it did not match those either

regular or regex

Well, I'm out of ideas. Sorry I couldn't help.

np

Is it showing up as an elipses in the filename on Windows?

or as three periods?

I wonder if there is more than character code that displays as an elipses, and you're replacing a different code than is in the string? That's a "Hail Mary" for sure. ;-)

loops like elipses

esp when I copy/paste

Yeah i'm trying to just remove or replace w/ space

it doesn't detect the search pattern

phibs: file naming pattern can't change tags, so that doesn't make sense in the file naming pattern

what you could do is take the %title% in your file naming pattern and turn it into $replace(%title%,…,wat) maybe?

or even just wrap the entire file naming pattern in a $replace call (just put the whole existing file naming pattern as the first arg)

Moin

kepstin: I mean it definitely works if I do simple works

words *

the -> something else

so it should be able to strip the elipses

mo’’in’

phibs: The ellipsis replacement definitely works for me with your script and e.g. the second track on https://musicbrainz.org/release/da53e497-8c61-4... . But a few ideas:

- Options > Metadata > "Convert Unicode punctuation characters to ASCII" might be enabled

- Is some other script doing a conversion?

- Any plugin that might change the title?

also, if you do `$replace(%title%, …, wat)`, the extra space before `…` _does_ matter

[picard:master@e16c7c4 - build #150] CI passed! (https://travis-ci.org/phw/picard/builds/597527216)

metabrainz/picard#5113 (picard-2.2.3 - 4cfa625 : Philipp Wolfer): The build passed.

Change view : https://github.com/metabrainz/picard/compare/f9...

Build details : https://travis-ci.org/metabrainz/picard/builds/...

moooin!

pristine__: I've got some time to chat, finally.

Thanks

A sec

hi ruaok

Hi alastairp

to confirm - did we set a time tomorrow? I only have a small window of about 2h

hi pristine__

I saw that you wanted to talk with me - I'll be free on Thursday and Friday

alastairp: it is the same discussion i wanted to have with ruaok for like 5 min, but yeah we can have a follow up whenever you are free :)

I'll keep an eye on the conversation and make any comments if I have time

ruaok: https://gist.github.com/vansika/7ac1acfda613e64...

gimme 2 minutes.

all the queries should have same count

alastairp: what is your timeframe?

but they don't have the same count

I realised after lot of querying that it was beacuse of this

the last dataframe

ruaok: I'm free 12-2, morning meeting starts at 10, so likely won't run until 12, but I have another at 2 that I have to be back for

Messybrainz is case insensitive, it was unknown to me

now, the concern is

a sec

it's only 15 minutes for me on the bike between offices, so I think we should still have plenty of time

this

rn , we use this query to get recordings_df

basically distinct recording_msids and corresponding col

in this query we add a recording_id col

which is supposed to be a primary key.

but now when I realise that Messybrainz is not case sensitive

I queried the data

and

with 182986 listens

12:15 at the office then, alastairp ?

we should have 119929 distinct recording_msids

but we have 129124 distinct recording_msids

ruaok: sounds good. thanks

so aorund 600 are duplicate

it is because of the query we use

the ideal way is to only fetch the distinct recording_msids

and then perform a join

which can be expensive

as I see....

Spark needs lot of optimization in joins

but

primary key has to be distinct, according to basic norms of dbms...

so I think we should do something about it

we can optimize the join later

but data accuracy is a must

.

so, lots of things to address here.

:)

first off, messybrainz is intended to be messy. it has tons of duplicates and loads of crap.

which is why it is key to used MBIDs, rather than MSIDs for the collab filtering stuff -- as we discussed over the summer.

we will need the mapping for that. true

which is why the MSID <-> MBID mapping is so important.

and you were given the instructions to proceed with these flimsy assumptions, hoping that we would have a mapping before too long that would allow us to swap out the MSIDs for MBIDs.

so, how does that change this conversation?

instead of worrying about querying MSIDs in the best way possible, we should instead think about how to translate from MSID -> MBID as one of the precursor steps to the CF algorithm.

I know. Just wanted to tell you what all we need to look upon. We discussed the that we need a mapping, yup, so here are the cases that are a part of it. Thought it would be good to tell you the specifics :)

thank you for that -- those specifics give me nightmares. :)

oh. sorry :)

umm...

so ya...the mapping

not your fault -- this just the nature of messybrainz. :)

out of interest, how much data in listenbrainz comes with some other id (spotify or mbid?)

what I can do is work on a mapping this week that would be very... preliminary.

once we have it, I can start on it.

alastairp: I don't have an answer for you yet.

no hurry :)

pristine__: exactly.

so, for now assume that you will get three tables:

msid_mbid_artist_mapping (msid, mbid)

I was just querying data to understand the specifics. MBIDs are too less. yeah

msid_mbid_recording_mapping (msid, mbid)

and then one for release, I think.

yup

sounds good.

so, can you start working on a new translation layer as another pre-process step to the CF alg?

ummm.....yes. I was working on some cleaning stuff so that new people don't have a hard time, but yeah can start with it

and hopefully will start with the tests soon :)

so..

I won't open any PR on correcting the primary key stuff

mapping will solve the problem :)

well, it will make things worse before it makes them better, but yes this is the right path forward.

lol. I understand.

but the mapping can and would give us better results

like for instance we will not be using *artist-name* anymore.

so ya....nice

it *should*. but the mapping will be sparse initially, so fewer results.

and the blame will go on my shoulder. something that reosarevok can pick on me for the next decade. :)

If we get *nice* fewer results, that will like a very good news, will pave path for the rest of it.

lol

Yaaaaaay

I'm upgrading discourse