#metabrainz

samj1912: that approach makes sense to me.

basically we have normal boosts, based on field values

boosting has gone towards query time boosts, so this is the way to go, right?

and then we have these functions - https://wiki.apache.org/solr/FunctionQuery

ruaok: yup, solr 7 is query boost exclusive

index time boosts are gone

ok, then its the clear path ahead.

ruaok: Annotation and areas are done, I came up on artists, this is how it is boosted currently - https://github.com/metabrainz/search-server/blo...

it basically does it for a select few artists

artists is an utter freaking mess when it comes to boosting.

how do you want to do it then?

I was thinking about that db method we talked about

that wasn't a suggestion to do it differently -- just commiserating.

remind me of the DB method.

we were thinking of dumping all the popular artists from LB and then assigning a score and storing them somewhere

sir will then pull up these scores and store them in solr docs

then while querying we will simply boost docs with more score

ah yes.

the main problem is updating those scores

so, that solves only a portion of the problem though.

at least from what I understand.

yup

what if we did a test -- it might work, might not. but it should give us better clues about what might work....

how do you want to test it?

so, rather than waiting me to escape banking hell to setup the spark cluster, we fake the data.

if you count the number of tracks by an artist or number of releases and use that as a "popularity" score, then it should give us an idea of how well this might work.

hmmm

not perfect, but might be a useful test and the data is easy to come about.

count of listens right?

then shove that data in and score accordingly.

or simply the mbdb count?

no, leave LB data be for now.

MBDB count.

its a super quick test that will not be perfect.

okay

so basically more number of releases + recordings = more popular

actually, it might be better than the popularity one -- not all artists would get a score using the LB method.

but this way every MB artists gets one.

yes.

could be deeply flawed, might work great. I just don't know.

1 main prob - this is basically artist ref count right?

but it is an easy thing to try and to get the next step.

but I don't think this is as comprehensive.

do you remember we removed the ref count from solr because of the shit ton of updates

but if you got with releases as a measure, then it should be a simple aggregate query.

yes.

and if we don't use refcount we control when this gets updated.

hmm

that is a very good insight, but I think we can disregard it for the test I am proposing.

and remember, its just a test.

and if we use it, then we're incrementing the count for every new album release. totally fine as far as updates are concerned.

well here's the thing, the problem is cascading updates - let's say we update bach, 1 more release added, this will trigger an update for all the releases and recordings related to him since his doc got updated

this was the reason we removed ref count

1 release -> artist -> all related releases

which just causes a shitstorm when its one of the popular artists or VA

we can stop this however if we end triggers at artists

when a ref count is updated

don't we have more granular control over what cascades an update?

we do, I am just wondering how we will score releases/recordings then

ah, different context.

yup

so, you agree that for artists, this is viable?

yup

ok, lets go for releases then.

ill feed the data

okay

how much of a problem is this on a release level?

and does the release analyzer actively change the boosting right now

?

actively as in?

are boosts done on the release level?

I dont see any release boost file in the current search server

current = lucene one

and on recordings?

or, better question, what other searches do we boost that we need to address?

let me check

IIRC artist is the most important one and we just agreed that we can use this approach.

cool

ill integrate the ref count then, and stop the updates at artist

and boost acc. :p

great. I'm curious to see how effective that will be.

I know you're just testing right now, but how about artists that are not performers, such as engineers or mixers? Will this pose a problem long term?

hmm, ruaok so we dont have fine tuned control, I am wondering how to add it? just as a special case for ref_count?

all the entities that are updated when an artist is currently updated

update for an artist = if any of its indexed field changes

for now make it a stored field, or if it needs to be an indexed field, use that.

don't make it special like a refcount -- forget about the concept of refcount for the time being

then?

store a new field. num-releases. then use that to do query time result ordering.

I guess I don't understand what you're asking.

okay, how's this - lets add ref count, but not add a trigger for artist when refcount is encountered?

so basically the only time refcount will be updated is when any of the other things trigger an artist update

add no triggers,nothing.

just a dumb old field.

just don't even think about updating that field right now.

cool

LordSputnik: Hi, I will be not be present for next couple of hours. 😅

I'll send the relevant material later today.

bukwurm: I am available if you wanna talk about solr

ping me up when you are available

[sir] samj1912 opened pull request #76: Add refcount to artist index (master...refcount) https://git.io/vp5Gd

it is fancy :P

bukwurm: ok

Please also send an update of what you've done today :)

Guy, where do I get your public GPG key from, to verify replication packets?

... I've been going through the docs but can't see it anywhere. :(

its oretty awesome

toobad i cant have it.

(or it wouldn't be true if i did)

it's acrtually ridicolous.

chhavi_: tell your sister the design is awesome

Why can't you have it? Redbubble doesn't deliver?

metabrainz/picard#3412 (master - 848745d : Sambhav Kothari): The build passed.

Change view : https://github.com/metabrainz/picard/compare/1a...

Build details : https://travis-ci.org/metabrainz/picard/builds/...

metabrainz/picard#3412 (master - 848745d : Sambhav Kothari): The build passed.

Change view : https://github.com/metabrainz/picard/compare/1a...

Build details : https://travis-ci.org/metabrainz/picard/builds/...

UmkaDK: gpg --recv-keys C777580F

(from https://musicbrainz.org/doc/MusicBrainz_Databas... )