12:22 PM
ruaok
we use this to show who the artists are in a compound string like "Queen & David Bowie"
12:27 PM
MajorLurker has quit
12:28 PM
pristine___
Right, but I still don't understand how an artist mbid maps to multiple credit ids.
12:28 PM
ruaok
it doesn't
12:29 PM
its the other way around.
12:29 PM
one artist credit id maps to multiple artist_mbids
12:30 PM
pristine___
12:30 PM
ruaok
uhm. that doesn't seem right, lol.
12:31 PM
pristine___
right. I was working on a feature
12:31 PM
saw this.
12:33 PM
> one artist credit id maps to multiple artist_mbids
12:34 PM
ruaok: do you mean this?
12:34 PM
12:34 PM
ruaok
yes
12:35 PM
pristine___
12:35 PM
ruaok: I though it was this :p
12:36 PM
ruaok
no wait.
12:36 PM
hang on, I;m trying to debug this query. let me finish that.
12:36 PM
pristine___
cool.
12:38 PM
ping me when you are done with it :)
12:38 PM
pristine___ goes to mop the floor
12:41 PM
ruaok
actually, that is ok.
12:41 PM
that query "finds all of the artists creidts that a given artist is in". Which could be said as "it looks up all of the collaborations that the artist has been involved in"
12:45 PM
12:57 PM
MajorLurker joined the channel
12:59 PM
v6lur_ has quit
13:02 PM
MajorLurker has quit
13:03 PM
iliekcomputers
shivam-kapila: did you get a chance to work on the follower/following component at all?
13:18 PM
ishaanshah
iliekcomputers: ping
13:18 PM
iliekcomputers
pong
13:18 PM
ishaanshah
\o
13:18 PM
iliekcomputers
how goes it?
13:18 PM
ishaanshah
Going good, finally got some time today to work on the tests
13:18 PM
iliekcomputers
nice nice
13:18 PM
ishaanshah
I was reading your comments on the PR
13:19 PM
rn what I am doing is, check if an incremental import with id > x (last full import) is imported
13:19 PM
if not import it
13:19 PM
this way suppose RC crashes today
13:20 PM
we restart it tomorrow, and it imports tomorrows dump
13:20 PM
we wont skip todays
13:20 PM
iliekcomputers
if have the "last full import" filter on it, it could import the same incremental dump multiple times
13:20 PM
if you have*
13:20 PM
ishaanshah
no it wont
13:21 PM
I am checking if its already imported
13:21 PM
in the table
13:21 PM
iliekcomputers
ah
13:21 PM
ok, then why not just take the last imported dump from the table
13:21 PM
instead of doing the last full import thing
13:22 PM
ishaanshah
incase we miss some dump and the next dump is imported, we miss this dump forever
13:22 PM
iliekcomputers
what i'm trying to get at is a guarantee that the data in spark is never in an invalid state
13:22 PM
ishaanshah
just a bit more general case
13:22 PM
iliekcomputers
so essentially if we miss dump x, i'd enforce on request consumer that we do not import dump x+1 without importing x
13:22 PM
ishaanshah
> what i'm trying to get at is a guarantee that the data in spark is never in an invalid state
13:22 PM
yep I am making sure of this
13:23 PM
iliekcomputers
meaning that if request consumer crashes today, then tomorrow's import should import today's dump and tomorrow's dump.
13:23 PM
and if request consumer tries to import just tomorrow's dump somehow, it would error out
13:23 PM
ishaanshah
yes that will happen
13:23 PM
oh
13:23 PM
the error
13:24 PM
ah, I got it
13:24 PM
iliekcomputers
right, sorry for not being clear
13:24 PM
ishaanshah
so incase a dump doesnt have SHA
13:24 PM
and it fails
13:24 PM
we should just stop the import right?
13:24 PM
iliekcomputers
there should be no way for request consumer to go from x to x+2 at all
13:24 PM
yes
13:24 PM
ishaanshah
yep yep got it
13:24 PM
Yeah that is missing, I will add that
13:25 PM
good catch
13:25 PM
iliekcomputers
for full dump imports, we just need to check that the id of the dump we're importing is greater than the id in spark.
13:25 PM
ishaanshah
I was just importing the latest one
13:26 PM
it wont matter right?
13:26 PM
iliekcomputers
mhmm, adding a check would still be good.
13:26 PM
just to make sure we don't delete data :)
13:26 PM
ishaanshah
ok, will do
13:26 PM
iliekcomputers
like suppose if we tried to import a valid full dump with ID x - 3, when the cluster was at x
13:27 PM
ishaanshah
yeah but for the next incremental import
13:27 PM
it will import x+1, x+2, x+3
13:27 PM
so wont be an issue
13:27 PM
iliekcomputers
yeah, i guess that's true. but still keeping the dump IDs as linearly increasing as possible would be great
13:27 PM
ishaanshah
sorry x-2, x-1, x
13:28 PM
iliekcomputers
that way it's easy to reason about stuff
13:28 PM
ishaanshah
yep, that would be better, I will update it
13:29 PM
About the yearly Reports
13:29 PM
I would be happy to work on that after this PR
13:29 PM
My half sem course is ending next week so I'll get more time too
13:30 PM
I will spend some time on this weekend writing a rough draft of what we should do and how we should do it
13:30 PM
iliekcomputers
sounds good! That's exactly what i was gonna suggest :)
13:31 PM
ishaanshah
:D
13:31 PM
shivam-kapila
iliekcomputers: yep I had worked on it. Will get it completed by tomorrow
13:33 PM
iliekcomputers
shivam-kapila: awesome, a PR tomorrow would be great!
13:38 PM
shivam-kapila
iliekcomputers: I dont have that much transitions for the button like we have now
13:39 PM
Simple purple buttons like we have now
13:39 PM
Like I showed in figma***
13:39 PM
iliekcomputers
the FollowButton component is pretty plug and play
13:39 PM
we can just reuse that
13:40 PM
shivam-kapila
Yeah using it
13:40 PM
Just modified stylings
13:42 PM
iliekcomputers
sounds good
13:54 PM
thomasross joined the channel
14:00 PM
abhinavohri joined the channel
14:10 PM
pristine___
14:11 PM
ruaok
that is not valid.
14:11 PM
pristine___
A sec
14:11 PM
> an artist_credit_id can map to a variety of artist_mbids
14:11 PM
ruaok
an artist_credit_id will always map to on unique list of artist_mbids
14:11 PM
pristine___
Right
14:11 PM
> also one artist_credit_id will always map to a single *artist_mbids* (note that i said artist_mbids meaning the list of mibds of length >=1)
14:12 PM
I framed this idea so badly
14:12 PM
:p
14:12 PM
Unique list, that was the word 😂
14:12 PM
ruaok: thanks
14:12 PM
ruaok
np
14:14 PM
abhinavohri
i want to work on LB-682.Can someone suggest me how to do it.
14:14 PM
BrainzBot
14:16 PM
ishaanshah
abhinavohri: Hi, I think I fixed the first point in a PR before
14:16 PM
The other two points are still open though
14:17 PM
14:18 PM
these are the files related to the ticket
14:18 PM
14:18 PM
over here we should create the directory explicitly
14:19 PM
abhinavohri
ishaanshah: ok
14:20 PM
MajorLurker joined the channel
14:20 PM
ishaanshah Also please suggest some other ticket for me related to flask or react.
14:21 PM
ishaanshah
14:21 PM
BrainzBot
LB-643: Improve the listening activity query to make it more scalable
14:21 PM
ishaanshah
if you are interested in something more interesting after this you could take a look at this
14:21 PM
abhinavohri
@ishaanshah ok thank you.
14:22 PM
ishaanshah
14:22 PM
BrainzBot
LB-516: Rewrite the last.fm importer retry logic to be iterative.
14:23 PM
ishaanshah
this can be a good starting point for React based tickets
14:25 PM
MajorLurker has quit
14:26 PM
pristine___
ruaok: how often the mapping dump is updated? Was asking to understand if we should import the mapping into the spark cluster before generating recs.
14:26 PM
Every week
14:28 PM
ruaok
0 4 * * 1,5
14:29 PM
ruaok is sure that pristine___ speaks crontab now
14:29 PM
:)
14:33 PM
alastairp
14:33 PM
chaban
14:34 PM
pristine___
ruaok: thanks
14:35 PM
revi
14:35 PM
pristine___
alastairp: thanks. Will have a look
14:36 PM
alastairp
it looks like recsys is next week, and you'll have to pay to attend the (virtual) conference, but the paper will be available after
14:46 PM
Mr_Monkey
Hi zas! Could you please talk me through how to back files up on prince ? Up until now we were only generating public dumps for BB but now we have a need for private dumps too, and a way to store them somewhere. Needless to say I currently know next to nothing of how the other projects do it…
14:52 PM
alastairp
14:54 PM
pristine___
shivam-kapila: did you open the rec?page= ticket?
14:57 PM
Mr_Monkey
alastairp: Would that back up the entire node though?
15:02 PM
alastairp
15:02 PM
Mr_Monkey
Merci, I'll have a look
15:06 PM
pristine___
ruaok: if artist a and recording b are in MB, but not linked, what is the process to do that?