bitmap those are the joins I constructed by considering only the second to last paths
2017-11-13 31758, 2017
iliekcomputers
jwf: BigQuery standard sql
2017-11-13 31705, 2017
jwf
Ooh, okay.
2017-11-13 31716, 2017
jwf
Have any recommended reads on BigQuery other than the upstream docs?
2017-11-13 31705, 2017
samj1912
A problematic statement in that is if let's say we delete a release status. We won't know which releases to update.
2017-11-13 31711, 2017
bitmap
but we can't delete a release status without the releases being updated first...unless I misunderstand you
2017-11-13 31719, 2017
samj1912
Yes
2017-11-13 31723, 2017
samj1912
That is correct
2017-11-13 31750, 2017
samj1912
Although, I am not sure if I can 100% prove we will cover all the things, in case we add new paths
2017-11-13 31751, 2017
samj1912
Another problem is tables with multiple FKs
2017-11-13 31758, 2017
samj1912
*PKs
2017-11-13 31700, 2017
samj1912
Let's say we update area, see how the statement for annotation update triggered by it looks like
2017-11-13 31720, 2017
samj1912
On updating: `area`, We update entity: `annotation` via path: `areas`, ['type'] SELECT musicbrainz.annotation.id AS musicbrainz_annotation_id FROM musicbrainz.annotation JOIN musicbrainz.area_annotation ON musicbrainz.annotation.id = musicbrainz.area_annotation.annotation WHERE musicbrainz.area_annotation.area IN (:area_1) AND musicbrainz.area_annotation.annotation IN (:annotation_1)
2017-11-13 31723, 2017
bitmap
to generalize the thing above, when a row is deleted...it doesn't makes sense to join with other tables that reference that row
2017-11-13 31736, 2017
d4rkie joined the channel
2017-11-13 31719, 2017
samj1912
Hmm, yes
2017-11-13 31728, 2017
D4RK-PH0ENiX has quit
2017-11-13 31734, 2017
samj1912
But we are joining them simply as a way to get ids
2017-11-13 31755, 2017
samj1912
Which are already emitted from the trigger
2017-11-13 31722, 2017
samj1912
Hmm bitmap, so you are saying if we are deleting something, we don't need to care about updating entities which are not emitted as FKs(so we discard those SQL statements) and update the rest of the paths?
2017-11-13 31748, 2017
samj1912
Since they will be updated before we can delete the given row
2017-11-13 31740, 2017
bitmap
well, the direction of the FKs has to be taken into account...if a release_status is deleted, that should be completely ignored, because it's only referenced *by* other tables, and those will have been updated already
2017-11-13 31714, 2017
bitmap
if a release_alias is deleted, the release has to be updated because it references the release, not the other way around
2017-11-13 31722, 2017
bitmap
if a row has FKs in both directions, only those need to be followed
2017-11-13 31705, 2017
samj1912
Hmm, but the join method from the sql alchemy abstracts all of that, it is figures out which direction it is is from the path
2017-11-13 31733, 2017
samj1912
I'll have to manually inspect each path and figure out the fk direction again
2017-11-13 31704, 2017
exogen has quit
2017-11-13 31708, 2017
bitmap
I assume you only have to look at the direction of the first item in the path if it's a delete
2017-11-13 31750, 2017
samj1912
Direction of the last item and the deleted item you mean?
the worst culprit in terms of reindexing is recording
2017-11-13 31710, 2017
samj1912
we are indexing too much data for it
2017-11-13 31722, 2017
samj1912
solr cant keep up, even when making reindexing queries
2017-11-13 31743, 2017
samj1912
*not solr, psql
2017-11-13 31752, 2017
reosarevok has no idea about https://community.metabrainz.org/t/can-the-listenbrainz-db-be-updated-to-the-latest-musicbrainz-db/340633/12 - iliekcomputers and ruaok, I think we should have a clear pinned post about "Why only some LB listens are linked to MB", any takers? :p
2017-11-13 31735, 2017
MajorLurker has quit
2017-11-13 31700, 2017
Freso
reosarevok: The Last.FM imported only matches to MB if Last.FM itself has their track linked to an MB Recording ID (sometimes they're linked wrongly!). Last.FM importing doesn't give MBIDs for Artists, Releases, etc.
2017-11-13 31707, 2017
pingupingu joined the channel
2017-11-13 31707, 2017
pingupingu has quit
2017-11-13 31707, 2017
pingupingu joined the channel
2017-11-13 31715, 2017
gcilou joined the channel
2017-11-13 31743, 2017
jwf|matrix
iliekcomputers: Okay, no problem! If the docs are good, I'll work my way through them.
2017-11-13 31707, 2017
jsturgis joined the channel
2017-11-13 31731, 2017
pingupingu has quit
2017-11-13 31715, 2017
CatQuest
+1 about linked post
2017-11-13 31752, 2017
CatQuest
Thanks bitmap!
2017-11-13 31745, 2017
jwf
yvanzo: Oops, sorry, didn't realize MBS-9549 was a duplicate. But happy to see it's already fixed. :)
No worry, I am glad that people are actually using beta and report bugs. :)
2017-11-13 31747, 2017
CatQuest
usually i use beta anyway
2017-11-13 31750, 2017
jsturgis has quit
2017-11-13 31704, 2017
SothoTalKer
when are changes to beta usually applied? :)
2017-11-13 31733, 2017
jwf
beta++
2017-11-13 31733, 2017
SothoTalKer
its suprising that something gets fixed in production earlier than in beta (:
2017-11-13 31734, 2017
CatQuest
I've never heard of that happening
2017-11-13 31723, 2017
alastairp
Freso: hi, I won't be around tonight, no news
2017-11-13 31756, 2017
TOPIC: MetaBrainz Community and Development channel | MusicBrainz non-development: #musicbrainz| Google Code-In https://goo.gl/8xSZem | Meeting agenda: reviews
2017-11-13 31710, 2017
TOPIC: MetaBrainz Community and Development channel | MusicBrainz non-development: #musicbrainz| Google Code-In https://goo.gl/8xSZem | Meeting agenda: reviews, Design Tickets(chhavi_)
2017-11-13 31705, 2017
Freso
<BANG>
2017-11-13 31710, 2017
iliekcomputers
Hi!
2017-11-13 31714, 2017
chhavi_
hello
2017-11-13 31714, 2017
Freso
Hello :D
2017-11-13 31723, 2017
Freso
Freso: go!
2017-11-13 31724, 2017
Freso
🙋
2017-11-13 31703, 2017
Freso
I went back to Barcelona last week and had a meeting with Rob and then got my contract renewed, so you're stuck with me a while longer. :)
2017-11-13 31708, 2017
CatQuest
hi!
2017-11-13 31720, 2017
iliekcomputers
:D
2017-11-13 31721, 2017
CatQuest
Freso: yay!
2017-11-13 31727, 2017
Freso
I also helped review reosarevok's Mahler debriefing blog post, but I'm sure he'll say more about that.
2017-11-13 31739, 2017
Leo_Verto
hey!
2017-11-13 31741, 2017
Freso
I dealt a small bit with reported editors.
2017-11-13 31746, 2017
jwf is here
2017-11-13 31749, 2017
Freso
A large bit with Google Code-in.
2017-11-13 31718, 2017
Freso
And otherwise kept up with forums and IRC. (I think maybe there was also a support@ mail this week I answered.)
I was a little quiet, but getting back to speed. Working more with ListenBrainz statistics. I want to work with iliekcomputers and ruaok to add more example metrics.
2017-11-13 31713, 2017
ListMyCDs
I'm following but have nothing to report this time. Just skip me.
2017-11-13 31717, 2017
CatQuest
hey
2017-11-13 31717, 2017
jwf
Once I get the hang of one, I want to write out a document to plan more so we can make sure it will scale.
2017-11-13 31720, 2017
samj1912
chhavi_ you might wanna go early since you might fall asleep any moment :P
