21:10 PM
reosarevok
(still more than most editors, though :( )
2011-05-15 13501, 2011
21:16 PM
ruaok
ok, arrived in paso robles.
2011-05-15 13508, 2011
21:16 PM
ruaok
time for me to drive teh rest of the way.
2011-05-15 13524, 2011
21:16 PM
ruaok
I 'll be a few minuts late, gotta get fudz/
2011-05-15 13532, 2011
21:16 PM
ruaok
bbiab!
2011-05-15 13509, 2011
21:23 PM
navap
luks: What prompted the change to 2.3/utf-16?
2011-05-15 13558, 2011
21:23 PM
nikki
all the people in windows freaking out about their tags "breaking"
2011-05-15 13526, 2011
21:24 PM
navap
Silly people
2011-05-15 13554, 2011
21:24 PM
reosarevok
Well, they are using Windows after all
2011-05-15 13514, 2011
21:51 PM
djce joined the channel
2011-05-15 13509, 2011
21:54 PM
ijabz joined the channel
2011-05-15 13537, 2011
22:09 PM
ruaok joined the channel
2011-05-15 13501, 2011
22:10 PM
ruaok
3M edits migrated. looking good.
2011-05-15 13542, 2011
22:10 PM
ruaok
ah, and its only 23:10 BST, so there is more time.
2011-05-15 13552, 2011
22:15 PM
ruaok
let me read and ponder for a minuite
2011-05-15 13509, 2011
22:16 PM
ocharles
sure
2011-05-15 13516, 2011
22:17 PM
ruaok
does the edit system go back and modify the edit link tables when say an artist gets removed?
2011-05-15 13522, 2011
22:17 PM
ocharles
yea
2011-05-15 13528, 2011
22:17 PM
ocharles
well
2011-05-15 13531, 2011
22:17 PM
ocharles
it removes the link to the deleted artist
2011-05-15 13501, 2011
22:18 PM
ruaok
do we ever do anything else?
2011-05-15 13508, 2011
22:18 PM
ruaok
to the old links?
2011-05-15 13516, 2011
22:18 PM
ocharles
well, we move them if stuff gets merged
2011-05-15 13518, 2011
22:18 PM
ruaok
just remove ones to deleted entities?
2011-05-15 13543, 2011
22:18 PM
murdos
ruaok: no, editing history are merged when you're merging 2 entities
2011-05-15 13543, 2011
22:18 PM
ocharles
as murdos said, if you merge artist 1 into artist 2, then all edits to artist 1 are moved to be edits to artist 2
2011-05-15 13557, 2011
22:18 PM
ruaok
so,we would need to "replay" all the edits to the old database to fix things up, right?
2011-05-15 13515, 2011
22:19 PM
ocharles
we'd need to have some sort of log of what's been merged
2011-05-15 13527, 2011
22:19 PM
ocharles
we almost have that with the gid redirects
2011-05-15 13531, 2011
22:19 PM
ruaok
doesn't our edit history tell us this?
2011-05-15 13500, 2011
22:20 PM
ocharles
yes, it would probably be possible to build a mapping from edit history
2011-05-15 13522, 2011
22:20 PM
ruaok
here is my goal: if the three of us can figure out what it takes to *make* it possible to fix the migration data, then we'll do it.
2011-05-15 13539, 2011
22:20 PM
ruaok
otherwise we'll need to save more data/log stuff/whatever so that we *can* make it possible.
2011-05-15 13552, 2011
22:20 PM
ruaok
murdos: I'm not going back on my word, trust me.
2011-05-15 13504, 2011
22:21 PM
ruaok
I think we're all a little bit exhaused and overheated.
2011-05-15 13518, 2011
22:21 PM
ocharles
amen to that. but tomorrow I don't really have anything pressing
2011-05-15 13523, 2011
22:21 PM
ruaok
so, lets put our heads together to figure out HOW it can be done.
2011-05-15 13524, 2011
22:21 PM
ocharles
I have some JS stuff to do, but this is more improtant
2011-05-15 13535, 2011
22:21 PM
ruaok
ocharles: you will have pressing things to do. :)
2011-05-15 13541, 2011
22:21 PM
ocharles
ruaok: when you get up I will
2011-05-15 13548, 2011
22:21 PM
ruaok
true.
2011-05-15 13552, 2011
22:21 PM
ocharles
but there's a good few hours for me to work on things in the mean time
2011-05-15 13503, 2011
22:22 PM
ruaok
ok, sounds good.
2011-05-15 13509, 2011
22:22 PM
ruaok
no changes to the migration script though.
2011-05-15 13512, 2011
22:22 PM
ruaok
thats frozen now.
2011-05-15 13517, 2011
22:22 PM
ruaok
murdos: what do you think?
2011-05-15 13519, 2011
22:22 PM
murdos
sounds good to me too
2011-05-15 13526, 2011
22:22 PM
ocharles
then we can only fix it after migration?
2011-05-15 13530, 2011
22:22 PM
ruaok
is it possible to replay the data and replace the data in line later?
2011-05-15 13537, 2011
22:22 PM
ocharles
hmmm
2011-05-15 13538, 2011
22:22 PM
ocharles
idea
2011-05-15 13540, 2011
22:22 PM
ruaok
yes, only after migration.
2011-05-15 13545, 2011
22:22 PM
ruaok
migration is frozen now.
2011-05-15 13553, 2011
22:22 PM
ruaok listens to ocharles
2011-05-15 13559, 2011
22:22 PM
ocharles
we do the migration as we do at the moment. this will mean that edit_artist will have links to old artist ids, that are now split into collaborations
2011-05-15 13516, 2011
22:23 PM
ocharles
surely it would be quite easy to just find all these broken ids, and expand them into artist credit artist ids?
2011-05-15 13529, 2011
22:23 PM
ocharles
the edit itself will sort of look wrong, but it will have the right links
2011-05-15 13530, 2011
22:23 PM
ocharles
is this good enough?
2011-05-15 13535, 2011
22:23 PM
ruaok
you thinking about a post-migration script?
2011-05-15 13538, 2011
22:23 PM
ocharles
yea
2011-05-15 13549, 2011
22:23 PM
ruaok
murdos: ?
2011-05-15 13510, 2011
22:24 PM
murdos
ocharles: collaborations is just one of the issue
2011-05-15 13526, 2011
22:24 PM
murdos
other ones got moved to post-NGS
2011-05-15 13532, 2011
22:24 PM
ocharles
right, so you want you really want is more comprehensive linking of everything
2011-05-15 13539, 2011
22:24 PM
murdos
right
2011-05-15 13512, 2011
22:25 PM
ocharles
I guess it should be possible to do that in a few queries as well
2011-05-15 13517, 2011
22:25 PM
murdos
as I previously said, traceability is really important
2011-05-15 13533, 2011
22:25 PM
ocharles
sure, I definitely get why it's important
2011-05-15 13550, 2011
22:25 PM
ocharles
so what I'm thinking is:
2011-05-15 13554, 2011
22:25 PM
ocharles
do the migration as we do at the moment
2011-05-15 13511, 2011
22:26 PM
ocharles
then, select all recording ids from edit_recording, and make sure the edit has a corresponding row in edit_release
2011-05-15 13528, 2011
22:26 PM
ocharles
then, select all edit_release rows, and make sure they have a corresponding edit_release_group
2011-05-15 13536, 2011
22:26 PM
ocharles
then bubble that up to artist as well
2011-05-15 13548, 2011
22:26 PM
ocharles
it only leaves label, work and url a bit less linked
2011-05-15 13524, 2011
22:27 PM
ocharles
but i don't know if that's now too general
2011-05-15 13531, 2011
22:27 PM
murdos
I have to check, but since is not only about edit links
2011-05-15 13554, 2011
22:27 PM
murdos
some edit types should be migrated in a better way
2011-05-15 13508, 2011
22:28 PM
murdos
s/since/this/
2011-05-15 13530, 2011
22:28 PM
ruaok
lets finishing thinking about edit links.
2011-05-15 13538, 2011
22:28 PM
ruaok
is ocharles' approach workable, do you think?
2011-05-15 13500, 2011
22:29 PM
murdos
I've to think more about it, but it looks like too simple
2011-05-15 13517, 2011
22:29 PM
ruaok
hmm. ok.
2011-05-15 13539, 2011
22:29 PM
murdos
I'd prefer the "replay" scenario
2011-05-15 13544, 2011
22:29 PM
ruaok
the improvement of problematic edits should be easier since we dont go back and change the edits, right?
2011-05-15 13509, 2011
22:30 PM
ocharles
ruaok: as murdos said, the edits do depending on other underlying data though
2011-05-15 13529, 2011
22:30 PM
ocharles
so just rerunning a migration could be incorrect
2011-05-15 13532, 2011
22:30 PM
murdos
ruaok: indeed. the embedded json might change
2011-05-15 13542, 2011
22:30 PM
ruaok
ok
2011-05-15 13505, 2011
22:31 PM
ruaok
ok, then lets think about how a replay script would work.
2011-05-15 13527, 2011
22:31 PM
ruaok
we would only replay new edits since NGS switchover, right?
2011-05-15 13530, 2011
22:31 PM
ocharles
before we do that, I think we need to be very clear about the problems we are trying to address
2011-05-15 13546, 2011
22:31 PM
ocharles
i'm still hazy on exactly which edit types need their migration changing
2011-05-15 13546, 2011
22:31 PM
ruaok
should we create a page for that?
2011-05-15 13554, 2011
22:31 PM
ocharles
the linking I can understand the problems with
2011-05-15 13510, 2011
22:32 PM
ocharles
but as far as I understand, the data in the edits is correct, as it just matches the edits in pre-NGS
2011-05-15 13532, 2011
22:32 PM
murdos
let me check my open tickets
2011-05-15 13558, 2011
22:34 PM
murdos
2011-05-15 13512, 2011
22:35 PM
ocharles
1844 I can see being fixable after migration
2011-05-15 13546, 2011
22:35 PM
ocharles
hrm
2011-05-15 13500, 2011
22:36 PM
ocharles
no, might not work
2011-05-15 13537, 2011
22:36 PM
ocharles
though with a tiny change, it could be easier to fix that after migration
2011-05-15 13558, 2011
22:36 PM
ocharles
(If the old edit has nothing for AlbumId, then we can stick -1 as the release_id as a marker for now)
2011-05-15 13532, 2011
22:38 PM
murdos
I trust you on this one, I can't really say
2011-05-15 13502, 2011
22:39 PM
murdos
2011-05-15 13526, 2011
22:39 PM
murdos
what do you think?
2011-05-15 13535, 2011
22:39 PM
ocharles
that's what I was saying I can spend tomorrow working on
2011-05-15 13545, 2011
22:39 PM
ocharles
but rob doesn't want me to change the migration script
2011-05-15 13559, 2011
22:39 PM
ocharles
doing that after migration is a bit more difficult
2011-05-15 13538, 2011
22:40 PM
ocharles
We'd have to keep the collaboration table around, along with an artist merge table (for all merges that happened after NGS launched)
2011-05-15 13549, 2011
22:40 PM
ocharles
and then go back and try and piece things together. not sure it'd work so well
2011-05-15 13505, 2011
22:41 PM
ruaok
if we change the migration script, we can't test it before running it live.
2011-05-15 13515, 2011
22:41 PM
ocharles
I can test small amounts of edits
2011-05-15 13526, 2011
22:41 PM
ruaok
and if we decide we must change the script, then we need to postpone the launch
2011-05-15 13528, 2011
22:41 PM
ocharles
but probably not the whole edit database
2011-05-15 13525, 2011
22:42 PM
ocharles
The changes here would affect only edit types that have artist credits in them. that may be a small enough set to be an amount of edits I can do a full test on
2011-05-15 13552, 2011
22:42 PM
ruaok
how long will this change take?
2011-05-15 13501, 2011
22:43 PM
ruaok
murdos: what other edit types need fixing
2011-05-15 13502, 2011
22:43 PM
ruaok
?
2011-05-15 13519, 2011
22:43 PM
ocharles
I need to build some sort of artist id -> artist credit mapping in memory, then wire that into the necessary edit types.
2011-05-15 13548, 2011
22:43 PM
ocharles
I don't think the historic edit types expect to have artist credits in them, so those need to have the upgrade bit changed, along with how they are displayed
2011-05-15 13527, 2011
22:44 PM
murdos
after more thinking, I think behind MBS-1846 there are 2 issues: 1. displaying old/new values as artist credits 2. move editing history of a collab artist to each involved artists
2011-05-15 13540, 2011
22:44 PM
ocharles
2 can be done easily
2011-05-15 13557, 2011
22:44 PM
ocharles
I would personally want 2 to be done, and with 1 we'll have to put up with displaying just the name and having it appear as though the artist is deleted
2011-05-15 13505, 2011
22:45 PM
reosarevok
2 sounds quite useful
2011-05-15 13512, 2011
22:45 PM
reosarevok
1 too but not that much
2011-05-15 13519, 2011
22:46 PM
ocharles
2011-05-15 13504, 2011
22:47 PM
murdos
ocharles: but you would do that post migration, right?
2011-05-15 13520, 2011
22:47 PM
ocharles
right, it'd be a new script
2011-05-15 13523, 2011
22:47 PM
murdos
so won't be able to handle artists merged post NGS?
2011-05-15 13539, 2011
22:47 PM
ocharles
I think I could probably write that script now
2011-05-15 13554, 2011
22:47 PM
ocharles
Then we can run it before those edits make it in
2011-05-15 13503, 2011
22:48 PM
murdos
ok, so right after migration?
2011-05-15 13506, 2011
22:48 PM
ocharles
right
2011-05-15 13510, 2011
22:48 PM
ocharles
ideally I'd want to run it right after upgrade edits
2011-05-15 13511, 2011
22:48 PM
murdos
yep, that sound good
2011-05-15 13543, 2011
22:49 PM
ruaok
murdos: are those all the issues?
2011-05-15 13534, 2011
22:50 PM
murdos
2011-05-15 13527, 2011
22:52 PM
ruaok
ocharles: can you please make a wiki page (or something) that keeps track of what we've decided?
2011-05-15 13539, 2011
22:52 PM
ocharles
have we decided anything yet? :)
2011-05-15 13543, 2011
22:52 PM
ruaok
lets record what has been decided before moving on to the last two issues.
2011-05-15 13510, 2011
22:53 PM
ocharles
from what I understand: expanding artist credits to just links is OK, and the standalone recording problem can be fixed with hacking a -1 into the script
2011-05-15 13516, 2011
22:53 PM
ruaok
just jot down possible approaches to fixing theses problems as well as listing the problems.
2011-05-15 13520, 2011
22:53 PM
ocharles
does that sound right to everyone else?
2011-05-15 13511, 2011
22:54 PM
murdos
ocharles: it seems yes. so 1 of MBS-1846 is won't fix, right?
2011-05-15 13520, 2011
22:54 PM
ocharles
i'm afraid so
2011-05-15 13533, 2011
22:54 PM
murdos
I think we can live with that
2011-05-15 13552, 2011
22:54 PM
ruaok
ok, so far we have:
2011-05-15 13553, 2011
22:54 PM
ruaok
> expanding artist credits to just links is OK, and the standalone recording problem can be fixed with hacking a -1 into the script
2011-05-15 13500, 2011
22:55 PM
ruaok
lets move on to 1855
2011-05-15 13526, 2011
22:55 PM
murdos
(for the record, -1 is just a hack to ease the final fix post NGS)
