everywhere else we use the ones that go in both directions
2011-05-17 13746, 2011
murdos
nikki, ocharles: ok... fine :) so no db schema change required, that's even better :)
2011-05-17 13715, 2011
ocharles
yea :) it was meant to be part of the migration though... so I'm not entirely sure how to populate the column now
2011-05-17 13753, 2011
ocharles
I may run a bit of a migration to get all link_type's with phrases, then dump them out and make a script to do UPDATE short_link_phrase or something
2011-05-17 13752, 2011
murdos
ocharles: what do you think of http://tickets.musicbrainz.org/browse/MBS-1906 ? If we do something, we should not wait too long, because the more we wait, the more pre-NGS releases are being merged
2011-05-17 13729, 2011
ocharles
yea, I was thinking about that... how would we do it now?
2011-05-17 13706, 2011
ocharles
whip up a script to do it?
2011-05-17 13736, 2011
murdos
?
2011-05-17 13740, 2011
ocharles
yea
2011-05-17 13701, 2011
ocharles
but I'm now thinking if we're gonna do something like that, maybe we should consider the "bubble up" idea in a bit more detail
2011-05-17 13710, 2011
ocharles
the one that you said seems too simple (which by that I think you mean is too general)
2011-05-17 13732, 2011
murdos
yea, I see what you mean... add links for release edits to release_groups, right?
2011-05-17 13752, 2011
murdos
that might work too
2011-05-17 13757, 2011
murdos
would you do it in general, or just for this edit type?
2011-05-17 13715, 2011
ocharles
well, all recordings relate to releases, then all releases relate to release groups, then all release groups relate to artists
2011-05-17 13742, 2011
djce
you've found the source of the 502 errors?
2011-05-17 13757, 2011
ocharles
But I'm wondering if that's just *too much* linking, and just noise in the history
2011-05-17 13711, 2011
murdos
yes, I share this concern
2011-05-17 13721, 2011
djce
if not, check the nginx error log on astro. it's ... chatty.
2011-05-17 13758, 2011
ocharles
ie, do we really want "add puids" showing up in releases, release groups and artists?
2011-05-17 13730, 2011
murdos
oh, no, we don't want that! :)
2011-05-17 13737, 2011
ocharles
but we can probably put extra conditions on things... "bubble these recording edits up to releases" and then "bubble these release edits up to release group" etc
2011-05-17 13759, 2011
ocharles
if I write a skeleton script, would you be interested in having a look at seeing what should go where?
2011-05-17 13713, 2011
murdos
yes, but this can maybe wait a bit, we (or at least me) need to think more about it
2011-05-17 13720, 2011
ocharles
sure
2011-05-17 13729, 2011
ocharles
but as you said, we probably can't afford to wait toooo long
2011-05-17 13738, 2011
ocharles
or can we?
2011-05-17 13752, 2011
murdos
with your "bubble up" idea, you'll need to check each time if the link doesn't already exist?
2011-05-17 13707, 2011
ocharles
murdos: that, or drop constraints on the table and SELECT DISTINCT later
2011-05-17 13714, 2011
ocharles
that's probably going to be the quicker way to do it
2011-05-17 13744, 2011
ocharles
At the moment, the links all have integrity, so if you merge release groups the release still point to the correct release, so I think edits would still appear in the right place
it wasn't happening earlier, how come we had to change stuff?
2011-05-17 13735, 2011
ruaok
ocharles: good question
2011-05-17 13707, 2011
ocharles
warp MBS-1952 is a closed and a dupe btw
2011-05-17 13702, 2011
warp
ocharles: well, the issue is not fixed, it still occurs.
2011-05-17 13707, 2011
djce
ruaok: you've been tweaking the headers thing?
2011-05-17 13714, 2011
djce
which header was it, and how long?
2011-05-17 13714, 2011
ruaok
yeah
2011-05-17 13743, 2011
ruaok
its been hard to tell since the pages may be user dependent.
2011-05-17 13700, 2011
warp
ocharles: if that is really the cause of the crashes in MBS-1913 and MBS-1897 that's good, atleast we know how to reproduce that on ngs.musicbrainz.org.
2011-05-17 13710, 2011
ruaok
so far I've used what seem like abusurdly large numbers to make the errors go away...
2011-05-17 13723, 2011
ruaok
proxy_buffering on;
2011-05-17 13739, 2011
ruaok
those are the tweaks in place now. things have gotten quieter, but not perfect.
2011-05-17 13703, 2011
ruaok
2011/05/17 19:54:50 [warn] 22118#0: *1846 a client request body is buffered to a temporary file /var/lib/nginx/body/0000000002, client: 10.1.1.13, server: ngs.musicbrainz.org, request: "POST /release/b7fe94d6-f2e7-3b73-892f-a043d8138888/edit HTTP/1.0", host: "ngs.musicbrainz.org", referrer: "http://ngs.musicbrainz.org/release/b7fe94d6-f2e7-3b73-892f-a043d8138888/edit"
2011-05-17 13710, 2011
ruaok
I have NO idea what was in the POST
2011-05-17 13711, 2011
warp
ruaok: we hardly use cookies afaik, so it seems odd that we're exceeding that size.
2011-05-17 13741, 2011
ruaok
yeah.
2011-05-17 13745, 2011
djce
Could it be an insanely long Location header in a redirect?
2011-05-17 13704, 2011
ruaok
possibly.
2011-05-17 13708, 2011
djce
ISTR being asked to allow ridiculously long request URIs
2011-05-17 13713, 2011
ruaok
any way for us to dump headers?
2011-05-17 13729, 2011
ruaok
on requests that go bad?
2011-05-17 13744, 2011
djce
some combination of: nginx debug mode; strace; catalyst debug.
2011-05-17 13753, 2011
djce
or tcpdump
2011-05-17 13705, 2011
ruaok
oh boy.
2011-05-17 13710, 2011
ruaok
can I convince you to investigate?
2011-05-17 13716, 2011
nikki
uh oh
2011-05-17 13728, 2011
nikki
I was just editing that release
2011-05-17 13739, 2011
djce
Could do, but it's hardly occurring now.
2011-05-17 13749, 2011
ruaok
we can make it occur. :)
2011-05-17 13754, 2011
djce
indeed :-)
2011-05-17 13702, 2011
djce
I'll see what I can find out first.
2011-05-17 13706, 2011
ruaok
k
2011-05-17 13707, 2011
warp
ruaok: our POSTs could be fairly big in the release editor.
2011-05-17 13720, 2011
ruaok
can you quantify that?
2011-05-17 13757, 2011
warp
ruaok: json encoded tracklists with artist credits. the more discs you edit, the larger the post request for the tracklist and recordings page are.
2011-05-17 13709, 2011
ruaok
nikki: when you edited that, did it fail?
2011-05-17 13725, 2011
nikki
I don't *think* so
2011-05-17 13728, 2011
nikki
but I can't remember :/
2011-05-17 13733, 2011
ruaok
said warn, not fail.
2011-05-17 13737, 2011
ruaok
so I think that is ok.
2011-05-17 13744, 2011
warp
ruaok: the entire tracklist will be submitted as a json-encoded structure. so, normally that should be quite manageable, but if you edit 50 discs in one session.
2011-05-17 13759, 2011
ruaok
ok, I see.
2011-05-17 13715, 2011
ruaok
in this case, the response was too big and it went to disk, but it didn't fail.
2011-05-17 13720, 2011
ruaok
might be acceptable as is.
2011-05-17 13752, 2011
warp
probably. I'd still be interested to know what "too big" is, and how often that will occur for us. but those are things for a later date.
2011-05-17 13708, 2011
ruaok
I might just keep the settings as is. log is quiet and it had no noticable impact on memory footprint.
2011-05-17 13748, 2011
warp
ruaok: I have reproducable internal server errors, would those have stacktraces in logs?
2011-05-17 13724, 2011
ruaok
yes.
2011-05-17 13740, 2011
ruaok
can you make it happen now?
2011-05-17 13753, 2011
ruaok
if so, do.
2011-05-17 13754, 2011
warp
ruaok: yes
2011-05-17 13700, 2011
warp
ok, let me set it up again.
2011-05-17 13735, 2011
warp
ok, ready.
2011-05-17 13742, 2011
warp
here we go.
2011-05-17 13717, 2011
ocharles
warp: the internal error I got gave a stacktrace that I linked the dupe to
2011-05-17 20:06:48.071680500 [error] Caught exception in MusicBrainz::Server::Controller::ReleaseEditor::Edit->edit "Attribute (params) does not pass the type constraint because: Validation failed for 'HashRef' with value undef at /usr/local/share/perl/5.10.1/HTML/FormHandler.pm line 350
2011-05-17 13711, 2011
warp
yeah, same thing.
2011-05-17 13712, 2011
ruaok
1913
2011-05-17 13728, 2011
warp
so, good news is.. we can reproduce MBS-1897 / MBS-1913 on ngs.musicbrainz.org.
2011-05-17 13743, 2011
ruaok waits for the other shoe
2011-05-17 13747, 2011
warp
bad news is, it has nothing to do with the kind of edit someone is making in the release editor.
2011-05-17 13719, 2011
warp
ruaok: if you submit four of those edits in one go (as described on MBS-1952, usually two or three of them fail)
ruaok: ofcourse, usually people do not go around editing X releases at once. so it perhaps tracking it down now shouldn't be a priority, but it seems quite worrying to me.
2011-05-17 13759, 2011
ruaok
warp: agreed.
2011-05-17 13707, 2011
ruaok
something very odd is up
2011-05-17 13758, 2011
nikki finally gets round to going to bed
2011-05-17 13703, 2011
ruaok
nn nikki!
2011-05-17 13713, 2011
warp
ruaok: the exception itself just means the release editor didn't get all the data which should've been submitted to it. in theory it could be made to be more resilient to that and not crash, but that's not really a solution either.
