#musicbrainz-devel

so it would first fetch the ids, then the actual rg objects which are most likely in cache already

this seems good in theory, but I think the way we use our DAO stuff is going to be a problem. mostly due to the ability to construct partial objects

not sure I understand

well, it's possible to create a ReleaseGroup entity without a name, for example

why is that a problem

?

I doubt we do that, but if we end up putting something like that in the cache, something might pull it and expect a name to be there

oh, why would you add it to the cache?

well, it has to get into the cache somewhere, right?

I thought you use the cache that is used by get_by_id

which contains the full objects

oh... so find_by_artist would *only* find release group ids, and then it would do a get_by_ids?

yes

ahhh, that's an interesting look at it

well, for the caching query

in case you don't have the release group ids cached, you would fetch the whole thing from the db

and no bother with get_by_ids

*not

mm

I think it's the difference between hot and cold cache potentially returning different objects that makes me a bit uneasy

would it return different objects? I though get_by_id and find_by_artist returns the same amount of data

they *should*, but nothing guarantees that

so it's possible to introduce a bug there

and bugs where the warm cache and hot cache act differently are a pita to debug

that's always possible

not if you change the entitys to always be complete

I can't think of a situation where they are not complete

except when creating a new one

there are quite a few places where we don't load certain ids or something

i can't think any off the top of my head

do you want to cache *everything*?

just more, for now

that doesn't make much sense to me

I'd only select the most important methods

in BookBrainz, entities must be complete - you can't construct a partial entity. Because of this, they don't actually link directly to other entities. If you want the loaded country of an artist, you return a (Artist, Country) tuple. But due to Perl's general suckiness, I don't think it'll work very well :)

designing a generic caching system would be very hard

luks: right, that's why I wrote all this profiling stuff, to see what is actually important (and it seems to be relationship code)

but practically you don't need that

the front artist page is what gets most hits probably

big releases are the most problematic atm

so ReleaseGroup->find_by_artist is a good target

as in they timeout

yea, find_by_artist is also identified as a slow point by my debug panel, so that's why I mentioned it :)

do releases timeout because of ARs?

or just tracklists

there's no single problem sadly. A big whack of time spent rendering and a big whack of time constructing objects

ah, yeah

hopefully the caching skips the complicated moose object construction and gets us a speed up there

hm, does it?

it won't hit ->new, so 'probably' is my hunch

from what I understand, storable will just inflate it and rebless it

oh

I don't think so, because that could have side effects

but I guess it still hits the Moose attribute validation at some point

i'm not sure it does

Guess that needs more profiling though

btw, any idea where the duplicate artist credits might come from? http://tickets.musicbrainz.org/browse/MBS-4010

hmmm

maybe check for all duplicates and see if anything has a closer created date?

maybe we broke something in the past

another example - http://test.musicbrainz.org/artist/83d91898-776...

844324 | 703804 | 2 | 1 | 2011-07-31 00:02:24.098516+00

836596 | 703804 | 2 | 210 | 2011-07-05 08:57:08.116881+00

that doesn't look like a concurrency problem

sadly no idea

so back to the find_by_artist stuff... how would it handle cache invalidation? ie, a new release group is created - i guess insert is going to be responsible for clearing the find-by-artist cache?

there are two cases, single artist and va, I don't remember the code if they are in the same function

that's find_by_artist and find_by_track_artist, iirc

single artist RGs should be simple, insert/delete automatically invalidates cache

update only if the artist_credit is different

so basically, other daos are going to have to be aware of the other caches

or you might want to always invalidate on update as well

to make it simpler

oh actually, I guess that's all within the release group dao, so that's not so bad

yes, but the cache would belong to the artist

hello

hey warp

ocharles, getting lot of 503s on the mb server , has something been added to reduce traffic, or it just very busy ..

nothing has changed from me there

so its just very busy then, going to get worse over Xmas then I expect

best to prod ruaok about it later

y

sigh, ml post FAIL

hrm, maybe not - it's just failing to display on lists.musicbrainz.org

ocharles: did you add the On Beta workflow option yet?

yep

what status is the ticket in atm?

I'm on a ticket which is "in progress"

ok, it probably can't go from in progress to in beta :)

can it go to in review?

yes

do that then

that's how it's meant to go

not for this particular change :)

I thought you wanted everything to go on beta to go through code review :)

some changes are too small for code review, normally I would have made this change in master.

also, on jira I just want all transitions to be allowed :)

mmm

I just realized that branching from master for bugfixing is not a good idea anymore

it should be

as I cannot merge such branches into beta without bringing in unwanted stuff

but everything in master should be in beta

oh, you're right.

then stuff has been comitted to master which hasn't been comitted to beta.

well it should always be in master and beta

that's why the flow is to trickle the merge into next, then beta, then master

just wondering, do you need the beta branch at all?

luks: sort of

or rather, you do need it to be different from a snapshot of master at some point?

if something needs user testing, it shouldn't be in master, because if we want to do a server release while doing user testing, we'll end up releasing that too

why can't it go to master?

because then when we merge master into production, we've shipped it

I'd say pushing something to beta counts as shipping it

pushing to beta means we're allowing users who want to live with buggy stuff to use it

but we shouldn't enforce that on all site users

I kind of disagree with that

luks: if user testing on beta uncovers flaws in a particular branch/patch, it shouldn't be released.

in my opinion, beta should be "this will be shipped in the next release"

UI testing shouldn't be done on live data

luks: beta is "we intend to ship this in the next release". master is "this will actually be shipped in the next release".

when will something go from beta to master?

when it passes user testing. that's still the minor hole to work out when that decision has been reached

luks: people don't test unless they can do it on live data

when does something pass user testing?

luks: on the day of release, assuming no problems with that particular branch/bug fix have been reported.

whoa

now I very strongly disagree with that

i don't think that's been agreed, warp

I suggested a mailing list that we could post to, but jira should be enough

in my opinion, beta would be the most useful as the next musicbrainz.org

ocharles: you want users to specifically comment "ship it!" on jira?

if you keep something on beta but never ship it, it's not going to be very useful

warp: with my ML idea I would say "MBS-foo is now in beta.musicbrainz.org. Assuming no negative feedback this will ship on now() + x days"

it should be the final product, not a collection of branches

because essentially we currently have that, but x is 0

luks: beta resets after every release, just like next

and then we re-merge stuff that's still in user testing back into it

luks: not everything can be tested on live data, so it will not neccesarily contain all branches which will be part of the next release.

luks: but the intent is for all branches which are on beta to be part of the next release.

luks: but the whole point of beta is to allow particular things to be rejected as not ready for release.

I guess I misunderstood the goal of beta

luks: what did you think the goal of beta was?

making sure you are not going to ship something buggy in the next release