#musicbrainz

/

      • voiceinsideyou
        pronik: The Edit and Discuss stuff I think could be incorporated in the transcluded bottom line about which revision the page is based on. "To edit this page, or discuss its content... etc"

        • 2009-05-26 14642, 2009

      • navap
        On the VA page, it's there because the tracks contained on the release inside that RG are from that artist.

        • 2009-05-26 14657, 2009

      • navap
        pronik: I messed around a little with the Navbox template, I think we need to enable http://www.mediawiki.org/wiki/Manual:$wgUseTidy

        • 2009-05-26 14617, 2009

      • gnu_andrew
        ruaok, any idea what went wrong? that's what I don't understand

        • 2009-05-26 14624, 2009

      • voiceinsideyou
        navap: isn't the problem that you can set any release group to be of type "non-album track"?

        • 2009-05-26 14651, 2009

      • navap
        Except we also need 1.13 for that, and I believe the reason we're using an old version is because there isn't a package or something for the latest version.

        • 2009-05-26 14618, 2009

      • navap
        voiceinsideyou: That's a separate issue from what I'm talking about, one that needs to be looked at as well of course.

        • 2009-05-26 14632, 2009

      • navap
        Same thing with http://musicbrainz.org/show/artist/?mbid=0e6e1949…

        • 2009-05-26 14649, 2009

      • navap
        I think it should somehow show that Tom Wax is the RG artist.

        • 2009-05-26 14657, 2009

      • voiceinsideyou
        pronik: What I would like; to help me tidy the wiki better without getting confused and discouraged; would be a consistent, smallish and well defined set of templates to use. Currently we have a mess of different hacky templates all over the place that need consolidating.

        • 2009-05-26 14600, 2009

      • navap
        Even though it's listed under Talla meets Tom Wax

        • 2009-05-26 14619, 2009

      • voiceinsideyou
        but i don't have the wiki skills and time to build good templates myself :-/

        • 2009-05-26 14600, 2009

      • navap
        I think the building/designing template problem would go away very fast if we were running MediaWiki 1.13

        • 2009-05-26 14620, 2009

      • jacobbrett has quit

        • 2009-05-26 14628, 2009

      • navap
        It *might* be possible to get Navbox running on 1.11, or we could just hack up our own version of it :/

        • 2009-05-26 14650, 2009

      • jacobbrett joined the channel

        • 2009-05-26 14656, 2009

      • navap
        But I would like to avoid reinventing the wheel, especially when we have such an elegant wheel already designed for us.

        • 2009-05-26 14601, 2009

      • navap
        voiceinsideyou: http://bugs.musicbrainz.org/ticket/5186#comment:1 I'm not understanding your comment. What does it have to do with the fact that 8131 results are shown?

        • 2009-05-26 14649, 2009

      • jacobbrett
        Don't know wher to put this exactly, http://wiki.musicbrainz.org/wiki/opensearch_desc.… - the opensearch link on wiki.musicbrainz.org gives a 404

        • 2009-05-26 14609, 2009

      • voiceinsideyou
        navap; perhaps i misunderstood your comment; i thought you were complaining that it showed results for all artists?

        • 2009-05-26 14613, 2009

      • navap
        jacobbrett: Where is the link?

        • 2009-05-26 14624, 2009

      • navap
        voiceinsideyou: I'm complaining it showed 8131!!! results :p

        • 2009-05-26 14632, 2009

      • voiceinsideyou
        navap: well none of that pages like that across editing are paged?

        • 2009-05-26 14651, 2009

      • navap
        No the results are all shown on one page.

        • 2009-05-26 14611, 2009

      • navap
        And the query takes quite a while to run, something like 2-3 min for that search.

        • 2009-05-26 14612, 2009

      • jacobbrett
        in Firefox, I click the in-built search bar and choose "Add Musicbrainz Wiki (English)"

        • 2009-05-26 14617, 2009

      • luks
        that's actually the same as with standard direct search

        • 2009-05-26 14643, 2009

      • voiceinsideyou
        luks: yeah, that's what I was trying to say

        • 2009-05-26 14644, 2009

      • navap
        Maybe it should use the indexed search then? Wouldn't that be better for the servers?

        • 2009-05-26 14608, 2009

      • voiceinsideyou
        try matching a beethoven 12-track release by discID to artist :p

        • 2009-05-26 14609, 2009

      • luks
        you wouldn't be able to find recent RGs

        • 2009-05-26 14613, 2009

      • navap
        jacobbrett: I'm not undestanding you, where are you clicking?

        • 2009-05-26 14636, 2009

      • jacobbrett
        the Firefox UI search bar (not in-page)

        • 2009-05-26 14643, 2009

      • luks
        I know it's wrong, but it's really not easy to fix with the current setup

        • 2009-05-26 14603, 2009

      • navap
        jacobbrett: AH I think I know what's wrong.

        • 2009-05-26 14649, 2009

      • jacobbrett
        navap: Yeah?

        • 2009-05-26 14617, 2009

      • navap
        The file actually exists at http://wiki.musicbrainz.org/-/opensearch_desc.php not http://wiki.musicbrainz.org/wiki/opensearch_desc.…

        • 2009-05-26 14646, 2009

      • navap
        But there doesn't seem to be a way for FF to pick up that change, it's used to the default file locations of MediaWiki I guess.

        • 2009-05-26 14654, 2009

      • jacobbrett
        yeah, I figured it may have been a re-direct error

        • 2009-05-26 14642, 2009

      • navap
        I'm not sure how that could be fixed, perhaps a symlink on the actual server?

        • 2009-05-26 14623, 2009

      • navap
        Even if the wiki is accessed from http://wiki.musicbrainz.org/-/?title=Main_Page, FF still doesn't use the new path.

        • 2009-05-26 14653, 2009

      • jacobbrett
        No clue.. a line in .htaccess perhaps?

        • 2009-05-26 14623, 2009

      • navap
        Maybe.

        • 2009-05-26 14630, 2009

      • navap
        I have no idea :)

        • 2009-05-26 14621, 2009

      • ruaok
        gnu_andrew: how much do you understand databases?

        • 2009-05-26 14656, 2009

      • gnu_andrew
        ruaok, pretty well, I'm a software engineer

        • 2009-05-26 14603, 2009

      • ruaok
        good.

        • 2009-05-26 14621, 2009

      • ruaok
        the raw user data is stored on a physically separate box from the data on the artists.

        • 2009-05-26 14646, 2009

      • ruaok
        so in order to paginate the results, I had give it an order by clause so the results would be deterministic.

        • 2009-05-26 14606, 2009

      • ruaok
        problem is you cant sort by artist sortname -- thats stored on another server.

        • 2009-05-26 14620, 2009

      • gnu_andrew
        so how was it being achieved?

        • 2009-05-26 14653, 2009

      • ruaok
        before the query wasn't paged and not really ordered at all.

        • 2009-05-26 14659, 2009

      • jacobbrett has quit

        • 2009-05-26 14614, 2009

      • ruaok
        I'm not entirely clear on everything that is going on. I need to spend more time with it.

        • 2009-05-26 14650, 2009

      • jacobbrett joined the channel

        • 2009-05-26 14608, 2009

      • gnu_andrew
        ruaok, it's always run over multiple pages, so I assume you mean 'wasn't paged' in some other sense?

        • 2009-05-26 14649, 2009

      • ruaok
        well, the sql query was never paged (OFFSET, LIMIT) before, which is wasteful. now it is and thus ordering got more important.

        • 2009-05-26 14608, 2009

      • gnu_andrew
        ruaok, oh I see so now each page is a separate SQL query rather than one query, the results of which are then split?

        • 2009-05-26 14622, 2009

      • ruaok
        ding

        • 2009-05-26 14651, 2009

      • ruaok
        my latest style: http://www.flickr.com/photos/mayhem/3565598407/

        • 2009-05-26 14651, 2009

      • gnu_andrew
        ok no I see why it's behaving like it is

        • 2009-05-26 14613, 2009

      • ruaok
        the angle isn't quite right

        • 2009-05-26 14616, 2009

      • ruaok
        gnu_andrew: cool

        • 2009-05-26 14617, 2009

      • navap
        Go Canada :)

        • 2009-05-26 14619, 2009

      • ruaok
        its tricky.

        • 2009-05-26 14623, 2009

      • ruaok
        navap: :)

        • 2009-05-26 14642, 2009

      • voiceinsideyou
        order then page, rather than page then order?

        • 2009-05-26 14616, 2009

      • ruaok
        the only order we have to go on is release id

        • 2009-05-26 14633, 2009

      • ruaok
        which is essentally voodoo from the user's point of view

        • 2009-05-26 14608, 2009

      • voiceinsideyou
        so you don't have even the artist or the release title available at the query? :-o

        • 2009-05-26 14619, 2009

      • ruaok
        nope. just raw data.

        • 2009-05-26 14625, 2009

      • ruaok
        but luks and I are thinking this over.

        • 2009-05-26 14644, 2009

      • ruaok
        will the growth curve of the MB data outpace the growth of reasonably priced memory?

        • 2009-05-26 14646, 2009

      • voiceinsideyou
        ahh right; so before the paging was all handled at the web layer

        • 2009-05-26 14651, 2009

      • ruaok
        I'm not quite comfortable with this thought.

        • 2009-05-26 14623, 2009

      • ruaok
        something like that.

        • 2009-05-26 14609, 2009

      • luks
        ruaok: what about changing the raw data scheme and moving edits to a separate database?

        • 2009-05-26 14625, 2009

      • luks
        that's more or less independant data set and much larger than collections/tags

        • 2009-05-26 14635, 2009

      • ruaok
        move raw to readwrite and edits to raw?

        • 2009-05-26 14641, 2009

      • luks
        yes

        • 2009-05-26 14607, 2009

      • ruaok
        there is certainly less fancy joining happening between the two.

        • 2009-05-26 14618, 2009

      • luks
        you never need to sort or query edits using external data

        • 2009-05-26 14625, 2009

      • ruaok
        indeed.

        • 2009-05-26 14640, 2009

      • ruaok
        and the raw data is small compared to the edits.

        • 2009-05-26 14646, 2009

      • ruaok
        rawdata == mostly ints

        • 2009-05-26 14655, 2009

      • ruaok
        edits == large text fields

        • 2009-05-26 14644, 2009

      • ruaok
        I think I'm ok with that.

        • 2009-05-26 14649, 2009

      • ruaok
        but it makes me uneasy.

        • 2009-05-26 14606, 2009

      • luks
        regarding what?

        • 2009-05-26 14617, 2009

      • ruaok
        I guess I am worried that the rawdata will grow faster than edit data.

        • 2009-05-26 14631, 2009

      • ruaok
        and that could get us up against the wall for memory in our database servers.

        • 2009-05-26 14645, 2009

      • ruaok
        but moose only uses half its ram on any given day.

        • 2009-05-26 14605, 2009

      • pronik
        voiceinsideyou: edit and discuss stuff shouldn't be on transcluded pages anyway and in wiki it's there on the top

        • 2009-05-26 14606, 2009

      • luks
        hm

        • 2009-05-26 14614, 2009

      • ruaok
        which means that we have quite a bit of growth left even on a 12gb ram machine.

        • 2009-05-26 14620, 2009

      • ruaok
        maybe this is a non-issue.

        • 2009-05-26 14624, 2009

      • luks
        we will need a real-time replication or a cluster at that point

        • 2009-05-26 14631, 2009

      • pronik
        navap: tidy is not necessary since they've got some ready-to-use templates for download there. But we need an upgrade of mediawiki anyway

        • 2009-05-26 14634, 2009

      • luks
        partitioning the data doesn't work well, it seems

        • 2009-05-26 14653, 2009

      • ruaok
        luks: 8.5 will get us that and our existing growth should be fine in that timeframe.

        • 2009-05-26 14600, 2009

      • pronik
        voiceinsideyou: consolidating templates: yay. But only step by step

        • 2009-05-26 14600, 2009

      • luks
        and edits really can live on a separate machine

        • 2009-05-26 14607, 2009

      • luks
        even the editing code can be done as a web service

        • 2009-05-26 14609, 2009

      • ruaok
        yah.

        • 2009-05-26 14618, 2009

      • navap
        pronik: I think the templates require tidy to be used. And I somehow think there will be some hesitation on upgrading.

        • 2009-05-26 14620, 2009

      • ruaok
        how much easier would that change make your life going forward with NGS?

        • 2009-05-26 14641, 2009

      • pronik
        navap: somehow I figured that ;)

        • 2009-05-26 14647, 2009

      • ruaok
        or even a totally different subdomain.

        • 2009-05-26 14658, 2009

      • MBChatLogger
        ruaok probably meant ' edit.musicbrainz.org '

        • 2009-05-26 14658, 2009

      • ruaok
        edit.mb.org

        • 2009-05-26 14636, 2009

      • luks
        it will certainly make it easier

        • 2009-05-26 14638, 2009

      • pronik
        navap: http://en.wikipedia.org/wiki/Wikipedia:WikiProjec… says:

        • 2009-05-26 14640, 2009

      • ruaok
        luks: this decision has direct impact on pending patches, no?

        • 2009-05-26 14647, 2009

      • pronik
        "Many Wikipedia templates cannot be directly exported to other wikis because they include HTML code within a parser function that does not translate across wikis. Wikipedia has a "tidy" pass that cleans up HTML code. Without that pass, HTML code within templates get translated into quoted tags that get displayed.

        • 2009-05-26 14648, 2009

      • luks
        ruaok: no

        • 2009-05-26 14650, 2009

      • pronik
        To fix this problem the following template have been converted so that templates contain HTML code within a parser function rather than using wiki tags for table conversions."

        • 2009-05-26 14657, 2009

      • luks
        ruaok: the patch doesn't deal with rawdata yet

        • 2009-05-26 14616, 2009

      • luks
        it will not make the upgrade script easier

        • 2009-05-26 14627, 2009

      • ruaok
        and we could bring it into the fold faster that way?

        • 2009-05-26 14627, 2009

      • luks
        because I need to mess with multiple databases anyway

        • 2009-05-26 14632, 2009

      • ruaok
        harder, if anything.

        • 2009-05-26 14636, 2009

      • luks
        but it will definitely make development later easier

        • 2009-05-26 14651, 2009

      • ruaok
        since there are not FKs you need to deal with possible FK violations.

        • 2009-05-26 14652, 2009

      • navap
        pronik: Thanks for clarifying. Take that $randomonlineforum, shows what you know.

        • 2009-05-26 14600, 2009

      • ruaok
        luks: lets go do that.

        • 2009-05-26 14605, 2009

      • luks
        ruaok: I already have to do that :)

        • 2009-05-26 14620, 2009

      • luks
        there are tables with FK constraints in the main database

        • 2009-05-26 14623, 2009

      • luks
        er, without

        • 2009-05-26 14636, 2009

      • ruaok
        yeah. ARs.

        • 2009-05-26 14650, 2009

      • luks
        no, AR attribures only

        • 2009-05-26 14656, 2009

      • luks
        but also annotations

        • 2009-05-26 14607, 2009

      • ruaok
        I wonder how much DB load editing is vs serving read only traffic.

        • 2009-05-26 14608, 2009

      • luks
        I think there is a few more

        • 2009-05-26 14618, 2009

      • luks
        hm, good question

        • 2009-05-26 14639, 2009

      • luks
        editing probably produces most DB load hourly

        • 2009-05-26 14642, 2009

      • luks
        when modbot runs

        • 2009-05-26 14642, 2009

      • ruaok
        because the machine that does rawdata DB right now also doubles as the index creator.

        • 2009-05-26 14608, 2009

      • ruaok
        and compared to moose... its just not in the same class.

        • 2009-05-26 14628, 2009

      • ruaok
        I may need to be prepared to buy a new machine in that case.

        • 2009-05-26 14628, 2009

      • ruaok
        crap. I'm so screwed. bed time today is 2:30am. wednesday I am I have to leave my house at 5am.

        • 2009-05-26 14638, 2009

      • ruaok
        oh well.

        • 2009-05-26 14652, 2009

      • ruaok
        luks: yes go ahead with that new split. we'll make it work.,

        • 2009-05-26 14627, 2009

      • luks
        ok, I'll try to write a script to do that this week

        • 2009-05-26 14606, 2009

      • ruaok
        cool'

        • 2009-05-26 14653, 2009

      • navap
        uk.musicbrainz.org has been replaced with de. and nl. ?

        • 2009-05-26 14605, 2009

      • navap is looking at Picard's mirrors.

        • 2009-05-26 14617, 2009

      • navap
        Or rather, the mirrors that come with Picard.

        • 2009-05-26 14637, 2009

      • navap
        None of the three seem to work though, de at least returns a ping.

        • 2009-05-26 14639, 2009

      • pronik goes off for lunch

        • 2009-05-26 14629, 2009

      • ruaok
        lunch? bed!