#musicbrainz

/

      • MBLogger
        MBLogger (~musicbra-@client3.fre.communitycolo.net) has joined #musicbrainz

        • 2003-02-15 04606, 2003

      • djce tests the logger

        • 2003-02-15 04600, 2003

      • djce
        djce ([R5qESSxZR@195.60.9.122) has joined #musicbrainz

        • 2003-02-15 04615, 2003

      • djce
        djce ([0LywwsAd2@195.60.9.122) has joined #musicbrainz

        • 2003-02-15 04646, 2003

      • JavaGeek
        make up your mind... :)

        • 2003-02-15 04606, 2003

      • djce
        I'm testing!!! :-)

        • 2003-02-15 04616, 2003

      • JavaGeek
        yeah, I figured that much...

        • 2003-02-15 04656, 2003

      • JavaGeek
        what are you testing?

        • 2003-02-15 04610, 2003

      • djce
        A bot which logs & archives this channel.

        • 2003-02-15 04620, 2003

      • djce
        So we can post them on the web, like the mailing lists.

        • 2003-02-15 04627, 2003

      • djce has left the channel

        • 2003-02-15 04633, 2003

      • djce
        djce ([0LywwsAd2@195.60.9.122) has joined #musicbrainz

        • 2003-02-15 04644, 2003

      • djce
        djce ([NIvltWZoA@195.60.9.122) has joined #musicbrainz

        • 2003-02-15 04644, 2003

      • djce is away: I'm busy

        • 2003-02-15 04632, 2003

      • ruaok
        ruaok (~robert@adsl-63-194-101-14.dsl.snlo01.pacbell.net) has joined #musicbrainz

        • 2003-02-15 04658, 2003

      • ruaok likes MBLogger watching over us. :-)

        • 2003-02-15 04657, 2003

      • djce
        Yes, a rather handy out-of-the-box bit of kit from the w3c. Nice!

        • 2003-02-15 04632, 2003

      • salisan
        salisan (~salisan@ai40.nuaccess.net) has joined #musicbrainz

        • 2003-02-15 04651, 2003

      • salisan
        Hello

        • 2003-02-15 04659, 2003

      • djce
        Hi

        • 2003-02-15 04610, 2003

      • MBLogger
        MBLogger (~musicbra-@client3.fre.communitycolo.net) has joined #musicbrainz

        • 2003-02-15 04604, 2003

      • salisan
        How much bandwidth is the server limited to?

        • 2003-02-15 04639, 2003

      • djce
        That's a question for Rob. ruaok, you there?

        • 2003-02-15 04657, 2003

      • salisan
        Hmm, well. I suppose the best way to handle more traffic is to get some more webservers going. Finding bandwidth sponsors should be easier than finding money sponsors..

        • 2003-02-15 04634, 2003

      • djce
        Adding mirroring capabilities is in the works.

        • 2003-02-15 04600, 2003

      • djce
        Plus, we're always interested in finding willing FTP/WWW mirror servers.

        • 2003-02-15 04632, 2003

      • djce is back (gone 01:25:48)

        • 2003-02-15 04617, 2003

      • ruaok
        sorry, I was responding to email.

        • 2003-02-15 04629, 2003

      • ruaok
        I'm trying to work down the email pile that has accumulated.

        • 2003-02-15 04658, 2003

      • ruaok
        bandwidth limiting: we're on 10Mbps unswitched hub with a bunch of other servers.

        • 2003-02-15 04618, 2003

      • ruaok
        Its very low tech bandwidth limiting. :-)

        • 2003-02-15 04631, 2003

      • intrep
        intrep (~intrep@dsl092-134-144.chi1.dsl.speakeasy.net) has joined #musicbrainz

        • 2003-02-15 04634, 2003

      • djce
        Is that the limiting factor?

        • 2003-02-15 04639, 2003

      • ruaok
        yes. :-)

        • 2003-02-15 04645, 2003

      • djce
        Does CCCP want a donation for a 100Mb switch?

        • 2003-02-15 04600, 2003

      • djce
        I guess I'd have to ask them that :-)

        • 2003-02-15 04603, 2003

      • ruaok
        The problem is that CCCP has to pay for bandwidth spikes, and those can be VERY expensive.

        • 2003-02-15 04617, 2003

      • ruaok
        Yes and no.

        • 2003-02-15 04621, 2003

      • djce
        ?

        • 2003-02-15 04640, 2003

      • ruaok
        They always need more hardware to keep things running, but the 10Mbps hub is there for a reason: bandwidth limiting.

        • 2003-02-15 04642, 2003

      • intrep
        any plans to do incremental updates of the database db files like freedb does? :)

        • 2003-02-15 04607, 2003

      • djce
        That would probably be an integral part of the "mirroring" project.

        • 2003-02-15 04621, 2003

      • ruaok
        Currently the dataset is small enough (and will never grow to freedb sizes since we avoid duplication like the plague) to download in one batch.

        • 2003-02-15 04621, 2003

      • djce
        So plans, certainly.

        • 2003-02-15 04626, 2003

      • salisan
        It would be preferable if they had 10mbps switched ;)

        • 2003-02-15 04650, 2003

      • ruaok
        Also unlike Freedb our changes are not as atomic....

        • 2003-02-15 04613, 2003

      • ruaok
        I understand it would be nicer to have a 100Mbps switch.

        • 2003-02-15 04623, 2003

      • ruaok
        But we cant affort to pay for the bandwidth that would get sucked up.

        • 2003-02-15 04654, 2003

      • djce
        Is the CCCP co-lo free, unless you pass some threshold?

        • 2003-02-15 04610, 2003

      • intrep
        any particular reasons you guys use postgresql native dumps instead of sql command dumps?

        • 2003-02-15 04614, 2003

      • ruaok
        yes and no.

        • 2003-02-15 04627, 2003

      • ruaok
        CCCP does not charge anything.

        • 2003-02-15 04628, 2003

      • djce
        intrep: I'm working on alternative dump formats right now.

        • 2003-02-15 04631, 2003

      • intrep
        it seems it might be easier to diff sql dumps

        • 2003-02-15 04640, 2003

      • intrep
        djce: awesome

        • 2003-02-15 04640, 2003

      • salisan
        Ok, nothing is real cheap.

        • 2003-02-15 04648, 2003

      • ruaok
        However it asks for $50/month/server

        • 2003-02-15 04657, 2003

      • ruaok
        $25/month/virtual server.

        • 2003-02-15 04601, 2003

      • ruaok
        NOTHING, really.

        • 2003-02-15 04609, 2003

      • djce
        Currently I'm thinking: tab-separate, Excel CSV format, and MySQL SQL script.

        • 2003-02-15 04618, 2003

      • ruaok
        Relatable covers our $50/month, so we're off the hook.

        • 2003-02-15 04623, 2003

      • djce
        Any other formats you'd like to see? :-)

        • 2003-02-15 04629, 2003

      • djce
        ruaok: I see

        • 2003-02-15 04631, 2003

      • intrep
        sql command scripts

        • 2003-02-15 04646, 2003

      • djce
        Mysql, yes I'm doing that.

        • 2003-02-15 04648, 2003

      • salisan
        Is the database still mysql compatible or is that gone nowdays?

        • 2003-02-15 04650, 2003

      • ruaok
        I really love the concept of CCCP.

        • 2003-02-15 04603, 2003

      • ruaok
        mysql: no

        • 2003-02-15 04607, 2003

      • djce
        MySQL doesn't do foreign keys, AFAIK.

        • 2003-02-15 04613, 2003

      • ruaok
        or transactions.

        • 2003-02-15 04614, 2003

      • ruaok
        arrays

        • 2003-02-15 04629, 2003

      • salisan
        djce: Actually, innodb is supposed to have basic foregin key support now ..

        • 2003-02-15 04639, 2003

      • djce
        The bottom line is that the database can be 99% loaded into MySQL, but you can't run a MB server off MySQL

        • 2003-02-15 04659, 2003

      • djce
        (the missing 1% is, as ruaok says, array columns)

        • 2003-02-15 04615, 2003

      • ruaok
        But I don't really like those anymore. :-)

        • 2003-02-15 04622, 2003

      • ruaok
        They are too slow...

        • 2003-02-15 04622, 2003

      • djce doesn't trust MySQL for anything too important

        • 2003-02-15 04630, 2003

      • ruaok agrees with djce

        • 2003-02-15 04639, 2003

      • salisan
        djce: It's good for selects.

        • 2003-02-15 04647, 2003

      • djce
        As long as they are simple :-)

        • 2003-02-15 04654, 2003

      • ruaok
        djce: Your parsing code was really close.

        • 2003-02-15 04658, 2003

      • djce
        and you're not trying to INSERT at the same time!

        • 2003-02-15 04616, 2003

      • djce
        parsing - yes, I saw your final diff.

        • 2003-02-15 04617, 2003

      • intrep
        does mysql have finer than table grain locking yet?

        • 2003-02-15 04629, 2003

      • salisan
        intrep: Yes, the InnoDB table type does

        • 2003-02-15 04630, 2003

      • ruaok
        modpending, page) values ('Andy Breckman', 'Andy Breckman', '153ccb00-fcad-4c79-a178-207aae196a4c', 0, 83028003)'

        • 2003-02-15 04634, 2003

      • djce
        InnoDB does, apparently. I've not used it yet though

        • 2003-02-15 04638, 2003

      • ruaok
        I need to look into those errors....

        • 2003-02-15 04647, 2003

      • djce
        Yes, I see those every now and then.

        • 2003-02-15 04622, 2003

      • djce
        ruaok: did you notice that MBLogger also creates HTML and RDF logs?!

        • 2003-02-15 04634, 2003

      • djce
        Very flash... :-)

        • 2003-02-15 04636, 2003

      • intrep
        anyone worked on importing into oracle yet?

        • 2003-02-15 04639, 2003

      • ruaok
        116860 TRMs collected

        • 2003-02-15 04648, 2003

      • djce
        Not AFAIK

        • 2003-02-15 04651, 2003

      • djce
        (oracle)

        • 2003-02-15 04611, 2003

      • djce
        but if you have an Oracle server available, please feel free to give it a go.

        • 2003-02-15 04612, 2003

      • ruaok
        djce: haven't looked. where does the output go?

        • 2003-02-15 04614, 2003

      • salisan
        Can anyone actually afford Oracle? ;)

        • 2003-02-15 04616, 2003

      • intrep
        i have to admit, im really impressed with the trm technology

        • 2003-02-15 04629, 2003

      • intrep
        i can afford oracle

        • 2003-02-15 04635, 2003

      • intrep
        so can you

        • 2003-02-15 04643, 2003

      • intrep
        sign up for the developer copy

        • 2003-02-15 04645, 2003

      • intrep
        :)

        • 2003-02-15 04646, 2003

      • djce
        ruaok: MBLogger output is currently in /home/dave/irc-logger/irc-logs/musicbrainz

        • 2003-02-15 04604, 2003

      • ruaok
        djce: nice.

        • 2003-02-15 04641, 2003

      • djce
        With the capability to dump the data in multiple formats (which I'm doing now), the tab-sep format is particularly simple to parse, and therefore load into any other data store.

        • 2003-02-15 04605, 2003

      • intrep
        salisan: check out www.oracle.com you can sign up and download oracle 8i and 9i for free, but youre not supposed to use it for anything production

        • 2003-02-15 04611, 2003

      • djce
        Loading into Oracle should be trivial really

        • 2003-02-15 04632, 2003

      • djce
        I might give that a try, when I have a few hours download window to spare :-)

        • 2003-02-15 04645, 2003

      • djce
        It would be nice just to say that it can be done.

        • 2003-02-15 04659, 2003

      • intrep
        yeah, postgres and oracle are fairly sql compatible

        • 2003-02-15 04624, 2003

      • intrep
        unlike some other databases i know <coughmysqlcough>

        • 2003-02-15 04630, 2003

      • djce
        :-)

        • 2003-02-15 04630, 2003

      • salisan
        hehe

        • 2003-02-15 04646, 2003

      • salisan
        Uptime: 991616 Threads: 187 Questions: 62557009 Slow queries: 799 Opens: 60166 Flush tables: 1 Open tables: 64 Queries per second avg: 63.086

        • 2003-02-15 04649, 2003

      • djce
        cough cough msaccess cough splutter

        • 2003-02-15 04650, 2003

      • salisan
        I like my mySql ;)

        • 2003-02-15 04602, 2003

      • djce
        Maybe SQL server too. Anyone have that to play with?

        • 2003-02-15 04629, 2003

      • ruaok
        EEEEK. Not that.

        • 2003-02-15 04631, 2003

      • ruaok
        :-)

        • 2003-02-15 04636, 2003

      • intrep
        and if you do have SQL server, how was last week for you? ;)

        • 2003-02-15 04645, 2003

      • djce
        s/have/had/ ;-)

        • 2003-02-15 04621, 2003

      • djce
        Somehow "dinosaurs" and "asteroid" comes to mind :-)

        • 2003-02-15 04650, 2003

      • salisan
        Running sql server queries over internet must be an interesting exercise after that mess ..

        • 2003-02-15 04612, 2003

      • djce
        It's definitely gone reasonably quiet on port 1434 now.

        • 2003-02-15 04638, 2003

      • djce
        (I work for an ISP, and was monitoring that port the other day, just to see)

        • 2003-02-15 04602, 2003

      • salisan
        ruaok: I really doubt you will have any more success with Google now btw..

        • 2003-02-15 04626, 2003

      • ruaok
        it was worth a shot. who knows.

        • 2003-02-15 04633, 2003

      • ruaok
        what makes you say that though?

        • 2003-02-15 04614, 2003

      • salisan
        ruaok: Everyone is after their traffic to make money ..

        • 2003-02-15 04630, 2003

      • Mutiny
        Mutiny (~trivial@h0000deadbeef.ne.client2.attbi.com) has joined #musicbrainz

        • 2003-02-15 04627, 2003

      • ruaok
        salisan: understood. But we have a good relationship with Google so far....

        • 2003-02-15 04646, 2003

      • salisan
        Is there any way to implement Last-Modified: or Expires: headers for the generated pages? ..

        • 2003-02-15 04659, 2003

      • salisan
        Browsing around on 56k6 is not so nice ;)

        • 2003-02-15 04639, 2003

      • djce
        That's hard. Desirable, but hard.

        • 2003-02-15 04656, 2003

      • djce
        I expect it's waaaaay down the "TODO" list :-(

        • 2003-02-15 04616, 2003

      • salisan
        It would probably cut down the traffic per user alot also

        • 2003-02-15 04626, 2003

      • djce
        Good point.

        • 2003-02-15 04604, 2003

      • djce wonders why HTTP cacheing always caches exactly the opposite of what you want it to...

        • 2003-02-15 04608, 2003

      • JavaGeek has left the channel

        • 2003-02-15 04622, 2003

      • intrep
        are you talking about for album/artist/track searches?

        • 2003-02-15 04642, 2003

      • intrep
        or for moderation data?

        • 2003-02-15 04645, 2003

      • salisan
        intrep: Not just searches, album/artist pages .. everything generated pretty much

        • 2003-02-15 04606, 2003

      • djce
        searches are probably the one thing you'd never cache. Apart from anything POSTed

        • 2003-02-15 04639, 2003

      • djce
        But cacheing album/artist pages would be great. But like I said, really quite hard to do.

        • 2003-02-15 04639, 2003

      • intrep
        couldnt you keep a lastmod column in albumjoin and trackjoin?

        • 2003-02-15 04655, 2003

      • ruaok
        ick. not for caching purposes.

        • 2003-02-15 04624, 2003

      • djce
        Trouble is, from the server's point of view, by the time you've gone and checked all those timestamp values, wherever they are, you may as well had just generated the page anyway.

        • 2003-02-15 04638, 2003

      • intrep
        then when you return a page with album data you could set lastmod to max(lastmod) or something like that

        • 2003-02-15 04648, 2003

      • intrep
        true