#musicbrainz

sorry, I meant: back

You can optimise the fetching of release dates by not bothering if albummeta.firstfoo is null

but as for whether they should all be fetched if they're there... I don't know.

performance-wise, I don't really see an issue.

but API-wise, I've no idea if it's a good idea or not.

ok, performance is all I am worried about.

If would be more complete to kick them out with each album.

But I think they won't be used much and it would be a waste of time to retrieve them most of the time.

also true.

but if you feel that it is not much of a performance hit, then I'll just return them all the time.

its the classic query granularity vs performance issue again.

Well certainly as we are now, it should be fine, IMO.

ok, I'll go with that then.

i.e. there are not many releases per album, often zero.

orogor_ is now known as orogor

djce: what are your thoughts on merging the amazon branch?

I got around to extracting the diffs, but haven't read them yet.

ok.

mainly I want to make sure all right DB update scripts are in place.

they aren't yet.

I think I will work to nail down all the final changes for the branch right now.

I've tended not to use ServerUpdateNotes.txt, in favour of creating one or more scripts in ./admin/sql/updates, and a top-level "./upgrade" script.

understood -- I will move my stuff from there to the right updates sql files.

that schema changes are no problem, but how do we deal with loading the data that is on grunt?

ok, so it's one new table; loading the data into that table also updates albummeta, via the triggers.

it should, yes.

We need to test the performance of that approach, see if it's feasible.

or whether the albummeta data should also be imported instead of derived.

I can dump the table, save it and then see what the import is like.

I'll tinker with that.

the second approach is messier, but almost certain to be faster to run.

so #1 uses the PopulateAlbummeta sql script and the second uses two 'load' commands, right?

#1 would be, in essence:

create tables, alter tables, add indexes, triggers;

load data into new table (e.g. MBImport.pl)

(or just "\copy")

#2 would be:

create table (no indexes). create temp table for albummeta.

load both.

copy temp albummeta (2 new columns) to real albummeta

add indexes etc.

perfect.

I'll test #1 for feasability.

TP question for ya

shoot

if I get a albumtrackresult_t,

it has links to artist and album.

album also links to artist.

always the same, always different, or either?

(i.e. t->artist == t->album->artist : discuss)

it should always be the same.

would you like me to read the code to ensure that?

Hmmm, ok.

that was the one answer I assumed was wrong. :-(

it is always the same.

it uses the same object so even the pointers should be the same.

I expected it to be different for VA.

thanks for checking

VA: yes

npo

Assumption:

1) The "missing" tracks for an album are misclassified.

Is it sufficient to build a big table of all of the tracks and albums, and assume that any album with only a few tracks in it are misclassified.

Hmm.

Considering english is my first language, I should at least be able to construct coherent sentances in it.

Or spell most of the words. *sigh*

ruaok: well, the Perl tp_tagger seems to work now. Yay!

There was one hack I had to use which I don't really like,

namely that the three types of "result" objects are flattened into regular Perl hashes

instead of remaining as libtunepimp references.

but apart from that, it's a faithful conversion of the C version.

nice!

the notes I've kept for server update queries seems to have disappeared. :-(

back to deriving those or pulling them from the backscroll of psql...

djce: how do we deal with table rows in the amazon table that may have been deleted from the main database?

import without FKs then do a left join to find the mismatched ones and delete them

then enable FKs?

yes, that sounds good.

all in a transaction mind :-)

ok.

should that be done in the update.sql script?

do you have a preferred copy syntax??

copy tablename into file syntax that is...

how big is the data dump?

as many rows as the album table

hmmm

in that case I'd say, check in a SQL script into CVS which just says something like:

"copy foo from stdin"

\.

(i.e. no rows)

when we go live, we'll insert the dumped data into that script before we run it

ok

I'll try and get this all settled right now so we can merge the branch before you go to bed.

i hope

ok

djce; is the perl tp_tagger checked into CVS?

not yet, just doing it now.

it's in! Let me know how you get on with it.

djce: to update the functions and triggers can I just call the scripts as part of an update script?

the CreateFunctions, DropFunctions -- those should run fast, right?

and the indexes for the table should be created after the import, right?

I think it's safer not to call other scripts from the update script.

lots of copy+paste is the order of the day.

ok

e.g. see ./admin/sql/updates/20031231-3.sql

(this kind of crap -> updating databases)

FUCK.

My new drive is getting read errors while reading the amazon table.

hard drive is now in fridge.

is it possible to run a bad sector scan on a linux partition that holds data??

ruaok; what does the SMART on the drive say?

Nothing yet.

I had smart fail on another drive and maxtor just replaced it.

I think I'm done buying maxtor drives.

I'm hoping a sector check will allow me to keep using the drive.,

Once you can detect the bad sectors, the drive is cooked.

Once the drives can't hid the bad sectors any more, the problem is out of control..

I've seen lots of good reports about maxtors

Well, I've got a refurbished drive sitting right here.

maybe there are just bad batches/models

195 Hardware_ECC_Recovered 0x001a 063 058 000 Old_age Always - 198589042

I don't fancy the number of hours it will take to swtich to ot.

198589042 errors. Holy.

how did you get those stats?

smartctl version 5.1-14 Copyright (C) 2002-3 Bruce Allen

I'll have to try that.

That's 20 ECC faults a second.

maybe you have issues too. :-(

what kind of drive do you have??

Knio-school is now known as Knio

It's a seagate.

IIRC.

Possible that number is bogus.

I found a cool Postgres db diagram-generating program