#musicbrainz-devel

that import won't take very long

killed it already

ok, then I need to add support for the 'don't download what already exists' stuff

I can sit tight.

but I would like to head to lunch soon and it would be good to start this before I go

I think I have that fixed up

git pull?

just checking I am right

dah, * doesn't work if you have an empty directory

k

did you push?

it only updated the readme

no, I haven't finished my work yet

ruaok: done

no files nuked, it politely asked for each file

very british of it. :)

:)

but did it pull mbdump.tar.bz2?

now the download speed is sucking.

yea, its doing that now.

ok

I wonder if I should do a blog post asking if anyone has FTP mirror space/bandwidth in the EU

worth a try?

ok, will do after lunch

+n

Hi there

hi GNUton

ocharles: download almost complete.

ocharles: admin/import-db.sh: line 20: md5sum: command not found

on mac its md5

I dont get it, I thought the idea was you ran virtualbox with linux on it on your mac, PC ectera so that u dont have to worry about os differences ?

yes, but this is the code to *make* VMs. :)

I commented out the md5sum check

Hi ruaok!

so this is the vagrant script, so will need dmodified vagrant script for each OS you want to run MusicbrainzInVirtualMachine on ?

no.

in theory it should work to generate VMs on a mac, but practice differs.

so far its done pretty well.

but if it continues to be a pain, we'll say its linux only

seems to negate the only advantage of using a virtual machine then to me, that you can use it on any machine that supports the virtual machine

like I said its for MAKING the VMs.

which only the MB devs need.

oh right, ok

anyone can run it in any VM host/player anywhere they want

Hi guys, I would like to get all artist's wikipedia links from MB's table dumps... I can see the links in "url" table, but I don't understand how to get the ones which belong to artists.

you'll want to look at l_artist_url

ianmcorvidae: I tried.. but I cannot find for instance.. "Nirvana (Band)"

296d99c5574-096b-45af-bf20-c3dc3e94fde5http://e... 16:31:52+00

this is the record in the "url" table

I'm not sure how you've formatted that

sorry, 296 is the URL ID

somewhere in there is the 'id' column, which will correspond to the 'entity1' column of l_artist_url

the entity0 will then correspond to the row in the 'artist' table

from which you can get whatever data you need (possibly by joining other tables, e.g. artist_name, artist_alias, the stuff for releaseand other entities, etc.)

ok I'll give it another try..

thanks for the hint

you can see the specific link type by joining the 'link' table via the link column of the l_ table and then to link_type

ruaok: oh, joy

yeah.

ruaok: did you fix it?

and the mac version did not have a -c option.

no, I commented it out and kept importing.

time to find a man page

for the mac?

looks like it's md5 -verify

ah no, that's something else

let me paste the man page

not promising

one fix would be to check to see if md5sum exists. if not use md5

then check each one manually.

do you have "digest" ?

unfortunately brew doesn't have md5sum in its repo

wait, not sure that helps

negative

k

brew install md5sha1sum ?

installing. :)

ok, that works. :)

are you updating the readme or do you want a pull request?

Hi everyone! :)

hello

ruaok: what needs to be changed in the README?

dependencies?

mac prereqs

infact, import-db.sh isn't documented at all

Guys, is anyone aware of any memory related issues when the search server index is build??

os x isn't a supported platform, so we don't tend to document that type of stuff

yes it is.

and it seems to be working fairly well, so why not support it?

it seems a bad idea making osx fully supported when the people developing this don't have a mac

but I'll put it in for now

one of the people has only macs. :)

sure, but also has the choice to use a supported platform (ubuntu). I can't choose to use osx :)

right, so that guy should write and maintain those bits :P

I'm noticing that the memory on the server is getting gobbled up not that long -before- index process is run and the operations team keeps putting the blame for it at my door.

Has anyone experienced anything similar??

not during indexing.

ruaok: I feel like there is a "but" coming. ;)

ocharles: hmm > Error loading /tmp/MBImport-gpXBLwfc/mbdump/recording: DBD::Pg::db pg_putcopydata failed: server closed the connection unexpectedly

This probably means the server terminated abnormally

ruaok: is the VM still running?

UmkaDK: there might be, but you haven't asked that question yet. :)

ruaok: open up the VirtualBox GUI and see what state it's in

warp: is there anything in bash to print to stderr and then exit?

foo || (echo x ; exit 1 )

hmmm. I started spotify which made my whole machine unhappy.

but now I can log in again.

time to rerun the import. :(

ruaok: they are probably targeting your computer after you said bad things about them yesterday! :p

ruaok: before then it's worth understanding what happened

ruaok: was the VM down?

$ uptime

13:30:52 up 2:37, 1 user, load average: 1.12, 2.19, 2.48

no

did oomkiller take it out? (if osx even has oomkiller)

if the vm didn't die, you can check postgresql logs

reosarevok: was there a live stream of the panel, reosarevok ?

ruaok: dunno, I just saw people talk on twitter

vagrant ssh -c "sudo tail -n100 /var/log/postgresql/postgresql-9.1-main.log"

unexpected.

could not fdatasync log file 0, segment 223: Input/output error # does not sound good

ocharles: you can redirect to stderror with >&2, so echo "foo" >&2

check dmesg on that machine and the host too

warp: but that doesn't change the exit code

the checkpoint one is also not good.

checkpointing is normal

Oh… ok… let me try and phrase it as a question... :) Could anyone think of a reason why musicbrainz would force the server to eat up 4G memory in a metter of seconds everyday at pretty much the same time, and about an hour before search server builds its indexes?

but I wonder if that was because my machine was swapping

out the box defaults on check point segments are not tuned for bulk loading