ok, then I need to add support for the 'don't download what already exists' stuff
2013-06-28 17926, 2013
ruaok
I can sit tight.
2013-06-28 17941, 2013
ruaok
but I would like to head to lunch soon and it would be good to start this before I go
2013-06-28 17910, 2013
ocharles
I think I have that fixed up
2013-06-28 17917, 2013
ruaok
git pull?
2013-06-28 17922, 2013
ocharles
just checking I am right
2013-06-28 17945, 2013
ocharles
dah, * doesn't work if you have an empty directory
2013-06-28 17929, 2013
ruaok
k
2013-06-28 17953, 2013
ruaok
did you push?
2013-06-28 17958, 2013
ruaok
it only updated the readme
2013-06-28 17900, 2013
ocharles
no, I haven't finished my work yet
2013-06-28 17905, 2013
ocharles
ruaok: done
2013-06-28 17942, 2013
ruaok
no files nuked, it politely asked for each file
2013-06-28 17954, 2013
ruaok
very british of it. :)
2013-06-28 17956, 2013
ocharles
:)
2013-06-28 17934, 2013
ocharles
but did it pull mbdump.tar.bz2?
2013-06-28 17938, 2013
ruaok
now the download speed is sucking.
2013-06-28 17943, 2013
ruaok
yea, its doing that now.
2013-06-28 17947, 2013
ocharles
ok
2013-06-28 17932, 2013
ruaok
I wonder if I should do a blog post asking if anyone has FTP mirror space/bandwidth in the EU
2013-06-28 17944, 2013
nikki
worth a try?
2013-06-28 17928, 2013
ruaok
ok, will do after lunch
2013-06-28 17940, 2013
ruaok heads for luch
2013-06-28 17943, 2013
ruaok
+n
2013-06-28 17953, 2013
nikki_ joined the channel
2013-06-28 17930, 2013
LordSputnik has left the channel
2013-06-28 17916, 2013
djce joined the channel
2013-06-28 17907, 2013
ocharles goes to grab lunch
2013-06-28 17902, 2013
GNUton__ joined the channel
2013-06-28 17910, 2013
GNUton__
Hi there
2013-06-28 17946, 2013
voiceinsideyou joined the channel
2013-06-28 17913, 2013
ruaok
hi GNUton
2013-06-28 17922, 2013
ruaok
ocharles: download almost complete.
2013-06-28 17931, 2013
ruaok
ocharles: admin/import-db.sh: line 20: md5sum: command not found
2013-06-28 17937, 2013
ruaok
on mac its md5
2013-06-28 17932, 2013
djce joined the channel
2013-06-28 17934, 2013
cherry1
I dont get it, I thought the idea was you ran virtualbox with linux on it on your mac, PC ectera so that u dont have to worry about os differences ?
2013-06-28 17908, 2013
ruaok
yes, but this is the code to *make* VMs. :)
2013-06-28 17922, 2013
cherry1 has left the channel
2013-06-28 17940, 2013
cherry1 joined the channel
2013-06-28 17959, 2013
cherry1 has left the channel
2013-06-28 17903, 2013
ruaok
I commented out the md5sum check
2013-06-28 17941, 2013
GNUton
Hi ruaok!
2013-06-28 17950, 2013
ijabz joined the channel
2013-06-28 17952, 2013
ijabz
so this is the vagrant script, so will need dmodified vagrant script for each OS you want to run MusicbrainzInVirtualMachine on ?
2013-06-28 17951, 2013
ruaok
no.
2013-06-28 17917, 2013
ruaok
in theory it should work to generate VMs on a mac, but practice differs.
2013-06-28 17927, 2013
ruaok
so far its done pretty well.
2013-06-28 17936, 2013
fractalizator joined the channel
2013-06-28 17941, 2013
ruaok
but if it continues to be a pain, we'll say its linux only
2013-06-28 17901, 2013
ijabz
seems to negate the only advantage of using a virtual machine then to me, that you can use it on any machine that supports the virtual machine
2013-06-28 17923, 2013
ruaok
like I said its for MAKING the VMs.
2013-06-28 17929, 2013
ruaok
which only the MB devs need.
2013-06-28 17941, 2013
ijabz
oh right, ok
2013-06-28 17944, 2013
ruaok
anyone can run it in any VM host/player anywhere they want
2013-06-28 17915, 2013
mb-chat-logger joined the channel
2013-06-28 17958, 2013
GNUton
Hi guys, I would like to get all artist's wikipedia links from MB's table dumps... I can see the links in "url" table, but I don't understand how to get the ones which belong to artists.
2013-06-28 17914, 2013
ianmcorvidae
you'll want to look at l_artist_url
2013-06-28 17948, 2013
GNUton
ianmcorvidae: I tried.. but I cannot find for instance.. "Nirvana (Band)"
somewhere in there is the 'id' column, which will correspond to the 'entity1' column of l_artist_url
2013-06-28 17948, 2013
ianmcorvidae
the entity0 will then correspond to the row in the 'artist' table
2013-06-28 17930, 2013
ianmcorvidae
from which you can get whatever data you need (possibly by joining other tables, e.g. artist_name, artist_alias, the stuff for releaseand other entities, etc.)
2013-06-28 17947, 2013
GNUton
ok I'll give it another try..
2013-06-28 17951, 2013
GNUton
thanks for the hint
2013-06-28 17957, 2013
ianmcorvidae
you can see the specific link type by joining the 'link' table via the link column of the l_ table and then to link_type
one fix would be to check to see if md5sum exists. if not use md5
2013-06-28 17915, 2013
ruaok
then check each one manually.
2013-06-28 17923, 2013
ocharles
do you have "digest" ?
2013-06-28 17925, 2013
teuf joined the channel
2013-06-28 17926, 2013
ruaok
unfortunately brew doesn't have md5sum in its repo
2013-06-28 17932, 2013
ocharles
wait, not sure that helps
2013-06-28 17943, 2013
night199uk joined the channel
2013-06-28 17946, 2013
ruaok
negative
2013-06-28 17909, 2013
ocharles
k
2013-06-28 17905, 2013
ocharles
brew install md5sha1sum ?
2013-06-28 17939, 2013
ruaok
installing. :)
2013-06-28 17910, 2013
ruaok
ok, that works. :)
2013-06-28 17934, 2013
ruaok
are you updating the readme or do you want a pull request?
2013-06-28 17943, 2013
UmkaDK joined the channel
2013-06-28 17944, 2013
djce joined the channel
2013-06-28 17956, 2013
UmkaDK
Hi everyone! :)
2013-06-28 17902, 2013
ianmcorvidae
hello
2013-06-28 17908, 2013
ocharles
ruaok: what needs to be changed in the README?
2013-06-28 17914, 2013
ocharles
dependencies?
2013-06-28 17923, 2013
ruaok
mac prereqs
2013-06-28 17925, 2013
ocharles
infact, import-db.sh isn't documented at all
2013-06-28 17944, 2013
UmkaDK
Guys, is anyone aware of any memory related issues when the search server index is build??
2013-06-28 17944, 2013
ocharles
os x isn't a supported platform, so we don't tend to document that type of stuff
2013-06-28 17947, 2013
ruaok
yes it is.
2013-06-28 17904, 2013
ruaok means import-db.sh
2013-06-28 17916, 2013
ruaok
and it seems to be working fairly well, so why not support it?
2013-06-28 17920, 2013
ocharles
it seems a bad idea making osx fully supported when the people developing this don't have a mac
2013-06-28 17936, 2013
ocharles
but I'll put it in for now
2013-06-28 17948, 2013
ruaok
one of the people has only macs. :)
2013-06-28 17908, 2013
ocharles
sure, but also has the choice to use a supported platform (ubuntu). I can't choose to use osx :)
2013-06-28 17912, 2013
warp
right, so that guy should write and maintain those bits :P
2013-06-28 17932, 2013
UmkaDK
I'm noticing that the memory on the server is getting gobbled up not that long -before- index process is run and the operations team keeps putting the blame for it at my door.
2013-06-28 17956, 2013
UmkaDK
Has anyone experienced anything similar??
2013-06-28 17909, 2013
ruaok
not during indexing.
2013-06-28 17908, 2013
UmkaDK
ruaok: I feel like there is a "but" coming. ;)
2013-06-28 17922, 2013
ruaok
ocharles: hmm > Error loading /tmp/MBImport-gpXBLwfc/mbdump/recording: DBD::Pg::db pg_putcopydata failed: server closed the connection unexpectedly
2013-06-28 17923, 2013
ruaok
This probably means the server terminated abnormally
2013-06-28 17939, 2013
ocharles
ruaok: is the VM still running?
2013-06-28 17939, 2013
ruaok
UmkaDK: there might be, but you haven't asked that question yet. :)
2013-06-28 17949, 2013
ocharles
ruaok: open up the VirtualBox GUI and see what state it's in
2013-06-28 17901, 2013
ocharles
warp: is there anything in bash to print to stderr and then exit?
2013-06-28 17925, 2013
ocharles
foo || (echo x ; exit 1 )
2013-06-28 17902, 2013
ruaok
hmmm. I started spotify which made my whole machine unhappy.
2013-06-28 17913, 2013
ruaok
but now I can log in again.
2013-06-28 17922, 2013
ruaok
time to rerun the import. :(
2013-06-28 17936, 2013
reosarevok
ruaok: they are probably targeting your computer after you said bad things about them yesterday! :p
2013-06-28 17938, 2013
ocharles
ruaok: before then it's worth understanding what happened
2013-06-28 17946, 2013
ocharles
ruaok: was the VM down?
2013-06-28 17957, 2013
ruaok
$ uptime
2013-06-28 17958, 2013
ruaok
13:30:52 up 2:37, 1 user, load average: 1.12, 2.19, 2.48
2013-06-28 17959, 2013
ruaok
no
2013-06-28 17904, 2013
ocharles
did oomkiller take it out? (if osx even has oomkiller)
2013-06-28 17913, 2013
ocharles
if the vm didn't die, you can check postgresql logs
2013-06-28 17914, 2013
ruaok
reosarevok: was there a live stream of the panel, reosarevok ?
could not fdatasync log file 0, segment 223: Input/output error # does not sound good
2013-06-28 17912, 2013
warp
ocharles: you can redirect to stderror with >&2, so echo "foo" >&2
2013-06-28 17915, 2013
ocharles
check dmesg on that machine and the host too
2013-06-28 17921, 2013
ocharles
warp: but that doesn't change the exit code
2013-06-28 17921, 2013
ruaok
the checkpoint one is also not good.
2013-06-28 17927, 2013
ocharles
checkpointing is normal
2013-06-28 17933, 2013
UmkaDK
Oh… ok… let me try and phrase it as a question... :) Could anyone think of a reason why musicbrainz would force the server to eat up 4G memory in a metter of seconds everyday at pretty much the same time, and about an hour before search server builds its indexes?
2013-06-28 17933, 2013
ruaok
but I wonder if that was because my machine was swapping
2013-06-28 17935, 2013
ocharles
out the box defaults on check point segments are not tuned for bulk loading
