#metabrainz

/

      • stixx
        so now i can replicate?

        • 2016-05-28 14952, 2016

      • alastairp
        however there are 3 things we need to do first

        • 2016-05-28 14953, 2016

      • stixx
        was 5000 before

        • 2016-05-28 14913, 2016

      • alastairp
        that's different, that is the port number of the musicbrainz webserver

        • 2016-05-28 14938, 2016

      • stixx
        ok

        • 2016-05-28 14951, 2016

      • alastairp
        OK. now we need to change some default configuration of postgres so that the import process can connect to it

        • 2016-05-28 14900, 2016

      • stixx
        ok

        • 2016-05-28 14920, 2016

      • alastairp
        then, as the blog post says, you will need to download a recent dump of the musicbrainz database and import it

        • 2016-05-28 14932, 2016

      • alastairp
        first we should check how much disk space you have

        • 2016-05-28 14936, 2016

      • alastairp
        run df -h

        • 2016-05-28 14950, 2016

      • alastairp
        there will be a line that ends with /

        • 2016-05-28 14952, 2016

      • alastairp
        /dev/sda2 909G 694G 170G 81% /

        • 2016-05-28 14932, 2016

      • stixx
        ok

        • 2016-05-28 14959, 2016

      • stixx
        i have 68G available

        • 2016-05-28 14909, 2016

      • Gore
        administrating jira on saturday, changing all the things

        • 2016-05-28 14913, 2016

      • Gore
        \o/

        • 2016-05-28 14923, 2016

      • alastairp
        OK. I don't know if that'll be enough. I know that traditionally the VM doesn't have a lot of disk space. unfortunately I don't have any experience in increasing it

        • 2016-05-28 14926, 2016

      • alastairp
        we can try if you want

        • 2016-05-28 14929, 2016

      • opatel99 has quit

        • 2016-05-28 14940, 2016

      • stixx
        yeah lets do it

        • 2016-05-28 14945, 2016

      • alastairp
        do you have a commandline text editor that you know how to use?

        • 2016-05-28 14958, 2016

      • stixx
        i have nano

        • 2016-05-28 14902, 2016

      • alastairp
        great

        • 2016-05-28 14923, 2016

      • alastairp
        let's start with the database access

        • 2016-05-28 14946, 2016

      • alastairp
        sudo nano /etc/postgresql/9.5/main/pg_hba.conf

        • 2016-05-28 14914, 2016

      • stixx
        ok

        • 2016-05-28 14921, 2016

      • alastairp
        if you scroll down, there is a section that says

        • 2016-05-28 14922, 2016

      • alastairp
        # TYPE DATABASE USER ADDRESS METHOD

        • 2016-05-28 14946, 2016

      • stixx
        yep

        • 2016-05-28 14959, 2016

      • alastairp
        ah, one sec. just got to check something

        • 2016-05-28 14933, 2016

      • alastairp
        ok, let's go up a few lines:

        • 2016-05-28 14954, 2016

      • alastairp
        local all postgres peer

        • 2016-05-28 14901, 2016

      • alastairp
        *before* this line, write:

        • 2016-05-28 14904, 2016

      • alastairp
        local all all trust

        • 2016-05-28 14929, 2016

      • alastairp
        don't worry about aligning the columns. you only need at least one space between the words

        • 2016-05-28 14947, 2016

      • alastairp
        when you quit that, restart postgres with

        • 2016-05-28 14952, 2016

      • stixx
        ok

        • 2016-05-28 14902, 2016

      • alastairp
        sudo service postgresql restart

        • 2016-05-28 14921, 2016

      • alastairp
        now let's test that you can log into the database

        • 2016-05-28 14925, 2016

      • alastairp
        psql -U postgres template1

        • 2016-05-28 14908, 2016

      • stixx
        ok i have template1=#

        • 2016-05-28 14925, 2016

      • alastairp
        great!

        • 2016-05-28 14934, 2016

      • alastairp
        \q will quit

        • 2016-05-28 14957, 2016

      • alastairp
        now we need to tell postgres to use more ram

        • 2016-05-28 14905, 2016

      • stixx
        ok

        • 2016-05-28 14939, 2016

      • alastairp
        sudo vim /etc/init.d/postgresql-config

        • 2016-05-28 14948, 2016

      • alastairp
        change 9.3 to 9.5 (in 2 places)

        • 2016-05-28 14930, 2016

      • alastairp
        uh, sudo nano if you want :)

        • 2016-05-28 14932, 2016

      • alastairp
        sorry, I use vim

        • 2016-05-28 14935, 2016

      • stixx
        done

        • 2016-05-28 14937, 2016

      • stixx
        np

        • 2016-05-28 14936, 2016

      • alastairp
        sudo cp /etc/postgresql/9.5/main/postgresql.conf /etc/postgresql/9.5/main/postgresql.conf.in

        • 2016-05-28 14944, 2016

      • alastairp
        sudo nano /etc/postgresql/9.5/main/postgresql.conf.in

        • 2016-05-28 14949, 2016

      • alastairp
        we have to edit 3 parts:

        • 2016-05-28 14934, 2016

      • alastairp
        there is a line that starts with #shared_buffers

        • 2016-05-28 14939, 2016

      • alastairp
        delete it and replace it with

        • 2016-05-28 14939, 2016

      • alastairp
        shared_buffers

        • 2016-05-28 14949, 2016

      • alastairp
        sorry, replace it with @shared_buffers@

        • 2016-05-28 14912, 2016

      • alastairp
        the line #work_mem replace with @work_mem@

        • 2016-05-28 14939, 2016

      • alastairp
        the line #effective_cache_size replace with @effective_cache_size@

        • 2016-05-28 14948, 2016

      • stixx
        do i delete the values too?

        • 2016-05-28 14903, 2016

      • stixx
        eg shared_buffers = 128MB

        • 2016-05-28 14923, 2016

      • alastairp
        yes. the whole line

        • 2016-05-28 14952, 2016

      • alastairp
        because we will run a program which calculates optimal values for the amount of ram that you have configured in your VM

        • 2016-05-28 14901, 2016

      • stixx
        not seing this #effective_cache_size is it on the same section?

        • 2016-05-28 14942, 2016

      • alastairp
        no, it's much further down the file

        • 2016-05-28 14939, 2016

      • stixx
        found it

        • 2016-05-28 14955, 2016

      • stixx
        done

        • 2016-05-28 14908, 2016

      • alastairp
        cool. let's see if it worked

        • 2016-05-28 14935, 2016

      • alastairp
        sudo /etc/init.d/postgresql-config start

        • 2016-05-28 14959, 2016

      • alastairp
        it should print some text starting with "Using the following postgres optimization settings:"

        • 2016-05-28 14923, 2016

      • stixx
        Using the following postgres optimization settings: shared_buffers: 492MB effective_cache_size = 1501MB work_mem = 20MB shmmax = 578269511 kernel.shmmax = 578269511

        • 2016-05-28 14953, 2016

      • alastairp
        great

        • 2016-05-28 14903, 2016

      • alastairp
        let's restart postgres, and this step should be done

        • 2016-05-28 14910, 2016

      • alastairp
        sudo service postgresql restart

        • 2016-05-28 14925, 2016

      • stixx
        kol

        • 2016-05-28 14930, 2016

      • stixx
        *kool

        • 2016-05-28 14941, 2016

      • stixx
        mate how do you know all this?

        • 2016-05-28 14950, 2016

      • alastairp
        I've done this a lot

        • 2016-05-28 14951, 2016

      • stixx
        you must be a wizard

        • 2016-05-28 14905, 2016

      • alastairp
        I even have a wizard hat which I wear sometimes

        • 2016-05-28 14909, 2016

      • stixx
        lol

        • 2016-05-28 14911, 2016

      • JonnyJD has quit

        • 2016-05-28 14949, 2016

      • alastairp
        did you do git fetch in the musicbrainz-server folder?

        • 2016-05-28 14911, 2016

      • stixx
        not sure

        • 2016-05-28 14936, 2016

      • alastairp
        ok, we can do it again, no problem

        • 2016-05-28 14900, 2016

      • alastairp
        we're going to be following the instructions from the musicbrainz server install document: https://github.com/metabrainz/musicbrainz-server/…

        • 2016-05-28 14920, 2016

      • stixx
        kool

        • 2016-05-28 14940, 2016

      • alastairp
        I think we'll have to run step 1, step 2 we have already done, step 3.ii is the import step

        • 2016-05-28 14903, 2016

      • alastairp
        also, from the blog post here: https://blog.musicbrainz.org/2016/05/25/schema-ch…

        • 2016-05-28 14929, 2016

      • stixx
        ok

        • 2016-05-28 14929, 2016

      • alastairp
        we have to run steps 2-10

        • 2016-05-28 14959, 2016

      • alastairp
        hmm. this might take a while

        • 2016-05-28 14902, 2016

      • stixx
        ready

        • 2016-05-28 14924, 2016

      • stixx
        sorry for taking up all your time

        • 2016-05-28 14927, 2016

      • alastairp
        does anyone know how repication cron is configured?

        • 2016-05-28 14951, 2016

      • alastairp
        if you run crontab -e, do you see a line that contains "replication" ?

        • 2016-05-28 14938, 2016

      • stixx
        no crontab for vm - using an empty one

        • 2016-05-28 14945, 2016

      • alastairp
        hmm, ok. not there

        • 2016-05-28 14957, 2016

      • alastairp
        ohwell, let's not worry. it shouldn't cause too much of a problem

        • 2016-05-28 14917, 2016

      • alastairp
        let's run from the blog, steps 4, 5, 6

        • 2016-05-28 14906, 2016

      • stixx
        ok

        • 2016-05-28 14939, 2016

      • stixx
        The program 'git' is currently not installed. looks like i dont have this

        • 2016-05-28 14947, 2016

      • The_Freso joined the channel

        • 2016-05-28 14950, 2016

      • alastairp
        whoops

        • 2016-05-28 14958, 2016

      • alastairp
        sudo apt-get install git

        • 2016-05-28 14930, 2016

      • Freso has quit

        • 2016-05-28 14930, 2016

      • The_Freso is now known as Freso

        • 2016-05-28 14938, 2016

      • stixx
        Usage: cpanm [options] Module [...] getting this when I run cpanm --installdeps --notest

        • 2016-05-28 14926, 2016

      • stixx
        wait

        • 2016-05-28 14932, 2016

      • stixx
        forgot the dot

        • 2016-05-28 14915, 2016

      • stixx
        --> Working on . Configuring /home/musicbrainz/musicbrainz-server ... FAIL ! Timed out (> 60s). Use --verbose to retry. ! Configuring . failed. See /home/vm/.cpanm/work/1464432287.12460/build.log for details. vm@musicbrainzvm:/home/musicbrainz/musicbrainz-server$

        • 2016-05-28 14908, 2016

      • stixx
        sorry got a error

        • 2016-05-28 14951, 2016

      • alastairp
        Hmm. Not sure, sorry

        • 2016-05-28 14903, 2016

      • alastairp
        I don't often work with this part of musicbrainz

        • 2016-05-28 14938, 2016

      • alastairp
        It may be that you vm is having trouble connecting to the Internet or to the cpan site

        • 2016-05-28 14949, 2016

      • alastairp
        You could try again soon

        • 2016-05-28 14928, 2016

      • stixx
        tried a couple times can we skip that?

        • 2016-05-28 14958, 2016

      • alastairp
        I'm not sure

        • 2016-05-28 14915, 2016

      • alastairp
        I think for the next step we can

        • 2016-05-28 14927, 2016

      • alastairp
        Try step 1 in the install document

        • 2016-05-28 14910, 2016

      • alastairp
        you may have to apt-get install postgresql-server-dev-9.5 in order for the commands to work

        • 2016-05-28 14953, 2016

      • stixx
        vm@musicbrainzvm:/home/musicbrainz/musicbrainz-server/postgresql-musicbrainz-unaccent$ make make: *** No targets specified and no makefile found. Stop.

        • 2016-05-28 14911, 2016

      • stixx
        any suggestions

        • 2016-05-28 14910, 2016

      • alastairp
        ah. yeah

        • 2016-05-28 14902, 2016

      • alastairp
        these folders are git submodules

        • 2016-05-28 14908, 2016

      • alastairp
        and perhaps they are not configured

        • 2016-05-28 14917, 2016

      • alastairp
        I don't know how to update them

        • 2016-05-28 14928, 2016

      • stixx
        ok

        • 2016-05-28 14932, 2016

      • alastairp
        what if you run

        • 2016-05-28 14939, 2016

      • alastairp
        git submodule update --init

        • 2016-05-28 14948, 2016

      • alastairp
        from inside the unaccent directory?

        • 2016-05-28 14955, 2016

      • stixx
        ok it did something

        • 2016-05-28 14935, 2016

      • stixx
        and gave me this Submodule path '../postgresql-musicbrainz-unaccent': checked out '62989e884ba1008da8a5dd0aa2ba0b7bf0ecb725'

        • 2016-05-28 14945, 2016

      • stixx
        is that good?

        • 2016-05-28 14923, 2016

      • stixx
        ?

        • 2016-05-28 14913, 2016

      • stixx
        alastairp

        • 2016-05-28 14923, 2016

      • stixx
        if your busy I can catch you later mate?

        • 2016-05-28 14920, 2016

      • reosarevok
        I *think* that's good

        • 2016-05-28 14952, 2016

      • stixx
        ok so where should I go next

        • 2016-05-28 14955, 2016

      • reosarevok
        Now do the cd postgresql-musicbrainz-unaccent - make - etc stuff again

        • 2016-05-28 14909, 2016

      • reosarevok
        See if you get something different

        • 2016-05-28 14949, 2016

      • reosarevok
        I'm off for lunch, but if that works you can try to work down the install doc until something doesn't work

        • 2016-05-28 14958, 2016

      • reosarevok
        If not, then hopefully alastairp will be back soonish :)

        • 2016-05-28 14916, 2016

      • stixx
        thanks mate ill give it a shot

        • 2016-05-28 14908, 2016

      • stixx
        and thanks alastairp for all the help I have learned alot today. I will try on my own from here and come back if I cant get it to work but your help has been greatly appreciated!!

        • 2016-05-28 14949, 2016

      • alastairp
        hey. no problem

        • 2016-05-28 14958, 2016

      • rickatnight11 joined the channel

        • 2016-05-28 14901, 2016

      • alastairp
        you should be able to follow the instructions in INSTALL.md now to get the database imported

        • 2016-05-28 14941, 2016

      • stixx
        thanks will do. Have a great weekend