19:35 PM
nelgin
musicbrainz@wibble:/restore/musicbrainz/musicbrainz-docker$ sudo docker-compose ps
2021-04-28 11812, 2021
19:35 PM
nelgin
Name Command State Ports
2021-04-28 11812, 2021
19:35 PM
nelgin
------------------------------------------------------------------------------------------------------------------------------------------
2021-04-28 11812, 2021
19:35 PM
nelgin
musicbrainz-docker_db_1 docker-entrypoint.sh postg ... Up 5432/tcp
2021-04-28 11812, 2021
19:35 PM
nelgin
musicbrainz-docker_indexer_1 python -m sir amqp_watch Exit 1
2021-04-28 11821, 2021
19:35 PM
nelgin
Yes, it shows it,but it shows it's dead.
2021-04-28 11819, 2021
19:36 PM
yvanzo
admin/configure rm live-indexing-search
2021-04-28 11828, 2021
19:36 PM
yvanzo
sudo docker-compose up -d
2021-04-28 11844, 2021
19:36 PM
nelgin
and for some reason, Ihave to do all my admin commands with sudo.
2021-04-28 11845, 2021
19:36 PM
yvanzo
2021-04-28 11800, 2021
19:37 PM
nelgin
configure: cannot set docker command: please either
2021-04-28 11800, 2021
19:37 PM
nelgin
* add the user 'musicbrainz' to the group 'sudo' or 'wheel'
2021-04-28 11800, 2021
19:37 PM
nelgin
* or set the variable $DOCKER_CMD
2021-04-28 11809, 2021
19:37 PM
nelgin
Is that usual?
2021-04-28 11831, 2021
19:37 PM
yvanzo
Yes
2021-04-28 11846, 2021
19:37 PM
yvanzo
It's because configure is using docker command.
2021-04-28 11858, 2021
19:37 PM
nelgin
Ah ok.
2021-04-28 11858, 2021
19:39 PM
nelgin
musicbrainz-docker_indexer_1 /sbin/my_init
2021-04-28 11801, 2021
19:40 PM
nelgin
Does that look better?
2021-04-28 11805, 2021
19:40 PM
yvanzo
So running sudo admin/configure works, but you can set the shell variables instead: export DOCKER_CMD='sudo docker' DOCKER_COMPOSE_CMD='sudo docker-compose'
2021-04-28 11812, 2021
19:40 PM
yvanzo
Yes
2021-04-28 11833, 2021
19:40 PM
yvanzo
2021-04-28 11824, 2021
19:41 PM
nelgin
Installing indexer AMQP extension into PostgreSQL ...
2021-04-28 11824, 2021
19:41 PM
nelgin
BEGIN
2021-04-28 11824, 2021
19:41 PM
nelgin
psql:/tmp/CreateExtensionAMQP.sql:5: ERROR: extension "amqp" already exists
2021-04-28 11824, 2021
19:41 PM
nelgin
I assume that's ok.
2021-04-28 11846, 2021
19:41 PM
yvanzo
Yes, sorry, just: sudo docker-compose exec indexer python -m sir amqp_setup
2021-04-28 11810, 2021
19:42 PM
nelgin
Ah ok
2021-04-28 11812, 2021
19:42 PM
nelgin
DOne
2021-04-28 11825, 2021
19:42 PM
nelgin
then admin/configure add live-indexing-search ?
2021-04-28 11835, 2021
19:42 PM
yvanzo
Does mq have queues now?
2021-04-28 11850, 2021
19:42 PM
nelgin
It does.
2021-04-28 11857, 2021
19:43 PM
yvanzo
Did you build search indexes already?
2021-04-28 11805, 2021
19:44 PM
nelgin
Yes, did those last night
2021-04-28 11822, 2021
19:44 PM
yvanzo
Cool, so yes, you can proceed with step 4 :)
2021-04-28 11830, 2021
19:44 PM
nelgin
so if I do sudo docker-compose down; sudo docker-compose up -d I should still see the queues right, before I proceed?
2021-04-28 11842, 2021
19:44 PM
yvanzo
Yes.
2021-04-28 11855, 2021
19:44 PM
nelgin
Right...let me try that.
2021-04-28 11813, 2021
19:45 PM
yvanzo
The queues are stored in 'mqdata' volume.
2021-04-28 11850, 2021
19:45 PM
nelgin
Just says "no items"
2021-04-28 11856, 2021
19:45 PM
nelgin
I'll give it a minute
2021-04-28 11841, 2021
19:46 PM
nelgin
Nope, just says "no items" still.
2021-04-28 11858, 2021
19:48 PM
nelgin
What do I check in the indexer container to see if the amqp setup stuck?
2021-04-28 11853, 2021
19:54 PM
nelgin
So open to suggestions cos I'm stuck and don't want to have to run the setup everytime I restart the containers.
2021-04-28 11838, 2021
19:55 PM
yvanzo
2021-04-28 11819, 2021
19:56 PM
nelgin
Will I find that in a file in the indexer container?
2021-04-28 11812, 2021
19:57 PM
nelgin
But yes, I got that output when I first ran sudo docker-compose exec indexer python -m sir amqp_setup
2021-04-28 11830, 2021
19:57 PM
yvanzo
Ok, no it's not logged elsewhere.
2021-04-28 11850, 2021
19:57 PM
nelgin
2021-04-28 11801, 2021
19:58 PM
yvanzo
I don't get why the queues disappear from rabbitmq, never seen that before.
2021-04-28 11852, 2021
19:58 PM
nelgin
Well, it's the same with cron. admin/configure just isn't making things stick, or not working or something.
2021-04-28 11820, 2021
19:59 PM
yvanzo
That is normal output (for indexer container without live indexing enabled).
2021-04-28 11840, 2021
19:59 PM
yvanzo
admin/configure is just editing .env
2021-04-28 11859, 2021
19:59 PM
nelgin
Right but sudo docker-compose exec mq rabbitmqadmin -u sir -p sir -V /search-index-rebuilder list queues shoudl list the queues, not "no items" right?
2021-04-28 11809, 2021
20:00 PM
nelgin
If I've not done step 4 as yet.
2021-04-28 11804, 2021
20:01 PM
yvanzo
Yes
2021-04-28 11817, 2021
20:01 PM
nelgin
2021-04-28 11824, 2021
20:01 PM
nelgin
There's my .env file. Maybe something is amiss there?
2021-04-28 11830, 2021
20:02 PM
yvanzo
What is in /restore/musicbrainz/musicbrainz-docker/local/indexer.ini ?
2021-04-28 11858, 2021
20:02 PM
nelgin
same as default.ini but with index_limit = 2000000 per your suggestion.
2021-04-28 11819, 2021
20:03 PM
nelgin
2021-04-28 11811, 2021
20:04 PM
yvanzo
I don’t see anything wrong.
2021-04-28 11831, 2021
20:05 PM
nelgin
I wonder if something got corrupted when /var/lib ran out of space, but that was only importing data into the database so I doubt it.
2021-04-28 11805, 2021
20:06 PM
nelgin
I could just blow it all away and start again to be sure. I don't want to keep on a wild goose chase.
2021-04-28 11827, 2021
20:06 PM
yvanzo
I can try to reproduce that but it will take some time.
2021-04-28 11800, 2021
20:08 PM
nelgin
Before you waste time on that, let me just start over. What's the best way to clean up ?
2021-04-28 11812, 2021
20:11 PM
nelgin
Finally, is there a way to get createdb.sh to use the existing .gz files without downloading them again? If I leave off -fetch will that work?
2021-04-28 11831, 2021
20:11 PM
yvanzo
Yes
2021-04-28 11823, 2021
20:13 PM
yvanzo
I don’t think it is related to /var/lib ran out of space, but I can be wrong.
2021-04-28 11829, 2021
20:13 PM
yvanzo
What is your docker version?
2021-04-28 11843, 2021
20:13 PM
nelgin
Docker version 20.10.2, build 20.10.2-0ubuntu1~20.04.2
2021-04-28 11852, 2021
20:13 PM
nelgin
docker-compose version 1.25.0, build unknown
2021-04-28 11818, 2021
20:16 PM
yvanzo
nelgin: which packages? dpkg --list | grep docker
2021-04-28 11855, 2021
20:16 PM
nelgin
docker-compose 1.25.0-1
2021-04-28 11856, 2021
20:16 PM
nelgin
docker.io 20.10.2-0ubuntu1~20.04.2
2021-04-28 11856, 2021
20:16 PM
nelgin
python3-docker 4.1.0-1
2021-04-28 11856, 2021
20:16 PM
nelgin
python3-dockerpty 0.4.1-2
2021-04-28 11840, 2021
20:17 PM
yvanzo
Thanks
2021-04-28 11855, 2021
20:18 PM
nelgin
Running sudo docker-compose build - will check back in about 20 minutes :)
2021-04-28 11806, 2021
20:19 PM
nelgin
How much longer will you be around?
2021-04-28 11844, 2021
20:19 PM
yvanzo
Probably 1h30 at most.
2021-04-28 11816, 2021
20:20 PM
yvanzo
I’m setting up a fresh instance to try to reproduce these issues.
2021-04-28 11845, 2021
20:29 PM
nelgin
It would be nice if the Vm came with all the perl modules already installed. That's one of the time consuming parts.
2021-04-28 11811, 2021
20:31 PM
nelgin
build done.
2021-04-28 11840, 2021
20:32 PM
nelgin
Ihave the .gz files in my local system, how do I get them into the right container so I don't have to -fetch ?
2021-04-28 11801, 2021
20:40 PM
yvanzo
Using 'docker cp'
2021-04-28 11847, 2021
20:40 PM
nelgin
I'm just going to configure my linux box with ftp so it'll be reusable without having to worry about that.
2021-04-28 11822, 2021
20:41 PM
yvanzo
Or move the files directly to /var/lib/...
2021-04-28 11816, 2021
20:42 PM
nelgin
..docker/...
2021-04-28 11837, 2021
20:42 PM
yvanzo
sudo docker inspect -f '{{.Mountpoint}}' musicbrainz-docker_dbdump
2021-04-28 11800, 2021
20:43 PM
nelgin
Error: No such object: musicbrainz-docker_dbdump
2021-04-28 11821, 2021
20:43 PM
nelgin
I've done the build, not at the stage to bring up the containers yet.
2021-04-28 11817, 2021
20:44 PM
yvanzo
Ok, do you start with no .env (or a fresh .env without replication set?)
2021-04-28 11814, 2021
20:45 PM
nelgin
No .env file. Everything was removed and did a new git clone
2021-04-28 11831, 2021
20:46 PM
yvanzo
sudo docker-compose up musicbrainz # it will create dbdump volume at least, then kill it.
2021-04-28 11829, 2021
20:51 PM
nelgin
$ sudo docker inspect -f '{{.Mountpoint}}' musicbrainz-docker_dbdump
2021-04-28 11836, 2021
20:51 PM
nelgin
/var/lib/docker/volumes/musicbrainz-docker_dbdump/_data
2021-04-28 11841, 2021
20:51 PM
nelgin
So I throw the files in there?
2021-04-28 11803, 2021
20:52 PM
yvanzo
yes
2021-04-28 11828, 2021
20:52 PM
yvanzo
including the LATEST file
2021-04-28 11838, 2021
20:52 PM
nelgin
Yup.
2021-04-28 11852, 2021
20:53 PM
nelgin
Ok, copied. Now shut it down before running createdb ?
2021-04-28 11807, 2021
20:54 PM
yvanzo
Yes.
2021-04-28 11832, 2021
20:54 PM
nelgin
Since I didn't use -d can I just ^C to kill it or should I do the docker-compose down ?
2021-04-28 11806, 2021
20:55 PM
yvanzo
Yes ^C
2021-04-28 11820, 2021
20:55 PM
nelgin
Super.
2021-04-28 11813, 2021
20:56 PM
nelgin
musicbrainz is down, the others are up, should I should those down too?
2021-04-28 11855, 2021
20:56 PM
yvanzo
yep, sudo docker-compose down
2021-04-28 11830, 2021
21:04 PM
nelgin
the sudo docker-compose run --rm musicbrainz createdb.sh should do it.
2021-04-28 11834, 2021
21:04 PM
nelgin
right?
2021-04-28 11859, 2021
21:06 PM
repo has quit
2021-04-28 11808, 2021
21:07 PM
nelgin
OK, it's creating the databases
2021-04-28 11824, 2021
21:11 PM
nelgin
Can I do similar with the searchdata ?
2021-04-28 11823, 2021
21:14 PM
BrainzGit
2021-04-28 11823, 2021
21:14 PM
nelgin
Yes I can. :)
2021-04-28 11827, 2021
21:14 PM
BrainzGit
2021-04-28 11830, 2021
21:14 PM
BrainzGit
2021-04-28 11835, 2021
21:14 PM
BrainzGit
[bookbrainz-site] snyk-bot opened pull request #612 (master…snyk-upgrade-cbb35c04e742531e4ebe22439f641b47): [Snyk] Upgrade: @fortawesome/fontawesome-svg-core, @fortawesome/free-brands-svg-icons, @fortawesome/free-solid-svg-icons, @fortawesome/react-fontawesome
https://github.com/bookbrainz/bookbrainz-site/pul…
2021-04-28 11857, 2021
21:18 PM
nelgin
need to get ready to go out. I'll bbl. Thanks yvanzo for your help.
2021-04-28 11846, 2021
21:19 PM
travis-ci joined the channel
2021-04-28 11846, 2021
21:19 PM
travis-ci
2021-04-28 11846, 2021
21:19 PM
travis-ci has left the channel
2021-04-28 11840, 2021
21:28 PM
v6lur has quit
2021-04-28 11845, 2021
21:28 PM
travis-ci joined the channel
2021-04-28 11845, 2021
21:28 PM
travis-ci
2021-04-28 11845, 2021
21:28 PM
travis-ci has left the channel
2021-04-28 11844, 2021
21:33 PM
travis-ci joined the channel
2021-04-28 11844, 2021
21:33 PM
travis-ci
2021-04-28 11844, 2021
21:33 PM
travis-ci has left the channel
2021-04-28 11854, 2021
21:38 PM
yvanzo
No problem, am off now.
2021-04-28 11828, 2021
21:41 PM
thomasross joined the channel
2021-04-28 11802, 2021
21:43 PM
travis-ci joined the channel
2021-04-28 11802, 2021
21:43 PM
travis-ci
2021-04-28 11802, 2021
21:43 PM
travis-ci has left the channel
2021-04-28 11844, 2021
21:50 PM
MRiddickW has quit
2021-04-28 11800, 2021
22:31 PM
MRiddickW joined the channel
2021-04-28 11822, 2021
23:07 PM
MRiddickW has quit
