How do you build search indexes on a standalone server?
2021-04-28 11834, 2021
sumedh has quit
2021-04-28 11844, 2021
reosarevok
nelgin: we don't rely on one person for a whole project, but one person is mostly familiar with the docker part of it - I don't think that's weird in a team of 3 (for MB proper)
2021-04-28 11821, 2021
nelgin
I'm not asking about docker, I'm asking about running the server standalone now.
2021-04-28 11846, 2021
reosarevok
Well, that's not just one person :) (although most people seem to think that's a lot more annoying than docker)
2021-04-28 11855, 2021
reosarevok
I assume you set up PostgreSQL authentication as indicated in INSTALL.md?
2021-04-28 11857, 2021
nelgin
I feel like the instructions are half finished. Do I need rabbitmq for example? Anything special for searched?
2021-04-28 11810, 2021
nelgin
I can do a direct database query
2021-04-28 11828, 2021
nelgin
in the web interface and it works.
2021-04-28 11834, 2021
reosarevok
You need to set up search separately, but that shouldn't matter for InitDB
2021-04-28 11850, 2021
reosarevok
But then, if you have a database working, then you don't need to InitDB?
2021-04-28 11801, 2021
nelgin
It did tell me "Creating search indexes" but it looked like it only took about 4 minutes.
Those are linked from INSTALL.md, although admittedly only here:
2021-04-28 11839, 2021
nelgin
Well, it'll be faster on my linux hardware than a vm on my windows machine.
2021-04-28 11840, 2021
reosarevok
"The server by itself doesn't rate limit any request it handles. If you're receiving 503s, then you're likely performing search queries without having set up a local instance of the search server along with the search index rebuilder. By default, search queries are sent to search.musicbrainz.org and are rate limited.
2021-04-28 11840, 2021
reosarevok
Once you set up your own instance, change SEARCH_SERVER in lib/DBDefs.pm to point to it."
2021-04-28 11830, 2021
nelgin
Yeah, I saw that, but you'd think it would give me results then, but it didn't.
2021-04-28 11831, 2021
nelgin
Should I do solr or sir first or doesn't it matter?
reosarevok needs to be off for a while, haven't had breakfast yet
2021-04-28 11809, 2021
nelgin
Thanks for the help.
2021-04-28 11811, 2021
sumedh has quit
2021-04-28 11819, 2021
sumedh joined the channel
2021-04-28 11800, 2021
yvanzo
nelgin: we use docker in production too, but with a far higher index_limit: 2000000
2021-04-28 11845, 2021
yvanzo has quit
2021-04-28 11800, 2021
yvanzo joined the channel
2021-04-28 11841, 2021
yvanzo
Is the "Index limit exceeded" error accompanied with other log lines? It can be it hits this limit in very specific cases only we should address.
2021-04-28 11828, 2021
yvanzo
We just don't use docker-compose in production (musicbrainz-docker should be better named musicbrainz-docker-compose).
2021-04-28 11845, 2021
yvanzo
If you don’t use docker, you won’t have to set container’s resources at least. That can solve some issues when they were not set appropriately, that can also make the website less responsive during reindex.
2021-04-28 11821, 2021
yvanzo
And yes, it is certainly faster to use a Linux host (docker or not) than a VM on Windows. :)
2021-04-28 11822, 2021
nelgin
yvanzo, I don't know about other errors. I've shut the vm down and trying to get sir to run on my ubuntu box
2021-04-28 11849, 2021
nelgin
The first hurdle is that there doesn't appear to be pip for phython2. Then I'm wondering, why is it still using python2?
2021-04-28 11820, 2021
yvanzo
no good reason, it is just late with conversion to python3, which happens to not be straightforward in this case.
Thought it doesn't really tell you how to rebuild it. Python isn't my thing
2021-04-28 11844, 2021
nelgin
I really really give up. I might try the docker on linux.
2021-04-28 11852, 2021
yvanzo
The docker image for sir uses an older version of ubuntu, that explains this error has never been seen before.
2021-04-28 11822, 2021
nelgin
TIme to join the 21st centrury :)
2021-04-28 11806, 2021
nelgin
I have a 12 core linux box with 32gb of memory. Can you suggest some sane parameters that'll give me some good performance? It's got 27gb free after flushing the cache so not much in use.
Any ideas on the network disconnect between the docker and the outside world?
2021-04-28 11855, 2021
nelgin
I have ufw turned off so only using iptables
2021-04-28 11807, 2021
nelgin
and not blocking outgoing connections anyway
2021-04-28 11817, 2021
nelgin
Ah, this is Ubuntu's stupid DNS at work
2021-04-28 11852, 2021
yvanzo
"Docker manipulates iptables rules to provide network isolation. While this is an implementation detail and you should not modify the rules Docker inserts into your iptables policies" https://docs.docker.com/network/iptables/
2021-04-28 11826, 2021
nelgin
if systemd-resolved is running it created this stupid 127.0.0.53 ip with the resolves behind it somehow. It's a mess. I'm about to fix it.
2021-04-28 11848, 2021
nelgin
I have Linux distros that think they're smarter than me and think they know what is best.
Can I add anything to docker-compose build to give more verbose output or logging?
2021-04-28 11853, 2021
yvanzo
No but you can probe the network from a temporary docker container: sudo docker run --rm metabrainz/python:2.7-20210115 bash
2021-04-28 11827, 2021
nelgin
Well, it said there was no image so downloaded on and when I run the command again I just get my prompt back
2021-04-28 11805, 2021
yvanzo
Sorry: sudo docker run -it --rm metabrainz/python:2.7-20210115 bash
2021-04-28 11837, 2021
nelgin
OK, there we go.
2021-04-28 11828, 2021
nelgin tries to remember how to find ip and routing without ip, netstat, route, ss, or netstat lol
2021-04-28 11836, 2021
yvanzo
From the linux host system, you can also see the details of docker networks: sudo docker network ls; sudo docker network inspect bridge; sudo docker network inspect musicbrainz-docker_default
ruaok, can you take a look again at the consul changes PR you reviewed yesterday. beta is not starting with those changes and i am unable to figure out what's wrong.
2021-04-28 11808, 2021
_lucifer
oh, i think just figured it out.
2021-04-28 11814, 2021
Etua has quit
2021-04-28 11809, 2021
ruaok
what was it?
2021-04-28 11823, 2021
ruaok sees the follow up config
2021-04-28 11855, 2021
ruaok
good thing that bug was fixed. previously that error would take down ALL sites. lol
2021-04-28 11817, 2021
_lucifer
it still doesn't work, looks like there is another bug somewhere. :-(
2021-04-28 11811, 2021
_lucifer
i started the master on beta and that works fine so other services are working properly.
