How do you build search indexes on a standalone server?
sumedh has quit
reosarevok
nelgin: we don't rely on one person for a whole project, but one person is mostly familiar with the docker part of it - I don't think that's weird in a team of 3 (for MB proper)
nelgin
I'm not asking about docker, I'm asking about running the server standalone now.
reosarevok
Well, that's not just one person :) (although most people seem to think that's a lot more annoying than docker)
I assume you set up PostgreSQL authentication as indicated in INSTALL.md?
nelgin
I feel like the instructions are half finished. Do I need rabbitmq for example? Anything special for searched?
I can do a direct database query
in the web interface and it works.
reosarevok
You need to set up search separately, but that shouldn't matter for InitDB
But then, if you have a database working, then you don't need to InitDB?
nelgin
It did tell me "Creating search indexes" but it looked like it only took about 4 minutes.
Those are linked from INSTALL.md, although admittedly only here:
nelgin
Well, it'll be faster on my linux hardware than a vm on my windows machine.
reosarevok
"The server by itself doesn't rate limit any request it handles. If you're receiving 503s, then you're likely performing search queries without having set up a local instance of the search server along with the search index rebuilder. By default, search queries are sent to search.musicbrainz.org and are rate limited.
Once you set up your own instance, change SEARCH_SERVER in lib/DBDefs.pm to point to it."
nelgin
Yeah, I saw that, but you'd think it would give me results then, but it didn't.
Should I do solr or sir first or doesn't it matter?
reosarevok needs to be off for a while, haven't had breakfast yet
nelgin
Thanks for the help.
sumedh has quit
sumedh joined the channel
yvanzo
nelgin: we use docker in production too, but with a far higher index_limit: 2000000
yvanzo has quit
yvanzo joined the channel
Is the "Index limit exceeded" error accompanied with other log lines? It can be it hits this limit in very specific cases only we should address.
We just don't use docker-compose in production (musicbrainz-docker should be better named musicbrainz-docker-compose).
If you don’t use docker, you won’t have to set container’s resources at least. That can solve some issues when they were not set appropriately, that can also make the website less responsive during reindex.
And yes, it is certainly faster to use a Linux host (docker or not) than a VM on Windows. :)
nelgin
yvanzo, I don't know about other errors. I've shut the vm down and trying to get sir to run on my ubuntu box
The first hurdle is that there doesn't appear to be pip for phython2. Then I'm wondering, why is it still using python2?
yvanzo
no good reason, it is just late with conversion to python3, which happens to not be straightforward in this case.
Thought it doesn't really tell you how to rebuild it. Python isn't my thing
I really really give up. I might try the docker on linux.
yvanzo
The docker image for sir uses an older version of ubuntu, that explains this error has never been seen before.
nelgin
TIme to join the 21st centrury :)
I have a 12 core linux box with 32gb of memory. Can you suggest some sane parameters that'll give me some good performance? It's got 27gb free after flushing the cache so not much in use.
Any ideas on the network disconnect between the docker and the outside world?
I have ufw turned off so only using iptables
and not blocking outgoing connections anyway
Ah, this is Ubuntu's stupid DNS at work
yvanzo
"Docker manipulates iptables rules to provide network isolation. While this is an implementation detail and you should not modify the rules Docker inserts into your iptables policies" https://docs.docker.com/network/iptables/
nelgin
if systemd-resolved is running it created this stupid 127.0.0.53 ip with the resolves behind it somehow. It's a mess. I'm about to fix it.
I have Linux distros that think they're smarter than me and think they know what is best.
Can I add anything to docker-compose build to give more verbose output or logging?
yvanzo
No but you can probe the network from a temporary docker container: sudo docker run --rm metabrainz/python:2.7-20210115 bash
nelgin
Well, it said there was no image so downloaded on and when I run the command again I just get my prompt back
yvanzo
Sorry: sudo docker run -it --rm metabrainz/python:2.7-20210115 bash
nelgin
OK, there we go.
nelgin tries to remember how to find ip and routing without ip, netstat, route, ss, or netstat lol
yvanzo
From the linux host system, you can also see the details of docker networks: sudo docker network ls; sudo docker network inspect bridge; sudo docker network inspect musicbrainz-docker_default
ruaok, can you take a look again at the consul changes PR you reviewed yesterday. beta is not starting with those changes and i am unable to figure out what's wrong.
oh, i think just figured it out.
Etua has quit
ruaok
what was it?
ruaok sees the follow up config
good thing that bug was fixed. previously that error would take down ALL sites. lol
_lucifer
it still doesn't work, looks like there is another bug somewhere. :-(
i started the master on beta and that works fine so other services are working properly.