[bookbrainz-site] 14MonkeyDo merged pull request #1106 (03import-entities…import-annotation-id): chore: Allow annotation revision ids to be null (rebased) https://github.com/metabrainz/bookbrainz-site/p...
Kladky has quit
Kladky joined the channel
rimskii[m] has quit
pranav[m] has quit
discordbrainz has quit
discordbrainz joined the channel
twodoorcoupe[m] has quit
atj[m] has quit
outsidecontext[m has quit
outsidecontext[m joined the channel
BobSwift[m] has quit
kellnerd[m]
<mayhem[m]> "https://blog.metabrainz.org/2024..." <- It took me a while, but I just realized that the 23rd birthday post is from exactly a year ago and today we are already celebrating the 25th... time flies by 😁
mayhem[m]
wut?
math, my nemesis.
well, no one noticed for a whole year, so 🤷
theflash[m] has quit
atj[m] joined the channel
atj[m]
[@lucifer:chatbrainz.org](https://matrix.to/#/@lucifer:chatbrainz.org): it would be good to have the LB Solr cluster managed using Ansible, considering all the work done for the MB cluster
Is it a standard SolrCloud installation?
lucifer[m]
atj: there is not cluster yet, we are still experimenting to see if it performs better than typesense (what we have now). if it does then we will create a proper cluster for production. but yes makes sense
atj[m]
OK, well let me know how it goes. MB cluster seems to work well on ARM VMs which are cheaper and offers better performance.
Sophist-UK has quit
Don't set the Solr heap size too high would be my advice. Ideally you want to fit the indexes in page cache but I don't know how big they will be for LB
lucifer[m]
makes sense, will keep it in mind.
atj[m]
Are you storing documents in Solr or just IDs?
Storing the MB document XML in Solr was a mistake IMV, it's resulted in the indexes being much bigger than they need to be and reduces performance significantly
But then I don't know the whole history behind it.
mayhem[m] uploaded an image: (546KiB) < https://matrix.chatbrainz.org/_matrix/media/v3/download/chatbrainz.org/ugbPRzkrNHFCnDzqTPgdgqPa/image.png >
mayhem[m]
LOLOLOLOL, I feel seen.
leftmostcatUTC-7 joined the channel
leftmostcatUTC-7
I feel trapped.
lucifer[m]
<atj[m]> "Are you storing documents in..." <- JSON docs, only non searchable fields are mbids.
<atj[m]> "But then I don't know the..." <- indeed, i have discussed with yvanzo in the past to get rid of it.
i would like to get rid of it too for performance reasons, its slows down writing the response too imo.
so a perf win in many ways if we can get rid of it.
should probably restart that discussion now that solr 9 upgrade is done.
mayhem[m]
<atj[m]> "But then I don't know the..." <- Back in the day we favored not having to do another DB query to fetch the data. we accepted the larger indexes as a tradeoff for less load on our DB server. but we're in a much different place now. but scaling wise, hosting capabilities and money.
lucifer[m]
would actually simplify a lot of stuff in Sir too.
mayhem[m]: also it is XML centric.
XML requests are served directly iirc.
where as JSON requests need to deserialize the xml and reserialize it to JSON to serve the request
atj[m]
and it's a shitty Java dependency that nobody in their right mind should want to deal with
lucifer[m]
yeah fair and solr has built in xml support too iirc so if we want xml we can get that directly anyway.
