yvanzo, bitmap: woops, I got BG3 and completely forgot we were meant to meet yesterday D:
2023-08-10 22254, 2023
kepstin has quit
2023-08-10 22208, 2023
kepstin joined the channel
2023-08-10 22234, 2023
Zhele has quit
2023-08-10 22236, 2023
Sophist-UK joined the channel
2023-08-10 22230, 2023
yvanzo
aerozol: good evening, can do today if around, or tomorrow.
2023-08-10 22257, 2023
FichteFoll joined the channel
2023-08-10 22228, 2023
aerozol
yvanzo: evening! Something came up for the next couple of hours, but I’ll see if you’re still free (and I’m still awake) afterwards if that’s okay
2023-08-10 22203, 2023
yvanzo
okay, just ping me if around later on, or we will have another try another day :)
2023-08-10 22212, 2023
Sophist-UK has quit
2023-08-10 22236, 2023
zas
mayhem, bitmap: we already had such load issues with queries using OFFSET in the past (but I don't think it was YandexBot). Perhaps we should target such queries to one read-only server, instead of hitting main rw db. We could do that for all identified indexing bots btw.
reosarevok: might be my own mess, will check, thanks
2023-08-10 22236, 2023
reosarevok
It probably is just having an extra space somewhere or whatever :)
2023-08-10 22241, 2023
atj
IIRC, last time it was some lidarr addon that was doing WS queries for recordings with large offsets
2023-08-10 22231, 2023
zas
reosarevok: I played BG3 much before release (alpha/beta, at least 3 times), and definitively will soon again ;) Another game I had a lot of fun with recently is Wartales
lol just had an insane idea: how about a n app that uses ai or some other bullshit wher yo ucna take images of instruments and thne it will identify for you the instrument and show the info we have of it in app
2023-08-10 22246, 2023
lucifer
mayhem: for top releases of artist, what all data do you need?
mayhem: Fixed a css linting issue, tests are re-running
2023-08-10 22247, 2023
Lotheric has quit
2023-08-10 22207, 2023
Lotheric joined the channel
2023-08-10 22219, 2023
monkey
lucifer: We should aim to finish up LB#1800 when we have a minute. I'm happy to merge the functional back-end parts and work on the UI in a separate PR if it makes it easier
Mmm, but the process can be safely repeated, right?
2023-08-10 22225, 2023
monkey
IMO that might be better considering the issues this creates
2023-08-10 22212, 2023
mayhem
I suppose one can create a zip bomb with other compression formats as well.
2023-08-10 22230, 2023
lucifer
yes probably
2023-08-10 22253, 2023
mayhem
one way to mitigate zip bombs would be to use a sub-process with CPU and memory restrictions.
2023-08-10 22209, 2023
mayhem
or to unzip into ram and if you generate more than X MB of data, killit.
2023-08-10 22247, 2023
lucifer
yeah makes sense. just pointing out there are issues with doing it server side.
2023-08-10 22213, 2023
lucifer
mayhem: how about a small executable to upload it from client side?
2023-08-10 22231, 2023
mayhem
there are other benefits of server side too. we can control the rate of which things get imported.
2023-08-10 22247, 2023
mayhem
executable? like a binary?
2023-08-10 22250, 2023
lucifer
yes
2023-08-10 22253, 2023
mayhem
HELL NO.
2023-08-10 22259, 2023
mayhem
do it client side then.
2023-08-10 22220, 2023
mayhem
the nightmare of producing 3 binaries is a nightmare.
2023-08-10 22234, 2023
mayhem
if you really don't like server side, then do client side.
2023-08-10 22238, 2023
mayhem
I don't really care that much.
2023-08-10 22225, 2023
monkey
Probably safer, to be honest. Will need some clear instructions though
2023-08-10 22231, 2023
lucifer
okay let's do it client side for now then. sending files over rabbitmq is also highly discouraged.
2023-08-10 22245, 2023
monkey
i.e. unzip the file, upload all the files ending with .json
2023-08-10 22205, 2023
monkey
Hm, one thing we might need lucifer is a way for users to set a date until when to import, to avoid duplicates. What do you think? Maybe a begin and end date, optional?
2023-08-10 22232, 2023
lucifer
monkey: yes makes sense.
2023-08-10 22203, 2023
lucifer
i am looking at js libraries to see if extracting in browser is feasible.
2023-08-10 22246, 2023
monkey
Oh, extracting the zip? interesting. never thought of doing that
anything else that should get merged for a release, lucifer ?
2023-08-10 22241, 2023
lucifer
mayhem: nope, seems good to me
2023-08-10 22252, 2023
mayhem
ok, i"ll release.
2023-08-10 22248, 2023
lucifer
monkey: yeah, if we can do that then let's do it client side otherwise server side. i think most people wouldn't go to the pain of submitting each file manually after extraction.
2023-08-10 22203, 2023
monkey
Fair
2023-08-10 22229, 2023
monkey fixes the conflict in import-extended-streaming
