zas: i figured some more things. connection fails on ipv6. psql tries ipv6 first, after a few retries gives up then connects on ipv4 and works. if i use ipv6 for direct connection to pink, it fails to pass firewall/.
monkey: hi! someone is looking to build an open source book recommendation engine and is looking for ISBN data. how much relevant data would BB have in your opinion?
lucifer: I need to change the other query to user_id, no need to review the query
2022-12-14 34823, 2022
lucifer
mayhem: ah cool 👍. indeed looks better.
2022-12-14 34827, 2022
lucifer
is this sorted in any way?
2022-12-14 34832, 2022
lucifer
bitmap: yes and no.
2022-12-14 34833, 2022
Sophist-UK joined the channel
2022-12-14 34808, 2022
mayhem
sorted feels dumb (because two tracks come back to back often if you listen to whole albums) so it is randomized.
2022-12-14 34813, 2022
yvanzo
zas, reosarevok, bitmap: What actually happened on Monday was about ~270 Mastodon instances trying to reach the same page in a range of ~5 minutes.
2022-12-14 34818, 2022
mayhem
but even that isn't great.
2022-12-14 34821, 2022
lucifer
makes sense.
2022-12-14 34826, 2022
reosarevok
Fun.
2022-12-14 34800, 2022
mayhem
oh fuck, mastondon as as a DDoS tool. what could possibly go wrong?
2022-12-14 34813, 2022
alastairp
in interesting
2022-12-14 34829, 2022
alastairp
since it's federated, every instance needs to get their own preview text?
2022-12-14 34841, 2022
mayhem
yep
2022-12-14 34815, 2022
mayhem
soon we'll see SRE's throttling mastondon instances and the mess starts unraveling.
2022-12-14 34830, 2022
yvanzo
There has been a separate similar request earlier so I guess that someone did the search and posted the link on Mastodon and their post got replicated among instances, then each instance tried to make an overview of the page. (I don’t know what Mastodon is doing about posted links really.)
2022-12-14 34842, 2022
alastairp
it's a shitty solution, but I wonder if we could do less sql queries on metadata scrapes (based on user agent sniffing)
2022-12-14 34805, 2022
alastairp
but yeah, that way leads to a slippery slope of configuration options
2022-12-14 34810, 2022
lucifer
270 requests to 1 page in ~5 mins should be fine in general though?
2022-12-14 34817, 2022
yvanzo
But it may also have revealed a potential issue on our side: Since it was an MB direct search, that is, it queries Postgres instead of Solr. I don’t if there is any cache to handle this kind of search.
2022-12-14 34801, 2022
yvanzo
lucifer: ~800 requests (each instance made up to 3 attempts) but yes, was my though too.
2022-12-14 34816, 2022
lucifer
ah ok, makes sense.
2022-12-14 34833, 2022
alastairp
what are you doing to poor gaga
2022-12-14 34843, 2022
yvanzo
bitmap, reosarevok: Do you think that caching direct search can be improved and/or that it should be unlinkable (e.g. using ws/js)?
2022-12-14 34845, 2022
mayhem
alastairp: that's me sorry.
2022-12-14 34856, 2022
jivte joined the channel
2022-12-14 34805, 2022
mayhem
recalculating data for playlists for yim
2022-12-14 34802, 2022
alastairp
mayhem: I'm listening to my daily mix
2022-12-14 34817, 2022
mayhem
thanks!
2022-12-14 34819, 2022
alastairp
as expected, it's relatively well known tracks of "safe" artists around my tastes
2022-12-14 34834, 2022
alastairp
very good as a "turn on and put in the background" list
2022-12-14 34838, 2022
mayhem
thats the goal. easy listening for the post-muzak age.
2022-12-14 34843, 2022
mayhem
yay!
2022-12-14 34852, 2022
mayhem
!m team LB
2022-12-14 34853, 2022
BrainzBot
You're doing good work, team LB!
2022-12-14 34803, 2022
alastairp
so i"m really interested to see how it'll change as I work through the well-known songs
2022-12-14 34816, 2022
mayhem
got 4-5 weeks to find the answer?
2022-12-14 34826, 2022
alastairp
that's the plan
2022-12-14 34811, 2022
jasje joined the channel
2022-12-14 34851, 2022
reosarevok
alastairp: my understanding is that it will start replacing them with Running up That Hill
2022-12-14 34856, 2022
mayhem
now if I could get zas to listen to daily jams for a week and tell me what he thinks....
2022-12-14 34805, 2022
mayhem
and on april 1st...
2022-12-14 34812, 2022
monkey
Oh yes
2022-12-14 34816, 2022
alastairp
reosarevok: tbh, I would't be surprised if it turns up on my list before long
2022-12-14 34832, 2022
monkey
Running Down That List
2022-12-14 34837, 2022
mayhem
rickroll has already shown up in my daily jams.
2022-12-14 34838, 2022
alastairp
mayhem: actually, I can see where that sqrt trick comes in handy
2022-12-14 34850, 2022
mayhem
already?
2022-12-14 34801, 2022
reosarevok
yvanzo: unlinkable as in, rather than have https://musicbrainz.org/search?query=test&typ…, you'd have a copyable URL that is always an indexed one and you can only pick Direct from there?
2022-12-14 34802, 2022
alastairp
especially for this kind of sudden-popularity-due-to-tv spike
2022-12-14 34832, 2022
mayhem
not 100% sure sqrt will help in this case.
2022-12-14 34843, 2022
jasje
hey lucifer! i wanted to ask you about to_ts(to_timestamp right?) what does the numbers mean (difference is 86400 ie day in seconds) but what is up with the extra stuff?
2022-12-14 34855, 2022
alastairp
is that for smoothing the number of listens of a user-track pair?
2022-12-14 34857, 2022
yvanzo
bitmap, reosarevok: I compared MB website logs with Postgres logs, it isn’t cached at all. 1min cache seems to be reasonable.
I guess I was kind of thinking of it in terms of the "beatles" issue
2022-12-14 34829, 2022
alastairp
if everyone listens to kate bush for 3 weeks
2022-12-14 34836, 2022
alastairp
then she becomes similar to everything
2022-12-14 34841, 2022
bitmap
yvanzo: 1m is probably fine
2022-12-14 34847, 2022
mayhem
yeah, but I think that is statistally unlikely to happen
2022-12-14 34853, 2022
mayhem
statistically, even.
2022-12-14 34855, 2022
alastairp
is there a way to surpress tat similarity a bit?
2022-12-14 34800, 2022
alastairp
sure, that's possible
2022-12-14 34812, 2022
mayhem
yes.
2022-12-14 34813, 2022
monkey
jasje: They're the timestamps for the beginning and end of a period (in this case for a day)
2022-12-14 34820, 2022
mayhem
I need to post the damn videos. I keep forgetting when I am home.
2022-12-14 34820, 2022
alastairp
though it's likely to be more of an issue the shorter our window is
2022-12-14 34825, 2022
monkey
We use those to render a graph
2022-12-14 34841, 2022
yvanzo
reosarevok: It was just an idea in the air, but it is probably much more complicated than caching it at first. Roughly: if the user selects the direct search option as a fallback from a search results page, then the search would be made through a Node component querying ws/js so that the "method=direct" is not used in the page URL.
2022-12-14 34826, 2022
monkey
Well, actually I don't think we directly use those timestamps to render the graph
2022-12-14 34853, 2022
reosarevok
yvanzo: I mean, we could just send everything via post :) But I guess we do want to still allow linking to searches
2022-12-14 34805, 2022
reosarevok
Testing a cache seems fine to me
2022-12-14 34800, 2022
yvanzo
reosarevok: Yes but direct search is only a fallback available when indexed search is broken.
2022-12-14 34818, 2022
lucifer
jasje: to_ts is ending timestamp of the period. from_ts starting timestamp. say you asked for this_week stats, from_ts will be timestamp of the start of the week and to_ts of the end of the period.
2022-12-14 34816, 2022
jasje
monkey: lucifer just for curiosity how do you determine the number assigned to first day of the year, i.e., 1st jan?
2022-12-14 34842, 2022
lucifer
jasje: the listens users submit to LB are timestamped (when they listened to a particular track is available in LB). so a simple grouping of each time period and count number of listens/timestamps in that period.
