switching back, cert issues are mostly fixed (I still have to sort out minor issues, but now grade A)
2020-05-31 15215, 2020
zas
ruaok: the 403s thing was related to the cert issue it seems
2020-05-31 15227, 2020
ruaok
heh, oops.
2020-05-31 15244, 2020
zas
curl was impacted ;)
2020-05-31 15217, 2020
zas
not much an issue, because the used bandwidth by those useless requests is very low
2020-05-31 15232, 2020
zas
I'll have more corners to round, but the core issue is more or less fixed (Gandi didn't updated their generated cert, which still embeds the expired cert)
[listenbrainz-server] ishaanshah opened pull request #884 (master…master): LB-610: The URL in the browser should update with parameters for the history page https://github.com/metabrainz/listenbrainz-server…
2020-05-31 15233, 2020
ishaanshah
iliekcomputers: I have fixed LB-610, I will have a look at the other tickets tomorrow.
the last 4 weeks of data are not on the server, nor are new ones being added.
2020-05-31 15251, 2020
ruaok
just testing deduping the data
2020-05-31 15215, 2020
Zastai
yeah I've been knee deep in Jenkins plugin development; not done much Brainz-related for a few weeks
2020-05-31 15207, 2020
Zastai
does that mean that if I reset my timestamp, and redo a last.fm import on test.lb.o, the listen count should stay the same?
2020-05-31 15234, 2020
Zastai
(I haven't been scrobbling to last.fm for months, so there's no new listens there)
2020-05-31 15235, 2020
ruaok
in theory, I believe so.
2020-05-31 15255, 2020
ruaok
I'll downgrade that to a solid maybe.
2020-05-31 15233, 2020
Zastai
Was anything changed wrt avoiding rate limiting on the last.fm side? ISTR always have one or two pages of listens not making it in
2020-05-31 15205, 2020
ruaok
the importer was just recently redone; there is a good chance it may behave better
2020-05-31 15237, 2020
Zastai
ok. and with deduping (solid maybe) working, redoing it once or twice isn't a problem
2020-05-31 15254, 2020
ruaok
can't hurt to try.
2020-05-31 15206, 2020
ruaok
though, the counts won't be the same, now that I think of it.
2020-05-31 15241, 2020
ruaok
the data pre april 2005 from last.fm is so dodgy that I simply do not import anything prior to then.
2020-05-31 15202, 2020
Zastai
4750-page import started
2020-05-31 15204, 2020
ruaok
I might go and take a closer look at it or undo just undo that change.
2020-05-31 15239, 2020
Zastai
is it the idea that this cleanup (including dropping pre-april-2005 listens) will be applied on the production data?
2020-05-31 15204, 2020
ruaok
yes.
2020-05-31 15211, 2020
Zastai
(mainly I'd like to have the last.fm listens cleaned up and redone without losing the spotify ones)
2020-05-31 15233, 2020
ruaok
I think we share that same goal.
2020-05-31 15211, 2020
Zastai
(a "delete all last.fm listens" button on My Profile might be nice as a means to testing/redoing last.fm imports without touching the spotify ones)
2020-05-31 15245, 2020
ruaok
we dont have a good way of identifying all last.fm listens. we do going forward, but not for past data
2020-05-31 15206, 2020
Zastai
ok. maybe a "delete listens older than <enter date>" button then; in my case, I know when I stopped using last.fm, and I enabled spotify some time after that.
2020-05-31 15232, 2020
ruaok
I'll just run that query by hand for you. :)
2020-05-31 15238, 2020
ruaok
(but I see what you are saying)
2020-05-31 15240, 2020
Zastai
at page 300/4750 - listens seems to have gone up by ~300. so it's definitely not duping all of them. either there used to be a bug where before the import was missing 1 listen on every page, or there are a handful here and there that aren't getting deduped
2020-05-31 15249, 2020
Zastai
does the dedupe logic run per import page, or per listen insert?
2020-05-31 15220, 2020
Zastai
(400 pages. increase seems to stay at about 1 per page. i stupidly did not screenshot the count before the import started, so can't tell exactly.)
2020-05-31 15247, 2020
ruaok
the logic that deduped the production data set for test is different than the dedup logic for normal incoming listens.
2020-05-31 15231, 2020
ruaok
the deduper from influx actively seeks out the weirdest cases of the last.fm data.
2020-05-31 15259, 2020
ruaok
going forward it will accept listens that are unique on track_name, listened_at and user_name
2020-05-31 15218, 2020
ruaok
and now I realize that anyone who ever repeats a last.fm import will just make a hash of this again.
2020-05-31 15259, 2020
rdswift
Looks like a problem with BrainzBot. Not printing the titles of tickets any longer. Something happen with the recent server maintenance?
2020-05-31 15219, 2020
rdswift
Testing... MBS-9009
2020-05-31 15228, 2020
Zastai
!m BrainzBot staying quiet
2020-05-31 15228, 2020
BrainzBot
You're doing good work, BrainzBot staying quiet!
2020-05-31 15233, 2020
Zastai
:p
2020-05-31 15227, 2020
Zastai
page 1400; ~1100 new listens, so it's dipped below 1/page
2020-05-31 15228, 2020
v6lur_ has quit
2020-05-31 15247, 2020
sumedh joined the channel
2020-05-31 15247, 2020
Zastai
ruaok: import finished. 100 listens not imported (expected 237,484 - imported 237,384). listen count went from ~242,500 to 242,931 - interestingly it had previously risen to above 243,000 during the import
2020-05-31 15222, 2020
Zastai
i can certainly live with ~400 possible dupes out of 240k
2020-05-31 15229, 2020
rdswift
What includes (?inc=) to a WS call should I use to retrieve a specific release with the track titles and track artists? Everything I've tried returns the recording title but not the track title.
2020-05-31 15212, 2020
sumedh has quit
2020-05-31 15208, 2020
Chinmay3199 has quit
2020-05-31 15246, 2020
KindTwo joined the channel
2020-05-31 15227, 2020
KindOne has quit
2020-05-31 15203, 2020
KindTwo is now known as KindOne
2020-05-31 15257, 2020
sumedh joined the channel
2020-05-31 15228, 2020
sumedh has quit
2020-05-31 15235, 2020
Sophist-UK has quit
2020-05-31 15233, 2020
ruaok
> i can certainly live with ~400 possible dupes out of 240k
2020-05-31 15243, 2020
ruaok
hellz to the yes!
2020-05-31 15252, 2020
ruaok
thanks for checking it out. :)
2020-05-31 15240, 2020
v6lur_ joined the channel
2020-05-31 15209, 2020
killme joined the channel
2020-05-31 15205, 2020
prabal
Mr_Monkey: search is not working for the main website
