#metabrainz

Good morninggg

Hullo!

Morning!

Morning

!m bitmap

You're doing good work, bitmap!

I caught another instance of one of our ban evaders last night thanks to the new tool. Thank you so much bitmap! ❤️

ruaok: ping

Pong

On mobile, in the middle of device surgery.

ok, there are problems with certs due to intermediate cert which expired

I noticed gandi *.mb.o cert is in pending state (not sure why)

ah no, one does, never removed, but there is another one, which is ok, gandi shit

Wait wait. I thought they were "no BS"?

😆

are these cert issues related to https://crt.sh/?id=1 as well?

I'll switch to herb to test updated ssl config

switching back to kiki

switching to herb again, fixes in progress

switching back, cert issues are mostly fixed (I still have to sort out minor issues, but now grade A)

ruaok: the 403s thing was related to the cert issue it seems

heh, oops.

curl was impacted ;)

not much an issue, because the used bandwidth by those useless requests is very low

I'll have more corners to round, but the core issue is more or less fixed (Gandi didn't updated their generated cert, which still embeds the expired cert)

I'm off for lunch

[bookbrainz-site] prabalsingh24 opened pull request #434 (UserCollection…editor-collections): Collections: Editor Profile Tab https://github.com/bookbrainz/bookbrainz-site/p...

[listenbrainz-server] ishaanshah opened pull request #884 (master…master): LB-610: The URL in the browser should update with parameters for the history page https://github.com/metabrainz/listenbrainz-serv...

iliekcomputers: I have fixed LB-610, I will have a look at the other tickets tomorrow.

Another dubious listening history… https://listenbrainz.org/user/merakil

All the more reasons for ORG-40

yeah; a user created only 12 hours ago on MB

I mean, there is no sandbox, so it _could_ be someone testing code that uses the web services

Zastai: long time no see! I have your listens deduped: https://test.listenbrainz.org/user/Zastai?max_t...

ooh

the last 4 weeks of data are not on the server, nor are new ones being added.

just testing deduping the data

yeah I've been knee deep in Jenkins plugin development; not done much Brainz-related for a few weeks

does that mean that if I reset my timestamp, and redo a last.fm import on test.lb.o, the listen count should stay the same?

(I haven't been scrobbling to last.fm for months, so there's no new listens there)

in theory, I believe so.

I'll downgrade that to a solid maybe.

Was anything changed wrt avoiding rate limiting on the last.fm side? ISTR always have one or two pages of listens not making it in

the importer was just recently redone; there is a good chance it may behave better

ok. and with deduping (solid maybe) working, redoing it once or twice isn't a problem

can't hurt to try.

though, the counts won't be the same, now that I think of it.

the data pre april 2005 from last.fm is so dodgy that I simply do not import anything prior to then.

4750-page import started

I might go and take a closer look at it or undo just undo that change.

is it the idea that this cleanup (including dropping pre-april-2005 listens) will be applied on the production data?

yes.

(mainly I'd like to have the last.fm listens cleaned up and redone without losing the spotify ones)

I think we share that same goal.

(a "delete all last.fm listens" button on My Profile might be nice as a means to testing/redoing last.fm imports without touching the spotify ones)

we dont have a good way of identifying all last.fm listens. we do going forward, but not for past data

ok. maybe a "delete listens older than <enter date>" button then; in my case, I know when I stopped using last.fm, and I enabled spotify some time after that.

I'll just run that query by hand for you. :)

(but I see what you are saying)

at page 300/4750 - listens seems to have gone up by ~300. so it's definitely not duping all of them. either there used to be a bug where before the import was missing 1 listen on every page, or there are a handful here and there that aren't getting deduped

does the dedupe logic run per import page, or per listen insert?

(400 pages. increase seems to stay at about 1 per page. i stupidly did not screenshot the count before the import started, so can't tell exactly.)

the logic that deduped the production data set for test is different than the dedup logic for normal incoming listens.

the deduper from influx actively seeks out the weirdest cases of the last.fm data.

going forward it will accept listens that are unique on track_name, listened_at and user_name

and now I realize that anyone who ever repeats a last.fm import will just make a hash of this again.

Looks like a problem with BrainzBot. Not printing the titles of tickets any longer. Something happen with the recent server maintenance?

Testing... MBS-9009

!m BrainzBot staying quiet

You're doing good work, BrainzBot staying quiet!

:p

page 1400; ~1100 new listens, so it's dipped below 1/page

ruaok: import finished. 100 listens not imported (expected 237,484 - imported 237,384). listen count went from ~242,500 to 242,931 - interestingly it had previously risen to above 243,000 during the import

i can certainly live with ~400 possible dupes out of 240k

What includes (?inc=) to a WS call should I use to retrieve a specific release with the track titles and track artists? Everything I've tried returns the recording title but not the track title.

> i can certainly live with ~400 possible dupes out of 240k

hellz to the yes!

thanks for checking it out. :)

this is working though

I think it's because I asked you to reindex ES...

but this is wokring https://bookbrainz.org/search?q=guy+debord

weird

project for this evening: https://github.com/mayhem/data-set-hoster

ARG how are you guys getting links with paramenters?!?!?!

I can't get one at all!