Thanks _lucifer. The last dump is from June 2020 and from what I could see in the history a lot has changed since then. Appreciate the help.
davic joined the channel
adhi001 joined the channel
Nishant-Chandla joined the channel
Nishant-Chandla9 joined the channel
Nishant-Chandla has quit
Nishant-Chandla9 has quit
Nishant-Chandla joined the channel
Nishant-Chandla has quit
culinko joined the channel
BharatKalluri joined the channel
BharatKalluri has quit
BharatKalluri joined the channel
BharatKalluri
There are huge data dumps open library provides of books (https://openlibrary.org/developers/dumps). Can we import all that data into bookbrainz? (I'm sorry if this has been discussed earlier. Could not find any such issue on Jira)
Mineo has quit
antlarr has quit
antlarr joined the channel
BrainzGit
[bookbrainz-site] BharatKalluri opened pull request #583 (master…fix/bug_blocking_profile_edit): fix(profile): Fix bug where profile page does not load due to undefined area https://github.com/bookbrainz/bookbrainz-site/p...
[bookbrainz-site] BharatKalluri opened pull request #584 (master…fix/added_editor_tabs_in_user_dropdown): feat(profile): Added revisions, achievements and collections in user dropdown (BB-599) https://github.com/bookbrainz/bookbrainz-site/p...
Mr_Monkey
BharatKalluri: Yes indeed, that's a goal for BookBrainz, but we haven't fully developed the tools for that yet.
BharatKalluri
I see, Is there anything I can do to help there?
I recently discovered bookbrainz after learning that goodreads API's have been discontinued. The project is exactly what I've been looking for! Thanks for all the hard work :)
Is there any particular reason bookbrainz does not entirely use typescript? Looks like typescript is a dev dep and some files in entity-editor/*/* are typescript and some in javascript. Is the idea to eventually migrate to typescript for both client and server?
Mr_Monkey
There was an unfinished GSoC project from a few years ago that set out to import datasets. If you wanted to work on that feature it might be a good starting point. Be advised it is very much outdated and the codebase has moved on, but it could serve as a good base.
Yes, typescript conversion is in progress
BharatKalluri
Awesome, I'm a full time typescript developer. Also code in python and golang.
Let me know what tickets are high priority and something I can pickup. :]
as a matter of fact, last week on Friday we just started a bulk lookup process, so we might have a dump soon
morning _lucifer
_lucifer
moin!!
yvanzo
good start of the week
c1e0 joined the channel
ruaok
moin moin!
radioman
alastairp: looking to do analysis against the full set to the extent it's mapped. (i'm aware the mappings are inherently fuzzy and the mapping routine itself seems to be under active development.) Don't wanna slam the API, but thanks for sharing that. Any idea when that dump would complete?
alastairp
radioman: ruaok will be able to tell you how far processed it is
ruaok: so you're adding MBIDs to listens that lack them on the fly? nice. is setting up the mapping stuff well-formed or just kinda hacked on your side for now? i got listenbrainz-server and musicbrainz-server going (tho listenbrainz history is a beast already as you know). also needs typesense + ?? + maybe some magic that only exists at MeB HQ for now? Kinda sounds best to wait but if
it's reasonable to set up maybe we can divide and conquer, lmk
ruaok
not sure what you mean by well formed. So far its is geared to run on our infrastructure inside docker, but there is nothing preventing you from running it on your own system, however you choose to host it. we have no real magic sauce at MeB HQ that we hold back, that's not how we roll.
the idea is that I am creating a new DB table that acts like a mapping and a history of the mapper, meaning that if I attempt to match a track, I record the result, its quality and when the match attempt was made.
this gives me the visibility to see what is good/poor and the learn from the poor/no matches and improve it for future passes.
my most immediate goal is to get current lookups working (having 1 month of fully matched listens will help our recommendation stuff). and then to match back in time, with a goal of 2 years clean matches at first.
as far as divide and conquer, I suppose I could use help in improving the matching script over time. but first we need to gather stats on what doesn't or gets poorly matched.
makes sense, just figured it was so bleeding edge it might be hard to follow. will read thru that stuff more closely tomorrow, thanks.
ruaok
it is, but it might also stabilize fairly quickly, since this is the second attempt at doing this. the first version was thrown away in proper engineering style.
radioman
i know that move well. thanks for the lead, nai nai from here
ruaok
ok, looking forward to chatting more.
alastairp: 👍 to that gist and +10 for naming them tasks. Looks good to me!
alastairp
ruaok: great, I'll push out a new BU PR this morning, and hopefully we can get LB released in the afternoon
ruaok
great.
alastairp
we'll need a few redis fields to store some mapping data, but nothing difficult
I guess we should have started with the data format spec originally 😅
Mr_Monkey: "In LB API, we have a MAX_TIME_RANGE a user can request listens for (365 days). Does that mean there's no way for a user to view their listens older than a year?"
that means that one single request cannot fetch more than 1 year. if more data is needed, we need to make a second request.
lorenzuru
Folks I'm struggling a little while trying to properly handle timezones in a LB scrobbling script
ruaok
hi lorenzuru!
lack of timezones was the biggest oversight when we first wrote LB. :(
its something we've been meaning to add, but haven't gotten there yet.
lorenzuru
Hi ruaok, is this strictly for MetaBrainz project dev or I can ask about personal project too?
ruaok
ask away, who knows!
CatQuest
BharatKalluri bharatkalluri[m]: I'm delighted you've decided to work on BB-495, it really needs doing (infact I feel a bit that it sohuld be priority "high") whne working with this, please feel free to ask me to test things (it's what I'm here for)
Right now it's 12:28 here in CET, I read the docs and as far as I can tell there is no way to show timestamp on the LB site, right?
CatQuest
lorenzuru: like a "current time" like e.g. wikipedia does?
ruaok
lorenzuru: correct. our failure to add that when we wrote LB. :(
CatQuest
ruaok: it's a bizarre oversight to, considering that we come for mall over :/ so typical
come from all over*
ruaok
lorenzuru: we suggest that you submit timestamps for your local timezone. before too long we're going to add support for submitting timezones as well as adding a "user's home timezone" so that we can give context to older listens without timezones.
lorenzuru
If I remember properly last.fm accepts UNIX timestamps and then it offset the timestamps with the timezone selected in the user settings
ruaok
lorenzuru: yes, we plan to add that.
BharatKalluri
CatQuest Thanks!
I would really appreciate if you could create those two subtasks in jira for now :] Looks like the title issue is a client side change and the author issue is a server side fix. If there are multiple sub tasks, I can post notes and corresponding MR's there.
lorenzuru
<ruaok "Lorenzo: we suggest that you sub"> The problem is that if I try to send a UNIX timestamp of my localtime what I get is a UNIX timestamp *from the future* (in Italy we are UTC+2) and that is not allowed as far as I can tell
At least, I'm getting some BAD REQUEST by doing that so I inferred it's not allowed
CatQuest
BharatKalluri: what are MR's?
(and yea the reason i hadn't made tickets was that I was unusre if my idea of sepration was correct)
ruaok
oh. crap. right. ok, submit UTC then.
BharatKalluri
CatQuest
MR's are merge requests, same thing as PR's (Pull requests on github)
> (and yea the reason i hadn't made tickets was that I was unusre if my idea of sepration was correct)
ah I see. cool (y)
CatQuest
oh I'm used to the "pr" terminology :D
y?
lorenzuru
* Right now it's 12:28 here in CET, I read the docs and as far as I can tell there is no way to show time-zoned timestamp on the LB site, right?
MRiddickW has quit
ruaok
correct.
CatQuest
yea. always my submissions are "one hour ago" :D
lorenzuru
Okay, I'll stick with UTC for the meantime
Thank you for your time folks
ruaok
np
CatQuest
oh. heck. it's comming like christmaseve on the wife. but this week in 1st april
do we have anything planned?
TOPIC: MetaBrainz Community and Development channel | MusicBrainz non-development: #musicbrainz | Channel is logged; see https://musicbrainz.org/doc/IRC for details | Agenda: Reviews, weblate (outsidecontext/zas) | 1st of April!
TOPIC: MetaBrainz Community and Development channel | MusicBrainz non-development: #musicbrainz | Channel is logged; see https://musicbrainz.org/doc/IRC for details | Agenda: Reviews, weblate (outsidecontext/zas) | 1st of April! (CatCat)
I am getting a shared memory error using the DB on williams and it appears the shared segment line didn't get copied. was that intentional?
shivam-kapila
I have two tables `tickets(id, valid_upto, user_id)` and `used_tickets(user_id, ticket_id)`. I want to to fetch list of `tickets(ticket_no, valid_upto, used)` for a given user where `used` is true if the `ticket_no` is in `used_tickets`. Is it possible to get in 1 query. Anone?
yvanzo
reosarevok: There is now a fair amount of your PRs that can be merged for today’s beta update. :)
reosarevok
Yay, thanks. Birding rn but will do that around meeting time
in order to have 1) hostnames and 2) all counters on a single endpoint we need to store some additional data in redis (lines 7-9 in gist). does this look sane?