#metabrainz

/

      • MRiddickW joined the channel
      • rdswift has quit
      • rdswift joined the channel
      • rdswift has quit
      • radioman joined the channel
      • rdswift joined the channel
      • radioman
        What's the best way to get MSID_MBID mappings?
      • Gotta set up listenbrainz_spark + musicbrainz_db + typesense (+ ..??) and run the mbid_mapping.py routine?
      • Or is there a table/dump somewhere with recent official output?
      • _lucifer
        radioman, there is a ftp dump available for mbid msid mapping.
      • ruaok would be able to better help you but he wont be available for another few hours.
      • radioman
        Thanks _lucifer. The last dump is from June 2020 and from what I could see in the history a lot has changed since then. Appreciate the help.
      • davic joined the channel
      • adhi001 joined the channel
      • Nishant-Chandla joined the channel
      • Nishant-Chandla9 joined the channel
      • Nishant-Chandla has quit
      • Nishant-Chandla9 has quit
      • Nishant-Chandla joined the channel
      • Nishant-Chandla has quit
      • culinko joined the channel
      • BharatKalluri joined the channel
      • BharatKalluri has quit
      • BharatKalluri joined the channel
      • BharatKalluri
        There are huge data dumps open library provides of books (https://openlibrary.org/developers/dumps). Can we import all that data into bookbrainz? (I'm sorry if this has been discussed earlier. Could not find any such issue on Jira)
      • Mineo has quit
      • antlarr has quit
      • antlarr joined the channel
      • BrainzGit
        [bookbrainz-site] BharatKalluri opened pull request #583 (master…fix/bug_blocking_profile_edit): fix(profile): Fix bug where profile page does not load due to undefined area https://github.com/bookbrainz/bookbrainz-site/p...
      • [bookbrainz-site] BharatKalluri opened pull request #584 (master…fix/added_editor_tabs_in_user_dropdown): feat(profile): Added revisions, achievements and collections in user dropdown (BB-599) https://github.com/bookbrainz/bookbrainz-site/p...
      • Mr_Monkey
        BharatKalluri: Yes indeed, that's a goal for BookBrainz, but we haven't fully developed the tools for that yet.
      • BharatKalluri
        I see, Is there anything I can do to help there?
      • I recently discovered bookbrainz after learning that goodreads API's have been discontinued. The project is exactly what I've been looking for!  Thanks for all the hard work :)
      • Is there any particular reason bookbrainz does not entirely use typescript? Looks like typescript is a dev dep and some files in entity-editor/*/* are typescript and some in javascript.  Is the idea to eventually migrate to typescript for both client and server?
      • Mr_Monkey
        There was an unfinished GSoC project from a few years ago that set out to import datasets. If you wanted to work on that feature it might be a good starting point. Be advised it is very much outdated and the codebase has moved on, but it could serve as a good base.
      • Yes, typescript conversion is in progress
      • BharatKalluri
        Awesome, I'm a full time typescript developer. Also code in python and golang.
      • Let me know what tickets are high priority and something I can pickup. :]
      • Mr_Monkey
        Sorry, I'm not in front of my computer for a couple of days but do look on our ticket tracker: https://tickets.metabrainz.org/browse/BB
      • alastairp
        radioman: hi, what are you trying to do? Just look up a few msids, or do a large-scale analysis/lookup task?
      • You can use this, which we're now using for lookups: https://labs.api.listenbrainz.org/mbid-mapping
      • as a matter of fact, last week on Friday we just started a bulk lookup process, so we might have a dump soon
      • morning _lucifer
      • _lucifer
        moin!!
      • yvanzo
        good start of the week
      • c1e0 joined the channel
      • ruaok
        moin moin!
      • radioman
        alastairp: looking to do analysis against the full set to the extent it's mapped. (i'm aware the mappings are inherently fuzzy and the mapping routine itself seems to be under active development.) Don't wanna slam the API, but thanks for sharing that. Any idea when that dump would complete?
      • alastairp
        radioman: ruaok will be able to tell you how far processed it is
      • _lucifer
        alastairp, ruaok, iliekcomputers: why do we use @api_login_required instead of @login_required? https://github.com/metabrainz/listenbrainz-serv...
      • alastairp
        _lucifer: at a guess it's so that it returns json in the case that you're not logged in, instead of redirecting you to the login screen
      • this endpoing is an api service for the music player
      • ruaok
        _lucifer: may also be that it doesn't redirect to https, if requested from http.
      • radioman: if you're looking for the full mbid mapping, I'm guessing that is going to take weeks to complete.
      • _lucifer
        ah ok, makes sense. thanks!
      • radioman
        yowsa, thanks for heads up
      • ruaok
        the lookup server I am using ran out of resources very quickly and needs some tuning before I restart it.
      • shivam-kapila
        It raised APIUnauthorised and returns a json _lucifer
      • ruaok
        radioman: my immediate goal is the get incoming listens mapped and having 1 month into the past is going to make a lot of difference for us.
      • alastairp
      • radioman
        ruaok: so you're adding MBIDs to listens that lack them on the fly? nice. is setting up the mapping stuff well-formed or just kinda hacked on your side for now? i got listenbrainz-server and musicbrainz-server going (tho listenbrainz history is a beast already as you know). also needs typesense + ?? + maybe some magic that only exists at MeB HQ for now? Kinda sounds best to wait but if
      • it's reasonable to set up maybe we can divide and conquer, lmk
      • ruaok
        not sure what you mean by well formed. So far its is geared to run on our infrastructure inside docker, but there is nothing preventing you from running it on your own system, however you choose to host it. we have no real magic sauce at MeB HQ that we hold back, that's not how we roll.
      • the idea is that I am creating a new DB table that acts like a mapping and a history of the mapper, meaning that if I attempt to match a track, I record the result, its quality and when the match attempt was made.
      • this gives me the visibility to see what is good/poor and the learn from the poor/no matches and improve it for future passes.
      • my most immediate goal is to get current lookups working (having 1 month of fully matched listens will help our recommendation stuff). and then to match back in time, with a goal of 2 years clean matches at first.
      • as far as divide and conquer, I suppose I could use help in improving the matching script over time. but first we need to gather stats on what doesn't or gets poorly matched.
      • this is the core of the mapping search: https://github.com/metabrainz/listenbrainz-serv...
      • radioman
        makes sense, just figured it was so bleeding edge it might be hard to follow. will read thru that stuff more closely tomorrow, thanks.
      • ruaok
        it is, but it might also stabilize fairly quickly, since this is the second attempt at doing this. the first version was thrown away in proper engineering style.
      • radioman
        i know that move well. thanks for the lead, nai nai from here
      • ruaok
        ok, looking forward to chatting more.
      • alastairp: 👍 to that gist and +10 for naming them tasks. Looks good to me!
      • alastairp
        ruaok: great, I'll push out a new BU PR this morning, and hopefully we can get LB released in the afternoon
      • ruaok
        great.
      • alastairp
        we'll need a few redis fields to store some mapping data, but nothing difficult
      • I guess we should have started with the data format spec originally 😅
      • loujine_ is now known as loujine
      • CatQuest
        hommik!
      • c1e0 has quit
      • c1e0 joined the channel
      • iliekcomputers
        good morning!
      • BrainzGit
        [musicbrainz-server] yvanzo merged pull request #1983 (master…mbs-11442): MBS-11442: Adding tags on add-cover-art page disables submit https://github.com/metabrainz/musicbrainz-serve...
      • ruaok
        Mr_Monkey: "In LB API, we have a MAX_TIME_RANGE a user can request listens for (365 days). Does that mean there's no way for a user to view their listens older than a year?"
      • that means that one single request cannot fetch more than 1 year. if more data is needed, we need to make a second request.
      • lorenzuru
        Folks I'm struggling a little while trying to properly handle timezones in a LB scrobbling script
      • ruaok
        hi lorenzuru!
      • lack of timezones was the biggest oversight when we first wrote LB. :(
      • its something we've been meaning to add, but haven't gotten there yet.
      • lorenzuru
        Hi ruaok, is this strictly for MetaBrainz project dev or I can ask about personal project too?
      • ruaok
        ask away, who knows!
      • CatQuest
        BharatKalluri bharatkalluri[m]: I'm delighted you've decided to work on BB-495, it really needs doing (infact I feel a bit that it sohuld be priority "high") whne working with this, please feel free to ask me to test things (it's what I'm here for)
      • BrainzBot
      • lorenzuru
        Right now it's 12:28 here in CET, I read the docs and as far as I can tell there is no way to show timestamp on the LB site, right?
      • CatQuest
        lorenzuru: like a "current time" like e.g. wikipedia does?
      • ruaok
        lorenzuru: correct. our failure to add that when we wrote LB. :(
      • CatQuest
        ruaok: it's a bizarre oversight to, considering that we come for mall over :/ so typical
      • come from all over*
      • ruaok
        lorenzuru: we suggest that you submit timestamps for your local timezone. before too long we're going to add support for submitting timezones as well as adding a "user's home timezone" so that we can give context to older listens without timezones.
      • lorenzuru
        If I remember properly last.fm accepts UNIX timestamps and then it offset the timestamps with the timezone selected in the user settings
      • ruaok
        lorenzuru: yes, we plan to add that.
      • BharatKalluri
        CatQuest Thanks!
      • I would really appreciate if you could create those two subtasks in jira for now :] Looks like the title issue is a client side change and the author issue is a server side fix. If there are multiple sub tasks, I can post notes and corresponding MR's there.
      • lorenzuru
        <ruaok "Lorenzo: we suggest that you sub"> The problem is that if I try to send a UNIX timestamp of my localtime what I get is a UNIX timestamp *from the future* (in Italy we are UTC+2) and that is not allowed as far as I can tell
      • At least, I'm getting some BAD REQUEST by doing that so I inferred it's not allowed
      • CatQuest
        BharatKalluri: what are MR's?
      • (and yea the reason i hadn't made tickets was that I was unusre if my idea of sepration was correct)
      • ruaok
        oh. crap. right. ok, submit UTC then.
      • BharatKalluri
        CatQuest
      • MR's are merge requests, same thing as PR's (Pull requests on github)
      • > (and yea the reason i hadn't made tickets was that I was unusre if my idea of sepration was correct)
      • ah I see. cool (y)
      • CatQuest
        oh I'm used to the "pr" terminology :D
      • y?
      • lorenzuru
        * Right now it's 12:28 here in CET, I read the docs and as far as I can tell there is no way to show time-zoned timestamp on the LB site, right?
      • MRiddickW has quit
      • ruaok
        correct.
      • CatQuest
        yea. always my submissions are "one hour ago" :D
      • lorenzuru
        Okay, I'll stick with UTC for the meantime
      • Thank you for your time folks
      • ruaok
        np
      • CatQuest
        oh. heck. it's comming like christmaseve on the wife. but this week in 1st april
      • do we have anything planned?
      • TOPIC: MetaBrainz Community and Development channel | MusicBrainz non-development: #musicbrainz | Channel is logged; see https://musicbrainz.org/doc/IRC for details | Agenda: Reviews, weblate (outsidecontext/zas) | 1st of April!
      • TOPIC: MetaBrainz Community and Development channel | MusicBrainz non-development: #musicbrainz | Channel is logged; see https://musicbrainz.org/doc/IRC for details | Agenda: Reviews, weblate (outsidecontext/zas) | 1st of April! (CatCat)
      • BharatKalluri has quit
      • ruaok
      • I am getting a shared memory error using the DB on williams and it appears the shared segment line didn't get copied. was that intentional?
      • shivam-kapila
        I have two tables `tickets(id, valid_upto, user_id)` and `used_tickets(user_id, ticket_id)`. I want to to fetch list of `tickets(ticket_no, valid_upto, used)` for a given user where `used` is true if the `ticket_no` is in `used_tickets`. Is it possible to get in 1 query. Anone?
      • yvanzo
        reosarevok: There is now a fair amount of your PRs that can be merged for today’s beta update. :)
      • reosarevok
        Yay, thanks. Birding rn but will do that around meeting time
      • BrainzGit
        [metabrainz.org] mayhem merged pull request #357 (master…patch-7): MEB-119: Update Google's donation amount https://github.com/metabrainz/metabrainz.org/pu...
      • Retr0Punk is now known as RetroPunk
      • ruaok
      • I tried building an update of meb.org but it fails with this less command now. :(
      • _lucifer
        ruaok, strange i didn;t get that error last time i built the image. i'll downgrade lessc for now and if that works open a PR.
      • ruaok
        me neither -- it just happened. :(
      • c1e0 has quit
      • c1e0 joined the channel
      • adhi001 has quit
      • CatQuest
        reosarevok: umean "birbing" (ok no)
      • alastairp
      • I'm looking at how to store the counters in redis
      • in order to have 1) hostnames and 2) all counters on a single endpoint we need to store some additional data in redis (lines 7-9 in gist). does this look sane?
      • BrainzGit
        [metabrainz.org] amCap1712 opened pull request #358 (master…downgrade): Downgrade lessc to v3 https://github.com/metabrainz/metabrainz.org/pu...
      • alastairp
        lmk if something is unclear
      • _lucifer
        iiuc, each container stores its metrics in redis and then web serves it on a endpoint?
      • alastairp
        correct