#metabrainz

/

      • travis-ci joined the channel
      • travis-ci
        Project bookbrainz-data-js build #1233: passed in 2 min 5 sec: https://travis-ci.org/bookbrainz/bookbrainz-dat...
      • travis-ci has left the channel
      • chaban has quit
      • Lotheric_ joined the channel
      • Lotheric has quit
      • Lotheric__ joined the channel
      • Lotheric_ has quit
      • d4rkie joined the channel
      • D4RK-PH0ENiX has quit
      • kori has quit
      • oknozor joined the channel
      • d4rkie has quit
      • D4RK-PH0ENiX joined the channel
      • D4RK-PH0ENiX has quit
      • D4RK-PH0ENiX joined the channel
      • kori joined the channel
      • CatQuest
        prell!
      • oknozor has quit
      • ruaok
        mooooin!
      • reosarevok
        hommik!
      • BrainzGit
        [listenbrainz-labs] mayhem merged pull request #48 (master…del-zip): Remove zip created on every run of spark-submit https://github.com/metabrainz/listenbrainz-labs...
      • [listenbrainz-labs] mayhem merged pull request #47 (master…del-redundant-files): Remove redundant files from codebase https://github.com/metabrainz/listenbrainz-labs...
      • [musicbrainz-server] reosarevok opened pull request #1173 (master…MBS-10299): MBS-10299: Add voting reports page https://github.com/metabrainz/musicbrainz-serve...
      • BrainzBot
        MBS-10299: “Voting Reports” (pre-set edit searches) https://tickets.metabrainz.org/browse/MBS-10299
      • reosarevok
        Freso: ^ wouldn't mind some checking of that
      • BrainzGit
        [listenbrainz-labs] mayhem merged pull request #46 (master…use-path-in-create-df): Import paths from path.py in create_dataframes https://github.com/metabrainz/listenbrainz-labs...
      • Gazooo has quit
      • Gazooo joined the channel
      • zas
        yvanzo: any idea abiut what happened at https://stats.metabrainz.org/d/000000061/mbstat...
      • ?
      • hmmm kiki's cpu temperature increased by 15°C at 1AM UTC... I guess hetzner still has crappy cpu fans in stock...
      • ruaok
        I bet. sigh. haven't we replaced all of them.. yet? at least once?
      • zas
        It seems replacement parts are no better... but I think that's the first time on kiki though
      • I'll switch to herb and create a ticket for hetzner techs to have a look
      • btw, it's a bit messy, since we have few things running on kiki that don't run on herb (git2consul, and various mb stats)
      • I'll open a ticket at hetzner
      • aidanlw17
        hi alastairp, are you around to talk?
      • zas
        the issue with traffic starting at 8:16 UTC is prolly due to a network issue between hetzner cloud (solr cloud) and other machines. MB website behavior when this happens isn't great, causing very long delays:
      • D4RK-PH0ENiX has quit
      • D4RK-PH0ENiX joined the channel
      • ruaok: did you see the Google mail about "New pricing for external IPs for Google Compute Engine VMs and Cloud NAT from January 1, 2020" ?
      • travis-ci joined the channel
      • travis-ci
        metabrainz/picard#4820 (master - c03f384 : Philipp Wolfer): The build passed.
      • travis-ci has left the channel
      • D4RK-PH0ENiX has quit
      • D4RK-PH0ENiX joined the channel
      • alastairp
        aidanlw17: hi, I'm here
      • aidanlw17
        alastairp: great, since this is the last week of gsoc I just wanted to touch base about what I need to finish up etc.
      • ruaok
        Zas: skimmed it.
      • aidanlw17
        alastairp: the eval will be ready for review today at some point. For the rest of the week, I'm not sure that I should start a new large task
      • although I feel that there is still a _ton_ of work that can done on the similarity project
      • alastairp
        absolutly, I agree
      • I think we should focus on integration this week
      • what was it that you mentioned yesterday about mistakes in similarity?
      • Lotheric__ is now known as Lotheric
      • aidanlw17
        ok cool, that's what I was thinking too
      • Mr_Monkey
        akhilesh: Just pushed some code regarding browse request query params validation. I opted for one utility for all entity types. I will now write down an exhaustive list of possible relationships between entities (in short, the array passed to `validateBrowseRequestQueryParameters` for each entity, and a reason for each.
      • As for other query params: I'm thinking about how to validate other filter params. Not sure how to go about it yet. It will probably involve passing a function used as a filter in `getBrowsedRelationships`.
      • `limit` and `count` are straightforward: if they are present, use their value, if not use a default.
      • Any other query params will be ignored (for example mistyped or non-existing ones)
      • alastairp
        with the changes that you made to the original PR, did you rebase the others with those changes?
      • aidanlw17
        alastairp: https://github.com/metabrainz/acousticbrainz-se... I mentioned the mistake briefly here
      • basically, when I changed the metrics classes to use the names of highlevel models instead of the ids, I realized an issue
      • The query that I used to collect the batch of recordings and their lowlevel data and highlevel models altogether wasn't working properly
      • in that it actually was only selecting one highlevel model with each row that contained a full lowlevel document
      • So we were missing some data and frequently, not all metrics could be computed (multiple highlevel metrics had vectors like {0, ..., 0}
      • alastairp
        ah, right
      • is there a test that we can include to check this?
      • aidanlw17
        yeah I haven’t added a test for that, but I will. Good idea
      • because of this, on bono we should recompute the metrics when we’re done with these changes
      • reosarevok rolls his eyes at https://tickets.metabrainz.org/browse/MBS-10325
      • also, I haven’t rebased them with these changes because I thought we might make more changes first
      • alastairp
        yeah, sure
      • I just came up with a realisation regarding the rebasing -
      • we have this annoying situation where we have to rewrite the history in order to delete a file from the repository
      • if I do this with the first PR then I don't think the subsequent one will apply properly
      • but what we can do is rewrite the final PR (which has _all_ commits in it)
      • and then recreate the branches of the intermediate PRs from this branch
      • I know how to do it, we just need to make sure that all branches are up-to-date with any rebasing that we might need to do
      • aidanlw17
        Hm okay so I'll be sure to get the other branches rebased
      • The other problem we're going to have with that is what we talked about a couple weeks ago
      • There are changes made on the third and fourth PRs that should really have happened on the second and third PRs, respectively
      • alastairp
        sure. I'm not too worried about that - I think that we'll apply the changes pretty quickly in succession
      • aidanlw17
        Okay, so will we just review PR 4 and not review PRs 2 and 3?
      • alastairp
        you know, I think that might actually be the easiest thing to do
      • I've learned a bit about this process as well, I don't think I'll recommend that we do something similar the next time we do a project like this
      • aidanlw17
        It's a lot of work in one place, now
      • alastairp
        are you looking at the test now?
      • aidanlw17
        for the query in AB-412, you mean?
      • BrainzBot
        AB-412: Compute recording similarity metrics on database https://tickets.metabrainz.org/browse/AB-412
      • alastairp
        yes, the one we just talked about
      • aidanlw17
        yes I am
      • alastairp
        nice
      • aidanlw17
        Currently I'm just submitting two recordings, calling add_metrics, and checking the calls that are made to submit_similarity_by_id to ensure that the data is all included
      • For next time, do you have thoughts on what would have been a better structure (for the PRs)?
      • alastairp
        I'd like to also see a test for the sql statement that gets this data
      • I think the better way of doing PRs would be to not base one off another
      • so always make a new one from master
      • but perhaps this means that we should be a bit more proactive in merging earlier work, so that it can be used in subsequent branches
      • aidanlw17
        How can we write a test specific for the sql statement, should I make the same query in a test and check what is fetched against what is expected?
      • makes sense, I think that would work
      • especially if the work was split into smaller PRs
      • alastairp
        there's a method in db/data.py that gets the data? and this is the query that you fixed?
      • aidanlw17
        so they were easier to review and merge quickly
      • its the query in db/similarity.py in the function add_metrics
      • alastairp
        ahh. I see
      • what we could do is move this specific query out into a new method
      • so then we'd have a method that just does a db query and returns results
      • then this `add_metrics` can call that method
      • this means that we can test each component individually, and then add_metrics just brings all of those components together
      • aidanlw17
        sure, and separate components will make it more clear too
      • I'll do that
      • alastairp
        great, thanks!
      • aidanlw17
        Should I still make a test for add_metrics to check the logic of calling everything, too?
      • alastairp
        sure, we could have one test with a minimal amount of data to check that everything works end-to-end
      • but I'd focus on the individual methods more
      • aidanlw17
        okay, sounds good.
      • some q's about evals: for work submission this week, I read https://developers.google.com/open-source/gsoc/... and thought blog post sounded like the best way to detail this work
      • alastairp
        yes, we typically do blog posts
      • aidanlw17
        along with some changes to the readme, and the similarity docs as well
      • cool. I'm gonna start that today, is there anything specific you'd like to see in it?
      • alastairp
        no, that's completely up to you. what you did, why we did it, things you learned, things that surprised you, what it's like working with metabrainz
      • aidanlw17
        perfect! I'm excited to write and reflect about it
      • GeneralDiscourse is now known as ephemer0l
      • chaban joined the channel
      • Mr_Monkey
        akhilesh: I also now pushed some code stub showing how to use the browse request filters for `/author`. You'll need to use the same mechanism for each endpoint.
      • travis-ci joined the channel
      • travis-ci
        Project bookbrainz-data-js build #1234: passed in 2 min 0 sec: https://travis-ci.org/bookbrainz/bookbrainz-dat...
      • travis-ci has left the channel
      • akhilesh
        Mr_Monkey: ok, thanks! I will update today.
      • BrainzGit
        [musicbrainz-server] reosarevok opened pull request #1174 (master…MBS-10324): MBS-10324: Make "takes" ETI (since "take" is) https://github.com/metabrainz/musicbrainz-serve...
      • BrainzBot
        MBS-10324: Lowercase "Takes" with guess case https://tickets.metabrainz.org/browse/MBS-10324
      • Gore joined the channel
      • Gore has quit
      • Cyna
        bitmap: will you be available tomorrow, i.e 24 hrs from now ?
      • I have a lot of query piled up, I'm almost through with convertions of edit forms.
      • will spend my day time tomorrow converting the remaining 2 entity editforms. then I'll ask about all the bugs tomorrow night (guess itll be morning for you 🤔)
      • travis-ci joined the channel
      • travis-ci
        metabrainz/picard#4822 (master - 88df4e0 : Philipp Wolfer): The build passed.
      • travis-ci has left the channel
      • pristine__
        > if it's low overhead then I'd just do it every time because you don't want to have a situation where you run a 3 hour job and then it fails just because you forgot to run a different command first
      • ruaok: I guess alastairp is right here.