#metabrainz

/

      • travis-ci joined the channel
      • 2019-08-20 23227, 2019

      • travis-ci
        Project bookbrainz-data-js build #1233: passed in 2 min 5 sec: https://travis-ci.org/bookbrainz/bookbrainz-data-…
      • 2019-08-20 23227, 2019

      • travis-ci has left the channel
      • 2019-08-20 23219, 2019

      • chaban has quit
      • 2019-08-20 23259, 2019

      • Lotheric_ joined the channel
      • 2019-08-20 23219, 2019

      • Lotheric has quit
      • 2019-08-20 23257, 2019

      • Lotheric__ joined the channel
      • 2019-08-20 23226, 2019

      • Lotheric_ has quit
      • 2019-08-20 23258, 2019

      • d4rkie joined the channel
      • 2019-08-20 23234, 2019

      • D4RK-PH0ENiX has quit
      • 2019-08-20 23222, 2019

      • kori has quit
      • 2019-08-20 23234, 2019

      • oknozor joined the channel
      • 2019-08-20 23210, 2019

      • d4rkie has quit
      • 2019-08-20 23222, 2019

      • D4RK-PH0ENiX joined the channel
      • 2019-08-20 23230, 2019

      • D4RK-PH0ENiX has quit
      • 2019-08-20 23207, 2019

      • D4RK-PH0ENiX joined the channel
      • 2019-08-20 23217, 2019

      • kori joined the channel
      • 2019-08-20 23202, 2019

      • CatQuest
        prell!
      • 2019-08-20 23247, 2019

      • oknozor has quit
      • 2019-08-20 23214, 2019

      • ruaok
        mooooin!
      • 2019-08-20 23246, 2019

      • reosarevok
        hommik!
      • 2019-08-20 23230, 2019

      • BrainzGit
        [listenbrainz-labs] mayhem merged pull request #48 (master…del-zip): Remove zip created on every run of spark-submit https://github.com/metabrainz/listenbrainz-labs/p…
      • 2019-08-20 23220, 2019

      • BrainzGit
        [listenbrainz-labs] mayhem merged pull request #47 (master…del-redundant-files): Remove redundant files from codebase https://github.com/metabrainz/listenbrainz-labs/p…
      • 2019-08-20 23236, 2019

      • BrainzGit
        [musicbrainz-server] reosarevok opened pull request #1173 (master…MBS-10299): MBS-10299: Add voting reports page https://github.com/metabrainz/musicbrainz-server/…
      • 2019-08-20 23237, 2019

      • BrainzBot
        MBS-10299: “Voting Reports” (pre-set edit searches) https://tickets.metabrainz.org/browse/MBS-10299
      • 2019-08-20 23200, 2019

      • reosarevok
        Freso: ^ wouldn't mind some checking of that
      • 2019-08-20 23250, 2019

      • BrainzGit
        [listenbrainz-labs] mayhem merged pull request #46 (master…use-path-in-create-df): Import paths from path.py in create_dataframes https://github.com/metabrainz/listenbrainz-labs/p…
      • 2019-08-20 23201, 2019

      • Gazooo has quit
      • 2019-08-20 23246, 2019

      • Gazooo joined the channel
      • 2019-08-20 23221, 2019

      • zas
        yvanzo: any idea abiut what happened at https://stats.metabrainz.org/d/000000061/mbstats?…
      • 2019-08-20 23224, 2019

      • zas
        ?
      • 2019-08-20 23254, 2019

      • zas
        hmmm kiki's cpu temperature increased by 15°C at 1AM UTC... I guess hetzner still has crappy cpu fans in stock...
      • 2019-08-20 23205, 2019

      • ruaok
        I bet. sigh. haven't we replaced all of them.. yet? at least once?
      • 2019-08-20 23206, 2019

      • zas
        It seems replacement parts are no better... but I think that's the first time on kiki though
      • 2019-08-20 23244, 2019

      • zas
        I'll switch to herb and create a ticket for hetzner techs to have a look
      • 2019-08-20 23229, 2019

      • zas
        btw, it's a bit messy, since we have few things running on kiki that don't run on herb (git2consul, and various mb stats)
      • 2019-08-20 23245, 2019

      • zas
        I'll open a ticket at hetzner
      • 2019-08-20 23207, 2019

      • aidanlw17
        hi alastairp, are you around to talk?
      • 2019-08-20 23213, 2019

      • zas
        the issue with traffic starting at 8:16 UTC is prolly due to a network issue between hetzner cloud (solr cloud) and other machines. MB website behavior when this happens isn't great, causing very long delays:
      • 2019-08-20 23213, 2019

      • zas
      • 2019-08-20 23244, 2019

      • D4RK-PH0ENiX has quit
      • 2019-08-20 23240, 2019

      • D4RK-PH0ENiX joined the channel
      • 2019-08-20 23202, 2019

      • zas
        ruaok: did you see the Google mail about "New pricing for external IPs for Google Compute Engine VMs and Cloud NAT from January 1, 2020" ?
      • 2019-08-20 23237, 2019

      • travis-ci joined the channel
      • 2019-08-20 23237, 2019

      • travis-ci
        metabrainz/picard#4820 (master - c03f384 : Philipp Wolfer): The build passed.
      • 2019-08-20 23237, 2019

      • travis-ci
      • 2019-08-20 23237, 2019

      • travis-ci
      • 2019-08-20 23237, 2019

      • travis-ci has left the channel
      • 2019-08-20 23226, 2019

      • D4RK-PH0ENiX has quit
      • 2019-08-20 23235, 2019

      • D4RK-PH0ENiX joined the channel
      • 2019-08-20 23237, 2019

      • alastairp
        aidanlw17: hi, I'm here
      • 2019-08-20 23258, 2019

      • aidanlw17
        alastairp: great, since this is the last week of gsoc I just wanted to touch base about what I need to finish up etc.
      • 2019-08-20 23246, 2019

      • ruaok
        Zas: skimmed it.
      • 2019-08-20 23228, 2019

      • aidanlw17
        alastairp: the eval will be ready for review today at some point. For the rest of the week, I'm not sure that I should start a new large task
      • 2019-08-20 23208, 2019

      • aidanlw17
        although I feel that there is still a _ton_ of work that can done on the similarity project
      • 2019-08-20 23243, 2019

      • alastairp
        absolutly, I agree
      • 2019-08-20 23202, 2019

      • alastairp
        I think we should focus on integration this week
      • 2019-08-20 23220, 2019

      • alastairp
        what was it that you mentioned yesterday about mistakes in similarity?
      • 2019-08-20 23254, 2019

      • Lotheric__ is now known as Lotheric
      • 2019-08-20 23243, 2019

      • aidanlw17
        ok cool, that's what I was thinking too
      • 2019-08-20 23252, 2019

      • Mr_Monkey
        akhilesh: Just pushed some code regarding browse request query params validation. I opted for one utility for all entity types. I will now write down an exhaustive list of possible relationships between entities (in short, the array passed to `validateBrowseRequestQueryParameters` for each entity, and a reason for each.
      • 2019-08-20 23252, 2019

      • Mr_Monkey
        As for other query params: I'm thinking about how to validate other filter params. Not sure how to go about it yet. It will probably involve passing a function used as a filter in `getBrowsedRelationships`.
      • 2019-08-20 23252, 2019

      • Mr_Monkey
        `limit` and `count` are straightforward: if they are present, use their value, if not use a default.
      • 2019-08-20 23252, 2019

      • Mr_Monkey
        Any other query params will be ignored (for example mistyped or non-existing ones)
      • 2019-08-20 23204, 2019

      • alastairp
        with the changes that you made to the original PR, did you rebase the others with those changes?
      • 2019-08-20 23256, 2019

      • aidanlw17
        alastairp: https://github.com/metabrainz/acousticbrainz-serv… I mentioned the mistake briefly here
      • 2019-08-20 23201, 2019

      • aidanlw17
        basically, when I changed the metrics classes to use the names of highlevel models instead of the ids, I realized an issue
      • 2019-08-20 23253, 2019

      • aidanlw17
        The query that I used to collect the batch of recordings and their lowlevel data and highlevel models altogether wasn't working properly
      • 2019-08-20 23223, 2019

      • aidanlw17
        in that it actually was only selecting one highlevel model with each row that contained a full lowlevel document
      • 2019-08-20 23218, 2019

      • aidanlw17
        So we were missing some data and frequently, not all metrics could be computed (multiple highlevel metrics had vectors like {0, ..., 0}
      • 2019-08-20 23226, 2019

      • alastairp
        ah, right
      • 2019-08-20 23239, 2019

      • alastairp
        is there a test that we can include to check this?
      • 2019-08-20 23237, 2019

      • aidanlw17
        yeah I haven’t added a test for that, but I will. Good idea
      • 2019-08-20 23230, 2019

      • aidanlw17
        because of this, on bono we should recompute the metrics when we’re done with these changes
      • 2019-08-20 23227, 2019

      • reosarevok rolls his eyes at https://tickets.metabrainz.org/browse/MBS-10325
      • 2019-08-20 23250, 2019

      • aidanlw17
        also, I haven’t rebased them with these changes because I thought we might make more changes first
      • 2019-08-20 23207, 2019

      • alastairp
        yeah, sure
      • 2019-08-20 23218, 2019

      • alastairp
        I just came up with a realisation regarding the rebasing -
      • 2019-08-20 23240, 2019

      • alastairp
        we have this annoying situation where we have to rewrite the history in order to delete a file from the repository
      • 2019-08-20 23208, 2019

      • alastairp
        if I do this with the first PR then I don't think the subsequent one will apply properly
      • 2019-08-20 23224, 2019

      • alastairp
        but what we can do is rewrite the final PR (which has _all_ commits in it)
      • 2019-08-20 23246, 2019

      • alastairp
        and then recreate the branches of the intermediate PRs from this branch
      • 2019-08-20 23207, 2019

      • alastairp
        I know how to do it, we just need to make sure that all branches are up-to-date with any rebasing that we might need to do
      • 2019-08-20 23234, 2019

      • aidanlw17
        Hm okay so I'll be sure to get the other branches rebased
      • 2019-08-20 23255, 2019

      • aidanlw17
        The other problem we're going to have with that is what we talked about a couple weeks ago
      • 2019-08-20 23207, 2019

      • aidanlw17
        There are changes made on the third and fourth PRs that should really have happened on the second and third PRs, respectively
      • 2019-08-20 23255, 2019

      • alastairp
        sure. I'm not too worried about that - I think that we'll apply the changes pretty quickly in succession
      • 2019-08-20 23211, 2019

      • aidanlw17
        Okay, so will we just review PR 4 and not review PRs 2 and 3?
      • 2019-08-20 23234, 2019

      • alastairp
        you know, I think that might actually be the easiest thing to do
      • 2019-08-20 23211, 2019

      • alastairp
        I've learned a bit about this process as well, I don't think I'll recommend that we do something similar the next time we do a project like this
      • 2019-08-20 23250, 2019

      • aidanlw17
        It's a lot of work in one place, now
      • 2019-08-20 23219, 2019

      • alastairp
        are you looking at the test now?
      • 2019-08-20 23200, 2019

      • aidanlw17
        for the query in AB-412, you mean?
      • 2019-08-20 23202, 2019

      • BrainzBot
        AB-412: Compute recording similarity metrics on database https://tickets.metabrainz.org/browse/AB-412
      • 2019-08-20 23213, 2019

      • alastairp
        yes, the one we just talked about
      • 2019-08-20 23223, 2019

      • aidanlw17
        yes I am
      • 2019-08-20 23235, 2019

      • alastairp
        nice
      • 2019-08-20 23233, 2019

      • aidanlw17
        Currently I'm just submitting two recordings, calling add_metrics, and checking the calls that are made to submit_similarity_by_id to ensure that the data is all included
      • 2019-08-20 23211, 2019

      • aidanlw17
        For next time, do you have thoughts on what would have been a better structure (for the PRs)?
      • 2019-08-20 23231, 2019

      • alastairp
        I'd like to also see a test for the sql statement that gets this data
      • 2019-08-20 23259, 2019

      • alastairp
        I think the better way of doing PRs would be to not base one off another
      • 2019-08-20 23209, 2019

      • alastairp
        so always make a new one from master
      • 2019-08-20 23237, 2019

      • alastairp
        but perhaps this means that we should be a bit more proactive in merging earlier work, so that it can be used in subsequent branches
      • 2019-08-20 23211, 2019

      • aidanlw17
        How can we write a test specific for the sql statement, should I make the same query in a test and check what is fetched against what is expected?
      • 2019-08-20 23253, 2019

      • aidanlw17
        makes sense, I think that would work
      • 2019-08-20 23211, 2019

      • aidanlw17
        especially if the work was split into smaller PRs
      • 2019-08-20 23231, 2019

      • alastairp
        there's a method in db/data.py that gets the data? and this is the query that you fixed?
      • 2019-08-20 23239, 2019

      • aidanlw17
        so they were easier to review and merge quickly
      • 2019-08-20 23213, 2019

      • aidanlw17
      • 2019-08-20 23226, 2019

      • aidanlw17
      • 2019-08-20 23250, 2019

      • aidanlw17
        its the query in db/similarity.py in the function add_metrics
      • 2019-08-20 23228, 2019

      • alastairp
        ahh. I see
      • 2019-08-20 23251, 2019

      • alastairp
        what we could do is move this specific query out into a new method
      • 2019-08-20 23205, 2019

      • alastairp
        so then we'd have a method that just does a db query and returns results
      • 2019-08-20 23214, 2019

      • alastairp
        then this `add_metrics` can call that method
      • 2019-08-20 23237, 2019

      • alastairp
        this means that we can test each component individually, and then add_metrics just brings all of those components together
      • 2019-08-20 23231, 2019

      • aidanlw17
        sure, and separate components will make it more clear too
      • 2019-08-20 23235, 2019

      • aidanlw17
        I'll do that
      • 2019-08-20 23242, 2019

      • alastairp
        great, thanks!
      • 2019-08-20 23206, 2019

      • aidanlw17
        Should I still make a test for add_metrics to check the logic of calling everything, too?
      • 2019-08-20 23238, 2019

      • alastairp
        sure, we could have one test with a minimal amount of data to check that everything works end-to-end
      • 2019-08-20 23248, 2019

      • alastairp
        but I'd focus on the individual methods more
      • 2019-08-20 23217, 2019

      • aidanlw17
        okay, sounds good.
      • 2019-08-20 23216, 2019

      • aidanlw17
        some q's about evals: for work submission this week, I read https://developers.google.com/open-source/gsoc/he… and thought blog post sounded like the best way to detail this work
      • 2019-08-20 23233, 2019

      • alastairp
        yes, we typically do blog posts
      • 2019-08-20 23255, 2019

      • aidanlw17
        along with some changes to the readme, and the similarity docs as well
      • 2019-08-20 23227, 2019

      • aidanlw17
        cool. I'm gonna start that today, is there anything specific you'd like to see in it?
      • 2019-08-20 23220, 2019

      • alastairp
        no, that's completely up to you. what you did, why we did it, things you learned, things that surprised you, what it's like working with metabrainz
      • 2019-08-20 23214, 2019

      • aidanlw17
        perfect! I'm excited to write and reflect about it
      • 2019-08-20 23222, 2019

      • GeneralDiscourse is now known as ephemer0l
      • 2019-08-20 23253, 2019

      • chaban joined the channel
      • 2019-08-20 23201, 2019

      • Mr_Monkey
        akhilesh: I also now pushed some code stub showing how to use the browse request filters for `/author`. You'll need to use the same mechanism for each endpoint.
      • 2019-08-20 23229, 2019

      • travis-ci joined the channel
      • 2019-08-20 23229, 2019

      • travis-ci
        Project bookbrainz-data-js build #1234: passed in 2 min 0 sec: https://travis-ci.org/bookbrainz/bookbrainz-data-…
      • 2019-08-20 23229, 2019

      • travis-ci has left the channel
      • 2019-08-20 23225, 2019

      • akhilesh
        Mr_Monkey: ok, thanks! I will update today.
      • 2019-08-20 23247, 2019

      • BrainzGit
        [musicbrainz-server] reosarevok opened pull request #1174 (master…MBS-10324): MBS-10324: Make "takes" ETI (since "take" is) https://github.com/metabrainz/musicbrainz-server/…
      • 2019-08-20 23248, 2019

      • BrainzBot
        MBS-10324: Lowercase "Takes" with guess case https://tickets.metabrainz.org/browse/MBS-10324
      • 2019-08-20 23201, 2019

      • Gore joined the channel
      • 2019-08-20 23247, 2019

      • Gore has quit
      • 2019-08-20 23239, 2019

      • Cyna
        bitmap: will you be available tomorrow, i.e 24 hrs from now ?
      • 2019-08-20 23215, 2019

      • Cyna
        I have a lot of query piled up, I'm almost through with convertions of edit forms.
      • 2019-08-20 23258, 2019

      • Cyna
        will spend my day time tomorrow converting the remaining 2 entity editforms. then I'll ask about all the bugs tomorrow night (guess itll be morning for you 🤔)
      • 2019-08-20 23256, 2019

      • travis-ci joined the channel
      • 2019-08-20 23256, 2019

      • travis-ci
        metabrainz/picard#4822 (master - 88df4e0 : Philipp Wolfer): The build passed.
      • 2019-08-20 23256, 2019

      • travis-ci
      • 2019-08-20 23256, 2019

      • travis-ci
      • 2019-08-20 23256, 2019

      • travis-ci has left the channel
      • 2019-08-20 23251, 2019

      • pristine__
        > if it's low overhead then I'd just do it every time because you don't want to have a situation where you run a 3 hour job and then it fails just because you forgot to run a different command first
      • 2019-08-20 23209, 2019

      • pristine__
        ruaok: I guess alastairp is right here.