#metabrainz

/

      • alastairp
      • 2020-06-04 15610, 2020

      • alastairp
        is this what you mean?
      • 2020-06-04 15612, 2020

      • Mr_Monkey
        Yes, that
      • 2020-06-04 15624, 2020

      • alastairp
        yeah, so I don't see a lot of duplicated code here
      • 2020-06-04 15647, 2020

      • alastairp
        if the result of duplicating 2 lines of code is that the initial page load is faster, I'm super happy with that tradeoff
      • 2020-06-04 15625, 2020

      • alastairp
        would love to see how to get server-side rendering going from flask->react like musicbrainz does :)
      • 2020-06-04 15627, 2020

      • alastairp
        later later
      • 2020-06-04 15629, 2020

      • Mr_Monkey
        I guess one issue is that it's not usual to see query parameters in our APIs that way (/api/data?page=1 would most likely be /api/data/1), so you would have a bit more code to parse that
      • 2020-06-04 15622, 2020

      • alastairp
        to me that's basically an implementation detail, so I'm not too worried
      • 2020-06-04 15635, 2020

      • alastairp
        so, how much extra work is the frontend to load data from both places? (I guess this is another tradeoff we need to consider?)
      • 2020-06-04 15639, 2020

      • yvanzo
        bitmap: would it be alright to map IP hashes to work titles at random in redis?
      • 2020-06-04 15605, 2020

      • Mr_Monkey
        Not a lot of extra work.
      • 2020-06-04 15607, 2020

      • bitmap
        hm?
      • 2020-06-04 15644, 2020

      • alastairp
        that's cool
      • 2020-06-04 15603, 2020

      • yvanzo
        bitmap: or just disable IP hashes on test.mb.o.
      • 2020-06-04 15610, 2020

      • alastairp
        do you have time to play with json-schema? do you think it's worth looking at what it can give us?
      • 2020-06-04 15638, 2020

      • bitmap
        I'd just add a dbdefs setting to disable it
      • 2020-06-04 15652, 2020

      • Mr_Monkey
        I mean, you see it there in the jsx: if there's data passed in props by the server, blindly use that. The other part is calling an API to get more results, which we should be doing anyway for the sake of being in 2020 and not having to reload a react page to get page 2…
      • 2020-06-04 15601, 2020

      • Mr_Monkey
        alastairp: Not sure how json-schema fits in there, to be honest
      • 2020-06-04 15619, 2020

      • zas
        reosarevok: it was a general issue with musicbrainz, but we still have an issue with Picard 1.4, which will be removed from website soon, as we don't have a fix (SSL cert seems to be embedded in Qt4)
      • 2020-06-04 15624, 2020

      • alastairp
        I found stuff like https://www.npmjs.com/package/react-jsonschema-fo…, but that doesn't seem useful
      • 2020-06-04 15634, 2020

      • reosarevok
        CatQuest: are you on 1.4 ?
      • 2020-06-04 15642, 2020

      • Mr_Monkey
        Well, that's how I've seen it used, for data validation inside react.
      • 2020-06-04 15651, 2020

      • zas
        anyone on 1.4 has to upgrade to 2.x
      • 2020-06-04 15651, 2020

      • CatQuest
        reosarevok: 1.4.2
      • 2020-06-04 15656, 2020

      • CatQuest
        yea great.
      • 2020-06-04 15604, 2020

      • CatQuest
        make it possible and i will :/
      • 2020-06-04 15606, 2020

      • Mr_Monkey
        What we're talking about is more akin to defining the component's props
      • 2020-06-04 15624, 2020

      • alastairp
        right, but that package is more for creating a form and validating user input
      • 2020-06-04 15641, 2020

      • CatQuest
        2.x doesn't work on older macs
      • 2020-06-04 15641, 2020

      • Mr_Monkey
        Correct. Seems not adapted to me
      • 2020-06-04 15657, 2020

      • CatQuest
        god I'm so sick of the mac-older-version-desert :///
      • 2020-06-04 15622, 2020

      • CatQuest
        I wish i never got one and made my workflow dependant on some quirks
      • 2020-06-04 15624, 2020

      • Mr_Monkey
        There's various ways of defining props in react, like this: https://github.com/bookbrainz/bookbrainz-site/blo…
      • 2020-06-04 15638, 2020

      • Mr_Monkey
        The question is how do we export that to be used in flask, or vice-versa
      • 2020-06-04 15652, 2020

      • CatQuest
        it's not liek i cna afford a never mac at all!
      • 2020-06-04 15658, 2020

      • alastairp
        Mr_Monkey: yeah, one sec
      • 2020-06-04 15658, 2020

      • CatQuest
        eh. newer
      • 2020-06-04 15639, 2020

      • CatQuest
        sight. it has this "S.M.A.R.T." error failing thing on the hardrive too. so whatever happesn i probably *have* to get a new machine of osme kind soon
      • 2020-06-04 15650, 2020

      • CatQuest
        fuck.
      • 2020-06-04 15649, 2020

      • reosarevok
        bitmap: so I guess after this we should drop all the Perl/TT/JS code for these edits?
      • 2020-06-04 15600, 2020

      • bitmap
        yup
      • 2020-06-04 15622, 2020

      • reosarevok
        Ok, let's see if the DELETE FROM edit finishes
      • 2020-06-04 15634, 2020

      • reosarevok
        Or if we forgot any other FK
      • 2020-06-04 15646, 2020

      • Mr_Monkey
      • 2020-06-04 15618, 2020

      • Mr_Monkey
      • 2020-06-04 15651, 2020

      • CatQuest
      • 2020-06-04 15652, 2020

      • BrainzBot
        BB-486: Enable URL identifiers for Entities
      • 2020-06-04 15642, 2020

      • alastairp
      • 2020-06-04 15651, 2020

      • alastairp
        Mr_Monkey: endpoint for automatic schema generation
      • 2020-06-04 15657, 2020

      • Mr_Monkey
        There's an idea in there somewhere. Export jsonschema files from flask, import them and trnasform them in typescript types, use the type to implement the components.
      • 2020-06-04 15657, 2020

      • Mr_Monkey
        Seems like a big hassle for keeping props in sync, IMO.
      • 2020-06-04 15613, 2020

      • Mr_Monkey
        neat
      • 2020-06-04 15623, 2020

      • alastairp
        mmm, right. if we have to do a manual step to keep them in sync then that sucks
      • 2020-06-04 15657, 2020

      • Mr_Monkey
        Well, I'm not sure what we'd do with an endpoint like that, considering the hints we need are not at runtime but ideally integrated in our IDEs
      • 2020-06-04 15610, 2020

      • alastairp
        yeah, I just realised that
      • 2020-06-04 15614, 2020

      • Mr_Monkey
        It could possibly be semi-automatic
      • 2020-06-04 15622, 2020

      • alastairp
        and we could have tests
      • 2020-06-04 15642, 2020

      • alastairp
        that compare the python object->schema->props and js props
      • 2020-06-04 15613, 2020

      • Mr_Monkey
        Probably easier yes
      • 2020-06-04 15639, 2020

      • alastairp
        you know, I'm cautiously optimistic that this is a good idea
      • 2020-06-04 15658, 2020

      • blinky42 joined the channel
      • 2020-06-04 15614, 2020

      • alastairp
        if you set props in typescript like that example you linked, will it raise a runtime error if you perform an API call and the data doesn't match the expected format?
      • 2020-06-04 15658, 2020

      • Mr_Monkey
        > if you perform an API call
      • 2020-06-04 15658, 2020

      • Mr_Monkey
        That's at runtime…
      • 2020-06-04 15611, 2020

      • alastairp
        yeah, that's what I'm asking
      • 2020-06-04 15618, 2020

      • Mr_Monkey
        Typescript is not at runtime at all
      • 2020-06-04 15626, 2020

      • alastairp
        that is to say - what functionality are we giving ourselves here?
      • 2020-06-04 15630, 2020

      • alastairp
        autocomplete in the IDE?
      • 2020-06-04 15638, 2020

      • Mr_Monkey
        Yes
      • 2020-06-04 15642, 2020

      • Mr_Monkey
        Not much else
      • 2020-06-04 15646, 2020

      • alastairp
        mmm, right
      • 2020-06-04 15655, 2020

      • Mr_Monkey
        Well, validation and autocomplete in IDE
      • 2020-06-04 15610, 2020

      • alastairp
        because in js you could easily take a jsonschema and an object and validate that the object is correct, right?
      • 2020-06-04 15621, 2020

      • Mr_Monkey
        And, if your tests run a typescrit checker, you can fail that test if it implements it wrong
      • 2020-06-04 15633, 2020

      • Mr_Monkey
        Yes.
      • 2020-06-04 15610, 2020

      • alastairp
        but in that case we'd probably have to commit both the json-schema _and_ .ts to the repository, and then have the js frontend download the schema to the client in order to do the validation
      • 2020-06-04 15626, 2020

      • alastairp
        which is naff
      • 2020-06-04 15640, 2020

      • Mr_Monkey
        Agreed
      • 2020-06-04 15621, 2020

      • alastairp
        OK, give me a few minutes, I'm going to try and automatically extract schemas and run this js package
      • 2020-06-04 15617, 2020

      • BrainzGit
        [musicbrainz-server] reosarevok opened pull request #1551 (master…MBS-6864): MBS-6864: Drop all PUID-related code https://github.com/metabrainz/musicbrainz-server/…
      • 2020-06-04 15618, 2020

      • BrainzBot
      • 2020-06-04 15617, 2020

      • BrainzGit
        [bookbrainz-site] MonkeyDo opened pull request #440 (master…fix-reindex-issue): BB-464 fix: Reindex aliases properly on entity edit https://github.com/bookbrainz/bookbrainz-site/pul…
      • 2020-06-04 15618, 2020

      • BrainzBot
        BB-464: Search indexing issue https://tickets.metabrainz.org/browse/BB-464
      • 2020-06-04 15602, 2020

      • v6lur has quit
      • 2020-06-04 15636, 2020

      • shivam-kapila
        ruaok: The test just helped me discover time_range=-1 only works with `to_ts` and not with `from_ts`. Is it something desired?
      • 2020-06-04 15634, 2020

      • travis-ci joined the channel
      • 2020-06-04 15634, 2020

      • travis-ci
        Project bookbrainz-site build #3043: passed in 2 min 49 sec: https://travis-ci.org/bookbrainz/bookbrainz-site/…
      • 2020-06-04 15634, 2020

      • travis-ci has left the channel
      • 2020-06-04 15653, 2020

      • BrainzGit
        [bookbrainz-site] MonkeyDo merged pull request #440 (master…fix-reindex-issue): BB-464 fix: Reindex aliases properly on entity edit https://github.com/bookbrainz/bookbrainz-site/pul…
      • 2020-06-04 15653, 2020

      • BrainzBot
        BB-464: Search indexing issue https://tickets.metabrainz.org/browse/BB-464
      • 2020-06-04 15646, 2020

      • alastairp
        Mr_Monkey: update code, npm install, and from an active virtualenv run `python generate_schema.py`
      • 2020-06-04 15616, 2020

      • alastairp
        note that I altered the return type of the API, instead of a list, it's now an object: `{"data": [the list]}`
      • 2020-06-04 15623, 2020

      • BrainzGit
        [musicbrainz-server] yvanzo opened pull request #1552 (master…sanitize-ip-hash): Sanitize IP hashes on staging servers https://github.com/metabrainz/musicbrainz-server/…
      • 2020-06-04 15647, 2020

      • travis-ci joined the channel
      • 2020-06-04 15647, 2020

      • travis-ci
        Project bookbrainz-site build #3045: passed in 3 min 17 sec: https://travis-ci.org/bookbrainz/bookbrainz-site/…
      • 2020-06-04 15647, 2020

      • travis-ci has left the channel
      • 2020-06-04 15602, 2020

      • Mr_Monkey
        alastairp: schema looks good. I'll have a look at how to integrate that
      • 2020-06-04 15624, 2020

      • alastairp
        I also want to see how python documentation gets rendered to the jsonschema file by pydantic
      • 2020-06-04 15634, 2020

      • alastairp
        and if that turns into documentation in the ts file
      • 2020-06-04 15607, 2020

      • BrainzGit
        [musicbrainz-server] reosarevok merged pull request #1550 (master…MBS-10836): MBS-10836: Fix edit / remove collection links https://github.com/metabrainz/musicbrainz-server/…
      • 2020-06-04 15608, 2020

      • BrainzBot
        MBS-10836: My Collections edit link doesn't show edit page https://tickets.metabrainz.org/browse/MBS-10836
      • 2020-06-04 15621, 2020

      • Mr_Monkey
        So the gist of it then is we can use typescript, which will definitely help solve our 2 main goals (IDE integration and fail test)
      • 2020-06-04 15629, 2020

      • alastairp
      • 2020-06-04 15605, 2020

      • alastairp
        yes, I would love to see IDE integration for the values of fields in API calls!
      • 2020-06-04 15633, 2020

      • alastairp
        how do you see the `export type Id = number` lines in that schema?
      • 2020-06-04 15652, 2020

      • alastairp
        doesn't it seem a bit annoying that it can't just do `id: number` directly in the interface?
      • 2020-06-04 15613, 2020

      • Mr_Monkey
        I guess it is, but not the biggest drawback
      • 2020-06-04 15625, 2020

      • alastairp
        ah, this is due to pydantic
      • 2020-06-04 15644, 2020

      • alastairp
        it generates jsonschema `"properties": {"id": {"title": "Id", "description": "the id of the data", "type": "integer"}`
      • 2020-06-04 15604, 2020

      • alastairp
        and then json-schema->ts takes the title as the export type
      • 2020-06-04 15635, 2020

      • Mr_Monkey
        Could you export a type with these properties ? That way we have typing for the expected properties in the IDE. https://github.com/alastair/flask-react-playgroun…
      • 2020-06-04 15636, 2020

      • alastairp
        I pushed an update to include field documentation in the python model definitions
      • 2020-06-04 15612, 2020

      • alastairp
        do you mean in python or in js?
      • 2020-06-04 15646, 2020

      • Mr_Monkey
        in python, the same way you generate the DataResponse
      • 2020-06-04 15603, 2020

      • alastairp
      • 2020-06-04 15613, 2020

      • alastairp
        which has added type hints to the data functions
      • 2020-06-04 15653, 2020

      • alastairp
        and this will give you autocompletion in python, if you want to go `items.<tab>` in pycharm
      • 2020-06-04 15622, 2020

      • alastairp
      • 2020-06-04 15644, 2020

      • Mr_Monkey
        Neat. It's working well in typescript too
      • 2020-06-04 15650, 2020

      • alastairp
        :o
      • 2020-06-04 15658, 2020

      • alastairp
        technology working? I don't believe it
      • 2020-06-04 15613, 2020

      • alastairp
        let's wait for iliekcomputers' critical eye
      • 2020-06-04 15640, 2020

      • ruaok
        happened to me too yesterday. #donttrustit
      • 2020-06-04 15653, 2020

      • alastairp
        Mr_Monkey: going back to the first thing we addressed, do you want to try any alternative data loading processes? or are we happy with this system?
      • 2020-06-04 15628, 2020

      • Mr_Monkey
        Nope, I think that's also solved the right way
      • 2020-06-04 15645, 2020

      • alastairp
        I'd like to see your ts schema integration when you can ;)
      • 2020-06-04 15600, 2020

      • Mr_Monkey
      • 2020-06-04 15643, 2020

      • alastairp
        sweet
      • 2020-06-04 15645, 2020

      • alastairp
        IDE integration?
      • 2020-06-04 15621, 2020

      • Mr_Monkey
        working well too
      • 2020-06-04 15649, 2020

      • Mr_Monkey
        I'd say that's a success!
      • 2020-06-04 15626, 2020

      • alastairp
        now we just have to use it!
      • 2020-06-04 15637, 2020

      • Mr_Monkey
        Hah
      • 2020-06-04 15633, 2020

      • alastairp
        I think we can make a unittest / integration test in python to verify if ts schema files are up to date
      • 2020-06-04 15658, 2020

      • alastairp
        I don't know if it makes sense to have a bot that will actually commit changes if necessary, or if failing tests is enough
      • 2020-06-04 15609, 2020

      • Mr_Monkey
        Or regenerate them all the time
      • 2020-06-04 15626, 2020

      • Mr_Monkey
        Why not commit definition files every time they change?
      • 2020-06-04 15629, 2020

      • alastairp
        we can have a manage.py command to do this, that's easy
      • 2020-06-04 15644, 2020

      • alastairp
        right - it's more if someone changes a python model and forgets to update the ts file
      • 2020-06-04 15650, 2020

      • alastairp
        how do we catch that?
      • 2020-06-04 15655, 2020

      • Mr_Monkey
        Hm, I see
      • 2020-06-04 15647, 2020

      • alastairp
        this is why I suggested a test. this is a good fit for a simple unittest that can run on PRs
      • 2020-06-04 15600, 2020

      • alastairp
        either travis, or jenkins, or gh actions
      • 2020-06-04 15617, 2020

      • alastairp
        so we can see "check failed" on a PR
      • 2020-06-04 15624, 2020

      • alastairp
        as we use PRs for everything, I think this is enough
      • 2020-06-04 15617, 2020

      • Mr_Monkey
        Yes
      • 2020-06-04 15635, 2020

      • Mr_Monkey
        Regenerate the typescript files and compared to what is committed?
      • 2020-06-04 15637, 2020

      • diru1100
        IRCloud app is glitching a lot these days https://usercontent.irccloud-cdn.com/file/noDcdee…
      • 2020-06-04 15654, 2020

      • Mr_Monkey
        OK, just pushed typescript integration if you want to check