#metabrainz

/

      • legoktm has quit

        • 2021-05-05 12538, 2021

      • legoktm joined the channel

        • 2021-05-05 12511, 2021

      • D4RK joined the channel

        • 2021-05-05 12509, 2021

      • d4rkie has quit

        • 2021-05-05 12501, 2021

      • thomasross has quit

        • 2021-05-05 12541, 2021

      • D4RK has quit

        • 2021-05-05 12517, 2021

      • d4rkie joined the channel

        • 2021-05-05 12544, 2021

      • _lucifer
        ruaok: alastairp: i was going through the time ranges PR and went down a rabbit hole of timescale versions. i was looking into upgrading it to the latest patch version. however, i found that we are using the `timescale/timescaledb:latest-pg11` in production. on inspecting it, I found it is on Timescale 2.2.0 version. I think yesterday when the container was restarted the image got updated inadvertently.

        • 2021-05-05 12528, 2021

      • shivam-kapila
        _lucifer: do we have materialised views in prod?

        • 2021-05-05 12537, 2021

      • _lucifer
        I do not recall any discussion or work on moving to timescale 2.*. We are using `timescale/timescaledb:1.7.4-pg12` in development and should probably be using the same in production or maybe upgrade to `timescale/timescaledb:1.7.5-pg12`.

        • 2021-05-05 12559, 2021

      • _lucifer
        shivam-kapila: probably, yes.

        • 2021-05-05 12554, 2021

      • _lucifer
        also, any particular reason we use pg11 image for timescale when our postgres version is 12.

        • 2021-05-05 12509, 2021

      • shivam-kapila
        LB-777

        • 2021-05-05 12510, 2021

      • BrainzBot
        LB-777: Pin the timescale release to latest 1.7.x version https://tickets.metabrainz.org/browse/LB-777

        • 2021-05-05 12533, 2021

      • shivam-kapila
        Something related though not totally

        • 2021-05-05 12549, 2021

      • _lucifer
        yeah, this ticker was for local development.

        • 2021-05-05 12553, 2021

      • shivam-kapila
        But if 2.2 didnt break anything in prod chances are that the dev can also be updated and versions can be synced

        • 2021-05-05 12508, 2021

      • _lucifer
        i do not think that is the case. also, it could be that something is wrong and we haven't just discovered it yet.

        • 2021-05-05 12533, 2021

      • shivam-kapila
        Totally Possible

        • 2021-05-05 12504, 2021

      • _lucifer
        another thing is that during local development we create the database and on timescale 2 it used to error out at the db creation point.

        • 2021-05-05 12550, 2021

      • _lucifer
        but for prod the db already existed. however, other differences in behaviour and errors might be more subtle to catch.

        • 2021-05-05 12535, 2021

      • shivam-kapila
        Good point. Probably we should visit the changelogs

        • 2021-05-05 12538, 2021

      • _lucifer
        https://www.irccloud.com/pastebin/l1PXsMNV/

        • 2021-05-05 12554, 2021

      • BrainzGit
        [listenbrainz-server] amCap1712 opened pull request #1426 (master…timescale-1.7.5): Update timescale image used in local development https://github.com/metabrainz/listenbrainz-server…

        • 2021-05-05 12506, 2021

      • sampsyo has quit

        • 2021-05-05 12514, 2021

      • sampsyo joined the channel

        • 2021-05-05 12541, 2021

      • ruaok
        _lucifer: oh wow. good catch.

        • 2021-05-05 12548, 2021

      • ruaok
        lucky lucky us.

        • 2021-05-05 12519, 2021

      • ruaok
        except I don't think our instance was migrated correctly then.

        • 2021-05-05 12501, 2021

      • ruaok
        there is an upgrade command that needs to be issued as the first command after startup in a new version and we didn't do that.

        • 2021-05-05 12510, 2021

      • ruaok
        unless what we upgraded to didn't need that.

        • 2021-05-05 12539, 2021

      • ruaok
        if we did upgrade, then I will want to change some TS queries in the timescale-giant-PR

        • 2021-05-05 12558, 2021

      • _lucifer
        yeah, i certainly expect the time ranges pr to break with timescale 2.

        • 2021-05-05 12540, 2021

      • _lucifer
        do we want to be on timescale 2 or revert to 1.7 till we have explored the changes?

        • 2021-05-05 12500, 2021

      • ruaok
        i dont think the PR will break.

        • 2021-05-05 12505, 2021

      • ruaok
        lets stay on 2.2

        • 2021-05-05 12517, 2021

      • ruaok
        can you sanity check me on what needs to happen post upgrade?

        • 2021-05-05 12536, 2021

      • _lucifer
        sure, these are instructions to upgrade. https://docs.timescale.com/timescaledb/latest/how… going through those currently.

        • 2021-05-05 12559, 2021

      • _lucifer
        ok these ones for docker, https://docs.timescale.com/timescaledb/latest/how…

        • 2021-05-05 12511, 2021

      • _lucifer
        i think we only need to do the last step

        • 2021-05-05 12526, 2021

      • _lucifer
        https://www.irccloud.com/pastebin/yGThBenX/

        • 2021-05-05 12535, 2021

      • _lucifer
        let's run that?

        • 2021-05-05 12500, 2021

      • _lucifer
        also, another thing is we are running pg11 image for timescale. shouldn't it be pg12 image instead?

        • 2021-05-05 12546, 2021

      • ruaok
        I think the alter extension needs to happen as the first command after startup, no?

        • 2021-05-05 12556, 2021

      • _lucifer
        yes

        • 2021-05-05 12507, 2021

      • ruaok
        and yes pg12 would be ideal, but lets not do that right now.

        • 2021-05-05 12518, 2021

      • ruaok
        lets not make whatever we're doing worse.

        • 2021-05-05 12526, 2021

      • _lucifer
        agreed.

        • 2021-05-05 12543, 2021

      • ruaok
        I need to pack up right now and head back to BCN to the office.

        • 2021-05-05 12501, 2021

      • BrainzGit
        [bookbrainz-site] akashgp09 opened pull request #620 (master…search-entity-with-identifiers): feat(elasticSearch): search entities by identifiers https://github.com/bookbrainz/bookbrainz-site/pul…

        • 2021-05-05 12542, 2021

      • ruaok
        once I am there, lets take a few minutes to run this command. (by taking listenstore & TSW offline, restarting TS, runnning update, restarting) at least we have the infrastructure to do this now.

        • 2021-05-05 12543, 2021

      • _lucifer
        ok, so let's do this once you are at the office?

        • 2021-05-05 12554, 2021

      • ruaok
        yes. a few hours from now.

        • 2021-05-05 12502, 2021

      • _lucifer
        sure.

        • 2021-05-05 12528, 2021

      • ruaok
        _lucifer: I think we should go to 2.2.0-pg11 in both prod and dev.

        • 2021-05-05 12556, 2021

      • _lucifer
        ruaok: i agree, i am testing and updating the dev changes currently.

        • 2021-05-05 12503, 2021

      • ruaok
        from the looks of it, we're only missing the word MATERIALIZED from the create view functions.

        • 2021-05-05 12543, 2021

      • _lucifer
        there are some changes in the aggregation policy stuff as well but i am not sure whether those affect us.

        • 2021-05-05 12523, 2021

      • ruaok
        we dont have any. but there are other changes not mentioned in that article. options have gone and views cannot be created in a transaction any longer.

        • 2021-05-05 12531, 2021

      • ruaok
        I'll have to spend more time on it.

        • 2021-05-05 12529, 2021

      • BrainzGit
        [musicbrainz-server] reosarevok opened pull request #2089 (master…MBS-11637): MBS-11637: Show event-work rels on work page https://github.com/metabrainz/musicbrainz-server/…

        • 2021-05-05 12535, 2021

      • BrainzGit
        [listenbrainz-server] MonkeyDo merged pull request #1402 (master…monkey-refactor-API-service): Implement global application context (API service injection & global app props) https://github.com/metabrainz/listenbrainz-server…

        • 2021-05-05 12516, 2021

      • BrainzGit
        [listenbrainz-server] amCap1712 merged pull request #1426 (be-gone-time-ranges…timescale-1.7.5): Update timescale image used in local development https://github.com/metabrainz/listenbrainz-server…

        • 2021-05-05 12512, 2021

      • BrainzGit
        [bookbrainz-site] MonkeyDo merged pull request #616 (master…search-collection): [BB-619]: Search results should not contain private collections https://github.com/bookbrainz/bookbrainz-site/pul…

        • 2021-05-05 12523, 2021

      • ruaok loves it when a new bottle of mezcal appears in his office

        • 2021-05-05 12527, 2021

      • travis-ci joined the channel

        • 2021-05-05 12527, 2021

      • travis-ci
        Project bookbrainz-site build #3935: passed in 4 min 32 sec: https://travis-ci.org/bookbrainz/bookbrainz-site/…

        • 2021-05-05 12527, 2021

      • travis-ci has left the channel

        • 2021-05-05 12538, 2021

      • _lucifer
        ruaok, why is listened_at column bigint and not timestamp? i ask because elsewhere we use timestamps for dates.

        • 2021-05-05 12557, 2021

      • ruaok
        its not supported.

        • 2021-05-05 12550, 2021

      • ruaok
        also as per your comment, using 30 days or better yet 1 month, doesn't work. or at least it didn't work in 1.7

        • 2021-05-05 12534, 2021

      • _lucifer
        oh ok. the examples all use timestamps https://docs.timescale.com/timescaledb/latest/how… . probably was added later?

        • 2021-05-05 12530, 2021

      • ruaok
        possibly.

        • 2021-05-05 12538, 2021

      • ruaok is getting sick of working on this.

        • 2021-05-05 12530, 2021

      • _lucifer
        yeah, the PR is open for a long time. let's merge and deploy once the timescale 2 bits are done and alastairp has reviewed.

        • 2021-05-05 12550, 2021

      • ruaok
        I would actually simply call this: "good enough". I'll try to see if "1 month" is supported now. if not, I'm leaving it.

        • 2021-05-05 12511, 2021

      • _lucifer
        no, 1 month isn't supported due to reasons.

        • 2021-05-05 12538, 2021

      • ruaok
        reasons I can understand. thats another can of worms.

        • 2021-05-05 12500, 2021

      • _lucifer
        yup, https://github.com/timescale/timescaledb/issues/4…

        • 2021-05-05 12557, 2021

      • _lucifer
        using timestamps instead of bigint might be useful on its own but that's for a later day anyway.

        • 2021-05-05 12522, 2021

      • ruaok
        what new features does that bring?

        • 2021-05-05 12558, 2021

      • _lucifer
        i don't think much but I would think that makes handling timezones simpler?

        • 2021-05-05 12529, 2021

      • ruaok
        good enough for now.

        • 2021-05-05 12525, 2021

      • ruaok
        _lucifer: should we try and work on the timescale restart/upgrade?

        • 2021-05-05 12536, 2021

      • _lucifer
        yeah, sure.

        • 2021-05-05 12525, 2021

      • ruaok
        \dx timescaledb

        • 2021-05-05 12527, 2021

      • _lucifer
        how do we go about it? bring down containers, bring down timescale, bring up containers, upgrade timescale?

        • 2021-05-05 12530, 2021

      • ruaok
        still shows 1.7.1

        • 2021-05-05 12539, 2021

      • ruaok
        so, clearly it needs doing.

        • 2021-05-05 12510, 2021

      • _lucifer
        ok, wait a sec. we might have to do 1.7.1 to 1.7.4 first. let me check the docs

        • 2021-05-05 12510, 2021

      • ruaok
        well, technically taking down the containers is no longer needed, but I think it is better if we do it.

        • 2021-05-05 12522, 2021

      • _lucifer
        https://docs.timescale.com/timescaledb/latest/how…

        • 2021-05-05 12500, 2021

      • _lucifer
        the docker one says 1.7.4 to 2 whereas without docker doesn't. what do you suggest?

        • 2021-05-05 12518, 2021

      • ruaok
        I think going straight to 2.2. should work fine.

        • 2021-05-05 12535, 2021

      • _lucifer
        cool, let's go ahead with that.

        • 2021-05-05 12548, 2021

      • _lucifer
        do you want to tweet downtime first?

        • 2021-05-05 12517, 2021

      • ruaok
        https://gist.github.com/mayhem/9307ce4c5c8593a2fd…

        • 2021-05-05 12526, 2021

      • ruaok
        check that please. I'll tweet.

        • 2021-05-05 12539, 2021

      • _lucifer
        👍

        • 2021-05-05 12547, 2021

      • ruaok
        do you want to drive lemmy, I drive gaga?

        • 2021-05-05 12526, 2021

      • _lucifer
        yes the steps look correct to me.

        • 2021-05-05 12533, 2021

      • _lucifer
        sure, i'll take lemmy

        • 2021-05-05 12543, 2021

      • ruaok
        please prepare the image in step 0 then.

        • 2021-05-05 12552, 2021

      • _lucifer
        on it

        • 2021-05-05 12518, 2021

      • ruaok
        tests for be-gone-time-ranges fixed.

        • 2021-05-05 12526, 2021

      • ruaok
        lets see what GH actions thinks about that.

        • 2021-05-05 12508, 2021

      • _lucifer
        the current master already has the strings blanked out. https://github.com/metabrainz/listenbrainz-server…

        • 2021-05-05 12525, 2021

      • _lucifer
        are any other changes needed or should i just build the master?

        • 2021-05-05 12534, 2021

      • ruaok
        I don't think we need other changes.

        • 2021-05-05 12539, 2021

      • _lucifer
        👍

        • 2021-05-05 12545, 2021

      • ruaok
        build it and then we revert that change later.

        • 2021-05-05 12515, 2021

      • _lucifer
        image building.

        • 2021-05-05 12501, 2021

      • _lucifer
        regarding the actions, i see there are merge conflict. those need to be resolved for the actions to run.

        • 2021-05-05 12506, 2021

      • ruaok
        Mr_Monkey: we've got javascript merge conflicts on this PR https://github.com/metabrainz/listenbrainz-server… -- Could you please resolve those?

        • 2021-05-05 12519, 2021

      • Mr_Monkey
        Sure thing

        • 2021-05-05 12517, 2021

      • BrainzGit
        [bookbrainz-site] akashgp09 opened pull request #621 (master…duplicate-suggestions): BB-432: move duplicate entity warning below sort name https://github.com/bookbrainz/bookbrainz-site/pul…

        • 2021-05-05 12524, 2021

      • _lucifer
        https://www.irccloud.com/pastebin/ECd00uNj/

        • 2021-05-05 12536, 2021

      • _lucifer
        Mr_Monkey: building the latest master fails with the above error.

        • 2021-05-05 12547, 2021

      • _lucifer
        I am building again with one commit before it.

        • 2021-05-05 12553, 2021

      • ruaok
        oy

        • 2021-05-05 12538, 2021

      • Mr_Monkey
        Let me look at that

        • 2021-05-05 12528, 2021

      • alastairp
        I'm here

        • 2021-05-05 12532, 2021

      • alastairp
        hi

        • 2021-05-05 12518, 2021

      • _lucifer
        hi alastairp! we are going to repeat yesterday's exercise to upgrade timescale.

        • 2021-05-05 12523, 2021

      • ruaok
        _lucifer: alastairp and I reviewed the gist and made some minor tweaks at the end. please refresh.

        • 2021-05-05 12530, 2021

      • _lucifer
        on it

        • 2021-05-05 12545, 2021

      • ruaok
        _lucifer: I clued him in, he's reviewed the gist. we're ready when you are.

        • 2021-05-05 12523, 2021

      • _lucifer
        image pushed. let's begin!

        • 2021-05-05 12534, 2021

      • Mr_Monkey
        _lucifer: I'm able to build the image locally using `./develop.sh build web`. Let me try with push.sh see if I get the same result.

        • 2021-05-05 12559, 2021

      • ruaok
        _lucifer: ok, sounds good. start with step 1.

        • 2021-05-05 12518, 2021

      • ruaok
        I'll tweet. let me know when step 3 is complete.

        • 2021-05-05 12532, 2021

      • _lucifer
        👍

        • 2021-05-05 12557, 2021

      • _lucifer
        done

        • 2021-05-05 12508, 2021

      • ruaok
        step 4 in progress

        • 2021-05-05 12521, 2021

      • ruaok
        timescale down.

        • 2021-05-05 12524, 2021

      • ruaok
        proceed with 5

        • 2021-05-05 12525, 2021

      • ruaok
        https://gist.github.com/mayhem/4c53cffc468af3fa47…

        • 2021-05-05 12526, 2021

      • ruaok
        sweet!

        • 2021-05-05 12538, 2021

      • alastairp
        that was easy

        • 2021-05-05 12502, 2021

      • ruaok
        basic query works.

        • 2021-05-05 12520, 2021

      • ruaok
        _lucifer: proced with step 8

        • 2021-05-05 12538, 2021

      • _lucifer
        👍

        • 2021-05-05 12533, 2021

      • _lucifer
        done

        • 2021-05-05 12544, 2021

      • ruaok
        listens page loads fine.

        • 2021-05-05 12548, 2021

      • ruaok
        feed page loads fine.

        • 2021-05-05 12507, 2021

      • ruaok
        playlists load fine.

        • 2021-05-05 12516, 2021

      • ruaok
        I say we proceed to finish.

        • 2021-05-05 12533, 2021

      • _lucifer
        yup

        • 2021-05-05 12547, 2021

      • Mr_Monkey
        _lucifer: I reproduced the typescript compilation issue. Odd that it would work in some image builds and not others. Will investigate, thanks !

        • 2021-05-05 12533, 2021

      • _lucifer
        ruaok: timescale writer and cron up.