#metabrainz

/

      • ruaok
        it will be a bit slower, but this query is run twice a week with nothing waiting on it.
      • 2020-11-04 30946, 2020

      • reosarevok
        bitmap: are you expecting to meet later today or does drinking and shuddering take priority?
      • 2020-11-04 30932, 2020

      • bitmap
        oh we could meet, I have plenty of time for that fun stuff
      • 2020-11-04 30941, 2020

      • bitmap
        ruaok: though an index might help in any case since PG should be able to use it to speed up sorting...might be worth creating one?
      • 2020-11-04 30917, 2020

      • ruaok
        not sure. how. its a complex query I am pulling data from, not another table.
      • 2020-11-04 30903, 2020

      • bitmap
        oh, I thought you meant you were doing the ORDER BY when selecting from the tmp table instead of relying on physical order
      • 2020-11-04 30935, 2020

      • ruaok
      • 2020-11-04 30945, 2020

      • ruaok
        is the query i am running.
      • 2020-11-04 30944, 2020

      • alastairp
        sorry, just had a call. back now
      • 2020-11-04 30907, 2020

      • bitmap
        yeah sorry, I didn't see you were ordering on a bunch of data outside the table
      • 2020-11-04 30919, 2020

      • ruaok
        and the sort order is very persnickety -- even the tiniest change and all hell breaks loose.
      • 2020-11-04 30927, 2020

      • ruaok
        no worries.
      • 2020-11-04 30942, 2020

      • ruaok
        I've tried creating a temp sequence, but that didn't work either.
      • 2020-11-04 30902, 2020

      • alastairp
        so issue is that even though the select has a sort, that's not the order that insert is working in?
      • 2020-11-04 30906, 2020

      • ruaok
        like I said, its not a lot of work to make python do the work -- that way we'll be guaranteed that the query runs right and doesn't break again.
      • 2020-11-04 30920, 2020

      • ruaok
        alastairp: I think that is a good way of putting it.
      • 2020-11-04 30928, 2020

      • alastairp
        you said that this was a "difference" between 9 and 12? Did you get as far as finding documentation that said that, or was it just an observation?
      • 2020-11-04 30957, 2020

      • ruaok
        no confirmation on that. I supposed I can go back and a look at paco to see how things are going there.
      • 2020-11-04 30913, 2020

      • ruaok
        but I tried to bisect the commits and the commits that FOR SURE worked, no longer work.
      • 2020-11-04 30935, 2020

      • ruaok
        and the only difference I am aware of is that we've gone from 9.5 (9.6?) to 12.x on bono.
      • 2020-11-04 30939, 2020

      • alastairp
        the commits of the mapping code?
      • 2020-11-04 30944, 2020

      • alastairp
        oh,
      • 2020-11-04 30945, 2020

      • ruaok
        correct.
      • 2020-11-04 30951, 2020

      • alastairp
        here's a super relevant question then
      • 2020-11-04 30956, 2020

      • alastairp
        is paco 9.5 or 9.6?
      • 2020-11-04 30902, 2020

      • ruaok
        and I was always a bit dubious that this would be stable over the long term.
      • 2020-11-04 30913, 2020

      • alastairp
        because prior to 9.6, this wasn't guaranteed in posgres: https://stackoverflow.com/questions/50809120/post…
      • 2020-11-04 30939, 2020

      • ruaok
        paco is 12.3
      • 2020-11-04 30952, 2020

      • alastairp
        OK, that's fine then
      • 2020-11-04 30913, 2020

      • ruaok
        I wonder if it is broken there too.
      • 2020-11-04 30916, 2020

      • ruaok
        not sure I care this much to dig further. it is clear that this is on the edge of undefined behaviour and changing the script to explicitly define the behaviour seems like the way to go.
      • 2020-11-04 30921, 2020

      • alastairp
        > In the past, Postgres was inconsistent in this area: you would get the desirable behavior if the ordering were performed via an indexscan, but not if it had to be done by an explicit sort step.
      • 2020-11-04 30924, 2020

      • alastairp
        yeah, that's interesting
      • 2020-11-04 30905, 2020

      • alastairp
        I think that you're right. either do it explicitly in python (if it can handle that much data), or your subquery of subquery idea
      • 2020-11-04 30928, 2020

      • ruaok
        I doubt the subquery of subquery will be stable either.
      • 2020-11-04 30949, 2020

      • ruaok
        and I am literally selecting 1 rowid for insertion into my table. zero problems for python.
      • 2020-11-04 30903, 2020

      • alastairp
        oh, I thought that this was acting on millions of rows. fine then
      • 2020-11-04 30937, 2020

      • ruaok
        yeah, its a few million, but the result is very skinny, so no biggie
      • 2020-11-04 30901, 2020

      • alastairp
        mmm
      • 2020-11-04 30905, 2020

      • alastairp
        I'm just trying to understand the query. It's returning all release ids, but in an order where the first occurrence of a release is probably the one that you want (because of media, date, etc)?
      • 2020-11-04 30924, 2020

      • alastairp
        and the table has columns (id, release), and you're inserting into release?
      • 2020-11-04 30949, 2020

      • ruaok
        yes to all.
      • 2020-11-04 30900, 2020

      • ruaok
        id is SERIAL
      • 2020-11-04 30904, 2020

      • alastairp
        and you tried explicitly inserting nextval of the id sequence as the id?
      • 2020-11-04 30912, 2020

      • alastairp
        what about using a window function as id?
      • 2020-11-04 30920, 2020

      • ruaok
        yes, it retains the incorrect sequence as it does without.
      • 2020-11-04 30944, 2020

      • alastairp
        so you no longer use the sequence, you just explicitly number all of the rows, and insert _that_ number as id
      • 2020-11-04 30947, 2020

      • ruaok
        window functions need a sort order to go by, but my sort order is quite nuts and I've not gotten that to work either.
      • 2020-11-04 30955, 2020

      • alastairp
        got it
      • 2020-11-04 30905, 2020

      • ruaok
        alastairp: that was my proposed fix, but no luck so far.
      • 2020-11-04 30914, 2020

      • alastairp
        so. use python then :)
      • 2020-11-04 30921, 2020

      • ruaok
        ding.
      • 2020-11-04 30906, 2020

      • v6lur has quit
      • 2020-11-04 30932, 2020

      • v6lur joined the channel
      • 2020-11-04 30951, 2020

      • alastairp
        ok. radiator is no longer dripping water, I think I'd call that a successful day
      • 2020-11-04 30920, 2020

      • v6lur has quit
      • 2020-11-04 30946, 2020

      • BrainzGit
        [musicbrainz-server] reosarevok merged pull request #1773 (master…update-AllowAmending-grace-period): Align comments with new code (very minor low priority PR) https://github.com/metabrainz/musicbrainz-server/…
      • 2020-11-04 30904, 2020

      • reosarevok
        bitmap, yvanzo: hi!
      • 2020-11-04 30911, 2020

      • BrainzGit
        [musicbrainz-server] mwiencek opened pull request #1776 (master…mbs-11092): Switch to SameSite=Strict for login cookies and preserve POST parameters on login (MBS-4555, MBS-11092) https://github.com/metabrainz/musicbrainz-server/…
      • 2020-11-04 30912, 2020

      • BrainzBot
        MBS-4555: Seeding the release editor when logged out loses the seeded information https://tickets.metabrainz.org/browse/MBS-4555
      • 2020-11-04 30912, 2020

      • BrainzBot
        MBS-11092: Some issue with userscripts/plugins and beta site https://tickets.metabrainz.org/browse/MBS-11092
      • 2020-11-04 30918, 2020

      • bitmap
        hey
      • 2020-11-04 30920, 2020

      • reosarevok
        React stuff: https://github.com/metabrainz/musicbrainz-server/… should be fixed again IIRC?
      • 2020-11-04 30923, 2020

      • reosarevok
        And https://github.com/metabrainz/musicbrainz-server/… is hopefully ready but it's a big thing and I expect we'll want to make changes, hopefully small
      • 2020-11-04 30924, 2020

      • yvanzo
        hello
      • 2020-11-04 30948, 2020

      • reosarevok
        (there's a few small PRs as usual too, but some are failing tests, so I need to figure those out)
      • 2020-11-04 30910, 2020

      • bitmap
        finally submitted the seeding/login stuff, just need to lookat test fixes
      • 2020-11-04 30900, 2020

      • reosarevok
        Yay
      • 2020-11-04 30928, 2020

      • bitmap
        lemme check out 1636 again
      • 2020-11-04 30937, 2020

      • bitmap
        oh, I also converted teh login page to react in that PR. should check if we had a ticket for that
      • 2020-11-04 30956, 2020

      • yvanzo
        reosarevok: about 1636 (and others), did you start using generate_edit_data_flow_type.js ?
      • 2020-11-04 30905, 2020

      • reosarevok
        Nope, not yet
      • 2020-11-04 30915, 2020

      • reosarevok
        Those were all older than that IIRC anyway
      • 2020-11-04 30927, 2020

      • reosarevok
        Maybe we should do a pass on existing React edits
      • 2020-11-04 30933, 2020

      • reosarevok
        See what possible bugs we find
      • 2020-11-04 30952, 2020

      • bitmap
        it's not really useful yet since we don't tend to use edit.data for display
      • 2020-11-04 30937, 2020

      • bitmap
        the idea is that it'd make it easier to build display data in JS that way though
      • 2020-11-04 30942, 2020

      • reosarevok
        Oh, yeah, I guess it should do the same for display_data or we should refactor them
      • 2020-11-04 30955, 2020

      • reosarevok
        Oh, ok, so send data raw and then refactor the build_display_data to JS?
      • 2020-11-04 30907, 2020

      • bitmap
        that would be ideal, yes
      • 2020-11-04 30928, 2020

      • bitmap
        I think we already send the data anyway
      • 2020-11-04 30931, 2020

      • reosarevok
        We could do that for a couple easy ones at first, I guess - where would you store the function, just on the same edit component file?
      • 2020-11-04 30943, 2020

      • bitmap
        yup
      • 2020-11-04 30946, 2020

      • reosarevok
        But maybe something to do post React conversion? Dunno tbh
      • 2020-11-04 30902, 2020

      • bitmap
        sure, wasn't suggesting now or anything
      • 2020-11-04 30929, 2020

      • bitmap
        just sayin' why the type info isn't necessary till then
      • 2020-11-04 30930, 2020

      • reosarevok
        Sure, just wondering whether we should convert as we do now and then refactor or try to do that for the conversions still missing
      • 2020-11-04 30906, 2020

      • bitmap
        I'd continue as usual for now
      • 2020-11-04 30917, 2020

      • reosarevok
        Ok
      • 2020-11-04 30947, 2020

      • bitmap
        unless it's something super simple that takes no effort to convert
      • 2020-11-04 30923, 2020

      • reosarevok
        I've probably done most super simple things, but not 100% sure
      • 2020-11-04 30953, 2020

      • reosarevok
        yvanzo: https://github.com/metabrainz/musicbrainz-server/… should be trivial and fix some broken bios and whatnot
      • 2020-11-04 30914, 2020

      • reosarevok
        bitmap: does https://github.com/metabrainz/musicbrainz-server/… seem legit or is it just papering over the bug?
      • 2020-11-04 30927, 2020

      • bitmap
        I'm not sure it makes sense to display stuff like cardinality which didn't exist when certain edits were entered, but the display seems fine otherwise
      • 2020-11-04 30918, 2020

      • reosarevok
        Hmm. What does it show for those? Is it broken?
      • 2020-11-04 30939, 2020

      • reosarevok
        If not, I think it's fine to claim that they used the (at that time only existing) cardinality
      • 2020-11-04 30905, 2020

      • bitmap
        1771 looks good to me (but didn't test)
      • 2020-11-04 30930, 2020

      • reosarevok
        I did (by giving some random Japanese name to hostname)
      • 2020-11-04 30952, 2020

      • reosarevok
        I just thought maybe we should check where it's being re-encoded and stop that, but I couldn't find it
      • 2020-11-04 30959, 2020

      • bitmap
        nothing broken, just makes it appear as if these were specifically added with that cardinality when the edit was entered
      • 2020-11-04 30900, 2020

      • reosarevok
        So I submitted what I did find that worked
      • 2020-11-04 30916, 2020

      • reosarevok
        I guess, but in most cases cardinality is filled in by default and untouched, so I don't feel bad about it
      • 2020-11-04 30941, 2020

      • reosarevok
        If you feel we should not display it for those though that's ok too
      • 2020-11-04 30908, 2020

      • bitmap
        I generally prefer to only show fields actually submitted with the edit, but it's not wrong or anything so I approved it regardless :)
      • 2020-11-04 30907, 2020

      • reosarevok
        yvanzo: https://github.com/metabrainz/musicbrainz-server/… adds a few more eslint rules to CI - not urgent since it doesn't touch many files though
      • 2020-11-04 30936, 2020

      • BrainzGit
        [musicbrainz-server] reosarevok merged pull request #1768 (master…MBS-10664): MBS-10664: Allow more tags in expand2react https://github.com/metabrainz/musicbrainz-server/…
      • 2020-11-04 30937, 2020

      • BrainzBot
      • 2020-11-04 30900, 2020

      • BrainzGit
        [musicbrainz-server] reosarevok merged pull request #1636 (master…MBS-11018): MBS-11018: Convert Add Relationship Type edit to React https://github.com/metabrainz/musicbrainz-server/…
      • 2020-11-04 30902, 2020

      • BrainzBot
        MBS-11018: Convert Add Relationship Type edit to React https://tickets.metabrainz.org/browse/MBS-11018
      • 2020-11-04 30941, 2020

      • reosarevok
      • 2020-11-04 30945, 2020

      • yvanzo
        I would also prefer to distinguish actually submitted fields from others in edit display in general, when possible.
      • 2020-11-04 30902, 2020

      • reosarevok
        Ok, I can submit a change for that into master :)
      • 2020-11-04 30946, 2020

      • reosarevok
        yvanzo: re that last linked PR: is it ok to remove /gakki in that situation? I wouldn't mind adding the three options to the regex in URL.pm either, but IIRC we'd need three separate lines for constants.js?
      • 2020-11-04 30902, 2020

      • reosarevok
        (you suggested adding it at first, so thought I'd explicitly ask :) )
      • 2020-11-04 30925, 2020

      • niceplace has quit
      • 2020-11-04 30932, 2020

      • yvanzo
        reosarevok: why was the path specified in the first place?
      • 2020-11-04 30948, 2020

      • reosarevok
        The site has stuff that seems unrelated to this
      • 2020-11-04 30906, 2020

      • reosarevok
        "Saisai Batake is a graphic tool dedicated to creating small images that uniquely color your homepage ."
      • 2020-11-04 30910, 2020

      • reosarevok
        is the main page :)
      • 2020-11-04 30918, 2020

      • yvanzo
        oh ok :)
      • 2020-11-04 30919, 2020

      • reosarevok
        It's just that the person also runs an instrument DB in the same domain
      • 2020-11-04 30949, 2020

      • niceplace joined the channel
      • 2020-11-04 30918, 2020

      • BrainzGit
        [musicbrainz-server] reosarevok opened pull request #1777 (master…hide-unchosen-cardinality): Small amends to AddRelationshipType https://github.com/metabrainz/musicbrainz-server/…
      • 2020-11-04 30928, 2020

      • reosarevok
        yvanzo, bitmap ^ then
      • 2020-11-04 30917, 2020

      • BrainzGit
        [musicbrainz-server] reosarevok merged pull request #1771 (master…MBS-11207): MBS-11207: Avoid double-encoding DBI errors https://github.com/metabrainz/musicbrainz-server/…
      • 2020-11-04 30918, 2020

      • BrainzBot
        MBS-11207: Error messages from DBI are badly encoded https://tickets.metabrainz.org/browse/MBS-11207
      • 2020-11-04 30924, 2020

      • reosarevok
        Also re ^ - would it be a problem to change the encode('utf-8', $whatever) lines to just use encode_utf8 directly?
      • 2020-11-04 30936, 2020

      • reosarevok
        Seems cleaner to me and an easy change, but maybe it's not a good idea
      • 2020-11-04 30951, 2020

      • yvanzo
        reosarevok: I don’t mind 3 liens in constants.js but maybe we should make this pattern matching more reliable in the future.
      • 2020-11-04 30907, 2020

      • bitmap
        reosarevok: there's apparently a difference between utf8 and UTF-8 and encode_utf8 uses the former
      • 2020-11-04 30901, 2020

      • reosarevok sighs at perl
      • 2020-11-04 30922, 2020

      • reosarevok
        yvanzo: nice casual langue française
      • 2020-11-04 30928, 2020

      • bitmap
        the strict version sounds preferrable, though I haven't looked at existing uses. probably best to leave it
      • 2020-11-04 30941, 2020

      • reosarevok
        So you'd rather I had 3 links than a generic one? Can change then :)
      • 2020-11-04 30949, 2020

      • reosarevok
        bitmap: ok, let's not touch it
      • 2020-11-04 30934, 2020

      • reosarevok
        yvanzo: updated the PR then :)
      • 2020-11-04 30937, 2020

      • reosarevok
        Thanks!
      • 2020-11-04 30901, 2020

      • v6lur joined the channel
      • 2020-11-04 30910, 2020

      • reosarevok
        yvanzo: if you have some time to check https://github.com/metabrainz/musicbrainz-server/… I can merge that already to make sure it gets released with the original commits :)
      • 2020-11-04 30932, 2020

      • BrainzGit
        [musicbrainz-server] reosarevok merged pull request #1777 (master…hide-unchosen-cardinality): Small amends to AddRelationshipType https://github.com/metabrainz/musicbrainz-server/…
      • 2020-11-04 30947, 2020

      • BrainzGit
        [musicbrainz-server] reosarevok merged pull request #1775 (master…MBS-11210): MBS-11210: Allow more subpaths for saisaibatake.ame-zaiku.com https://github.com/metabrainz/musicbrainz-server/…
      • 2020-11-04 30948, 2020

      • BrainzBot
      • 2020-11-04 30921, 2020

      • CatYes is now known as perl
      • 2020-11-04 30927, 2020

      • perl sighs at reosarevok
      • 2020-11-04 30929, 2020

      • perl is now known as CatQuest
      • 2020-11-04 30915, 2020

      • reosarevok sighs at sighing
      • 2020-11-04 30924, 2020

      • reosarevok
        https://github.com/metabrainz/musicbrainz-server/… seems to be the last one I've worked on recently that seems small
      • 2020-11-04 30925, 2020

      • CatQuest
        hey
      • 2020-11-04 30940, 2020

      • CatQuest
        i linked those saisaibatanke tickets
      • 2020-11-04 30942, 2020

      • reosarevok
        CatQuest: ^ your new links should be addable from beta whenever we release a new beta :)