that's not a perfect solution, because it concerns only stuff going through gateways
2022-05-03 12314, 2022
zas
but basically it prevents openresty to pass requests to backends
2022-05-03 12303, 2022
yvanzo
Thank you!
2022-05-03 12337, 2022
reosarevok
yvanzo: should I merge all the open sql PRs into a local branch and try to at the very least upgrade.sh the sample db?
2022-05-03 12300, 2022
reosarevok
I guess we should eventually run the upgrade on a full DB too, but maybe starting with a sample is good to see if we find any quick and obvious issues
2022-05-03 12344, 2022
reosarevok
And once we properly merge all, then we run a full proper upgrade test or two
2022-05-03 12334, 2022
yvanzo
something like schema-change-2022-alpha.0 so that it will never be mistakenly used as a final thing by anyone?
2022-05-03 12326, 2022
reosarevok
I was thinking literally local, as in, in my laptop only :D
2022-05-03 12331, 2022
yvanzo
ah ok :)
2022-05-03 12356, 2022
hackerman5 joined the channel
2022-05-03 12341, 2022
zas
yvanzo: I need your help on static files sync for rudi (sshd-nginx-static-resources container), it seems the server IP needs to be add to https://github.com/metabrainz/docker-server-confiā¦ , so that's not very dynamic (I guess it implies a restart of all musicbrainz-website-* containers
2022-05-03 12351, 2022
hackerman has quit
2022-05-03 12351, 2022
hackerman5 is now known as hackerman
2022-05-03 12349, 2022
zas
do you think we can improve that somehow?
2022-05-03 12311, 2022
yvanzo
zas: I would first have to understand how it works for MB website :/ (MB sitemaps is easier to get.) I guess it could be set through DBDefs instead? Can make a ticket not to forget about it?
2022-05-03 12307, 2022
yvanzo
reosarevok: Are you more familiar with how MB websites updates staticbrainz? (I donāt find any call to rsync-staticbrainz-files but in rsync-sitemaps.)
2022-05-03 12333, 2022
reosarevok
I am not, not directly, but IIRC this was discussed at some point, let me see
2022-05-03 12326, 2022
reosarevok
"in MB we have a STATIC_RESOURCES_LOCATION config option that points to staticbrainz in prod" is that relevant?
2022-05-03 12355, 2022
yvanzo
Not directly
2022-05-03 12309, 2022
reosarevok
There's also StaticBrainz.md in syswiki but I guess you've seen that?
2022-05-03 12327, 2022
yvanzo
STATIC_RESOURCES_LOCATION is for getting resources from staticbrainz, not the other way around.
2022-05-03 12310, 2022
reosarevok
Oh, sure
2022-05-03 12349, 2022
reosarevok
Well, StaticBrainz.md does mention a bit of the other way around
2022-05-03 12310, 2022
yvanzo
Yes but it is not complete. It might be in a private repo
2022-05-03 12356, 2022
BrainzGit
[listenbrainz-server] 14alastair opened pull request #1978 (03masterā¦metadata-cache-check-exists): Check if the canonical_release_redirect table exists before running https://github.com/metabrainz/listenbrainz-serverā¦
2022-05-03 12359, 2022
yvanzo
zas, reosarevok: found it, server IP is set in musicbrainz-staticbrainz project in Jenkins.
2022-05-03 12323, 2022
yvanzo
It currently has herb & kiki only.
2022-05-03 12349, 2022
zas
can you add rudi?
2022-05-03 12354, 2022
yvanzo
ok
2022-05-03 12337, 2022
zas
I'm not sure the setup is correct on rudi's side, we'll find out soon
2022-05-03 12329, 2022
yvanzo
I updated the project configuration, now trying to synchronize test.mb.o resources
(yvanzo: when you have some time please rebase that and fix conflicts :) )
2022-05-03 12354, 2022
yvanzo
zas, reosarevok: no error when rsync-ing MB staticbrainz files from Jenkins to rudi :) looking into prod and beta now
2022-05-03 12318, 2022
zas
yvanzo: perfect :)
2022-05-03 12331, 2022
zas
thank you for the help on this
2022-05-03 12349, 2022
yvanzo
zas: apparently it did rsync static files for all of prod/beta/test at once, further runs based on beta/prod images did not change anything to rudi.
2022-05-03 12323, 2022
BrainzGit
[listenbrainz-server] 14alastair opened pull request #1979 (03masterā¦metadata-cache-docs): LB-1079: Fix mb_metadata_cache update methods to work with bulk table code https://github.com/metabrainz/listenbrainz-serverā¦
2022-05-03 12333, 2022
yvanzo
zas: Should I also add MB sitemaps to rudi?
2022-05-03 12349, 2022
uZer has quit
2022-05-03 12338, 2022
zas
I think so
2022-05-03 12316, 2022
zas
In fact, anything set up for kiki/herb should also be for rudi
[listenbrainz-server] 14alastair opened pull request #1980 (03masterā¦consul-config-listenstore): Use consul-template if-statement to check if the listenstore is up https://github.com/metabrainz/listenbrainz-serverā¦
2022-05-03 12326, 2022
zas
yvanzo: ah yes, sorry, I think you can drop set -x, it was there for debugging purposes
2022-05-03 12338, 2022
zas
so LGTM for both
2022-05-03 12338, 2022
yvanzo
Ok, thanks!
2022-05-03 12324, 2022
yvanzo
I updated MB sitemaps to use rudi too, I will run again at :30, will check it then.
2022-05-03 12351, 2022
yvanzo
"cron job will run"
2022-05-03 12311, 2022
yvanzo
reosarevok: opened a PR, please review if that it is understandable enough :)
2022-05-03 12349, 2022
adhawkins
I think my slave container setup has corrupted its database.
2022-05-03 12321, 2022
adhawkins
Is it likely to be rescuable, or shall I just download a database dump and start again?
2022-05-03 12337, 2022
reosarevok
yvanzo: checking, let me see
2022-05-03 12316, 2022
yvanzo
adhawkins: how corrupted?
2022-05-03 12351, 2022
adhawkins
yvanzo: My replication was throwing errors. However, I've just kicked it off again and it seems to be proceeding. Will monitor it.
2022-05-03 12342, 2022
BrainzGit
[listenbrainz-server] 14alastair merged pull request #1978 (03masterā¦metadata-cache-check-exists): Check if the canonical_release_redirect table exists before running https://github.com/metabrainz/listenbrainz-serverā¦
2022-05-03 12342, 2022
BrainzGit
[listenbrainz-server] 14alastair merged pull request #1979 (03masterā¦metadata-cache-docs): LB-1079: Fix mb_metadata_cache update methods to work with bulk table code https://github.com/metabrainz/listenbrainz-serverā¦
2022-05-03 12324, 2022
odnes has quit
2022-05-03 12359, 2022
yvanzo
adhawkins: Can you share some errors through pastebin so I can check these?
2022-05-03 12332, 2022
adhawkins
If I see any more, I will yvanzo.
2022-05-03 12343, 2022
adhawkins
It was something along the lines of duplicate index entries I think.
2022-05-03 12353, 2022
adhawkins
Sorry, I appreciate that's not particularly helpful :)
2022-05-03 12320, 2022
outsidecontext
rdswift: Hi, I merged picard-docs master to release for new 2.8.0rc1, and pages build as far as I can see. But does not show up on the website. Any idea why? Usually that worked rather quickly, or is this just GH pages being slow with updating? gh-pages branch looks good to me
2022-05-03 12301, 2022
reosarevok
yvanzo: hmm. as of now, upgrade.sh fails with ERROR: schema "dbmirror2" does not exist
2022-05-03 12318, 2022
outsidecontext
zas: are you around later for publishing the website or shall we do it now already?
2022-05-03 12333, 2022
zas
I'll be around
2022-05-03 12355, 2022
reosarevok
So I guess the upgrade script needs to include CREATE SCHEMA dbmirror2;
2022-05-03 12309, 2022
outsidecontext
ok, I'll ping you. I'm a bit AFK for now
2022-05-03 12331, 2022
rdswift
outsidecontext, I have no idea. It looks like the build went okay. Let me check further.
2022-05-03 12310, 2022
mayhem
yvanzo: I think lucifer can help you with some of the things you might need help with for the schema change. can you please get lucifer up to speed on what you need help with?
2022-05-03 12300, 2022
atj
zas: any objection to adding "metabrainz.org" as a search param in the netplan DNS configuration?
2022-05-03 12350, 2022
outsidecontext
rdswift: Probably just a GitHub issue. I know it takes a few minutes for changes to be available. But this time it is unusually long, so I wondered if maybe made some mistake
2022-05-03 12317, 2022
HenryG has quit
2022-05-03 12324, 2022
outsidecontext
But if it looks ok to you maybe just wait a bit longer. I need to wait for the snap builds to finish anyway :)
2022-05-03 12308, 2022
rdswift
Actually, it looks like it might be something with the 404 remapping code. It doesn't seem to be doing the version check properly. I'm looking at it now.
2022-05-03 12303, 2022
HenryG joined the channel
2022-05-03 12322, 2022
reosarevok
yvanzo: pushed that. next issue: ERROR: trigger "restore_collection_sub_on_public" for relation "editor_collection" already exists
2022-05-03 12343, 2022
reosarevok
We've already added that on prod, so I guess I need to DROP TRIGGER IF EXISTS it first :)
2022-05-03 12300, 2022
yvanzo
Yes
2022-05-03 12320, 2022
yvanzo
This is what Iām changing in sir triggers too by the way
2022-05-03 12337, 2022
reosarevok
I wasn't even asking, I just keep talking in a rubber-ducky way for some reason :D But I'll send a PR with that + changing schema to 27 in upgrade.sh, then we can merge that already
Looks like we might have hit their 1GB limit for the site.
2022-05-03 12351, 2022
v6lur joined the channel
2022-05-03 12351, 2022
rdswift
Time to find a different hosting option?
2022-05-03 12346, 2022
akshaaatt
Hi elomatreb[m]! https://www.figma.com/file/ln8XiFLit634KC3YkUW2RSā¦ serves as the link for the firm redesign. As of now the release page redesign was initiated by aerozol on the community forum but we'll look to have it added to figma soon.
2022-05-03 12308, 2022
elomatreb[m]
thanks!
2022-05-03 12325, 2022
elomatreb[m]
I was thinking maybe we could show a colored world map (like the one you get in the statistics on Listenbrainz) instead of the huge list of flags and dates we currently show for imported digital releases
2022-05-03 12345, 2022
elomatreb[m]
(I'm resigned to this style being accepted by the community)
2022-05-03 12349, 2022
yvanzo
rdswift: How does it come this repository takes 1GB in the first place?
2022-05-03 12325, 2022
yvanzo
zas: answered, thank you
2022-05-03 12358, 2022
rdswift
Mainly because of the images in the documentation and it saves each version number separately for each supported language (readthe docs type format).
2022-05-03 12339, 2022
rdswift
I just checked the size on my local copy and it's just over 2GB
2022-05-03 12335, 2022
outsidecontext
but the pages themselves shouldn't be that large really
2022-05-03 12349, 2022
outsidecontext
maybe the PDF / epub downloads
2022-05-03 12327, 2022
outsidecontext
I'd expect to see some message in github preferences or such, but don't see anything. really strange
2022-05-03 12328, 2022
yvanzo
maybe optimizing images (color palette, zopflipng) could also help with moderating the overall size?
2022-05-03 12320, 2022
rdswift
The problem is that each image is stored for each version and language, so reducing image size should buy us some time.
2022-05-03 12300, 2022
rdswift
What would be really nice would be to find a way for sphinx to only save one copy of an image.
2022-05-03 12302, 2022
outsidecontext
overall image size for the latest version is 8 MB, don't think this will gain us much. we have 6 versions with 2 languages
2022-05-03 12344, 2022
atj
elomatreb[m]: not sure how that would work to be honest. However I think aerozol has taken it upon himself to try and come up with a solution, so it might be worth discussing it with him.
2022-05-03 12346, 2022
outsidecontext
maybe time to consider moving to readthedocs, don't know
2022-05-03 12349, 2022
yvanzo
It can probably be automated though (Git pre-commit hooks, or GitHub Actions).
2022-05-03 12321, 2022
elomatreb[m]
atj: I posted in the release redesign thread on the forums
2022-05-03 12336, 2022
yvanzo
8MB to 1GB, if you reduce image size by half, it saves a lot overall.
2022-05-03 12338, 2022
elomatreb[m]
Although I don't see what would be hard about technically, since you can just map country codes to some specific SVG element?
2022-05-03 12318, 2022
yvanzo
Obviously it wonāt solve the hosting issue in the longer term.
2022-05-03 12329, 2022
rdswift
outsidecontext, you're right. Maybe images aren't the problem. I'll do some more digging.
2022-05-03 12304, 2022
outsidecontext
rdswift: if it is about repository size we might be out of luck with GH pages without purging e.g. commit history
ok, but we probably don't get a quick solution today, so let's skip this for the 2.8rc1 release. not absolutely mandatory for that
2022-05-03 12337, 2022
outsidecontext
zas: could you publish the website?
2022-05-03 12337, 2022
outsidecontext
rdswift: what we could try is completely squashing the gh-pages branch into a single commit
2022-05-03 12359, 2022
outsidecontext
I wouldn't care about it's history much
2022-05-03 12318, 2022
rdswift
Agreed that might make the difference. Of the 2G in the directory, the .git directory eats up 1.5G
2022-05-03 12343, 2022
rdswift
Want to give it a try?
2022-05-03 12346, 2022
outsidecontext
rdswift: shall I try that?
2022-05-03 12352, 2022
outsidecontext
yes
2022-05-03 12308, 2022
alastairp
is this from a github action?
2022-05-03 12318, 2022
rdswift
Go for it. If we need to recover, I have the full history in my clone of the repo.
2022-05-03 12350, 2022
alastairp
there might also be ways to tell it to do a sparse checkout? (I wasn't following and am not sure if the problem is during build, or when trying to push your existing repo)
2022-05-03 12304, 2022
outsidecontext
alastairp: it's likely the actual repository size on github that hits quota limits for Github pages
2022-05-03 12338, 2022
alastairp
right, I just read your logs a bit more and saw that
