monkey: lucifer ansh : I think we need to pivot away from the album cover mosaic endpoint -- that is basically a DDoS waiting to happen. We need to make one image fetch from the IA, process that image and then *for* *each* pixel, make a DB query. Even as an undocumented API, this spells danger. I think I am going to fall back on making mosaics of all the top releases instead. if we make the individual pixels small enough, I doubt
2024-12-18 35346, 2024
mayhem[m]
anyone could hit us with a copyright complaint.
2024-12-18 35344, 2024
monkey[m]
That sounds wise, I suggested the same but fearing we would unknowingly participate in DDOSing the IA
2024-12-18 35306, 2024
mayhem[m]
yeah, another problematic angle.
2024-12-18 35320, 2024
monkey[m]
I'm setting up a gallery things as we speak, I'm sure the results will also be much better with a precomputed image
2024-12-18 35347, 2024
monkey[m]
(loading all those images on demand results in an early 2000s style of experience)
2024-12-18 35304, 2024
hypeventure[m]
monkey[m]: So basically a database for cover art and such
2024-12-18 35342, 2024
monkey[m]
Hi! We are talking specifically about a small part of our year in music feature
2024-12-18 35353, 2024
hypeventure[m]
Speaking of this: do we have a place for artist profile picture soon?
2024-12-18 35332, 2024
hypeventure[m]
Like... On the artist page or something like that
2024-12-18 35338, 2024
monkey[m]
Unlikely, there are a bunch of copyright infringement issue attached to that idea that MetaBrainz does not want to expose itself to
For ListenBrainz, provided we have links to, say, spotify, or apple music, tidal, etc. we could potentially load the artist cover that these services use without any risk
hypeventure[m] uploaded an image: (423KiB) < https://matrix.chatbrainz.org/_matrix/media/v3/download/matrix.org/TbQxyzkJzkXaygVdTZqeKNdZ/Screenshot_20241218-231706.png >
2024-12-18 35313, 2024
hypeventure[m] uploaded an image: (150KiB) < https://matrix.chatbrainz.org/_matrix/media/v3/download/matrix.org/SpMuWXimWObmjnzvPfAFNGWM/Screenshot_20241218-231747.png >
2024-12-18 35343, 2024
hypeventure[m]
Frankly this is where google knowledge panel database got the meta from
2024-12-18 35301, 2024
hypeventure[m]
Mostly musicbrainz and this is when I discovered what that is
2024-12-18 35317, 2024
hypeventure[m]
So I got all of my released tracks to here and update all the meta
2024-12-18 35337, 2024
hypeventure[m]
And done. After 3 songs, I got the board.
2024-12-18 35346, 2024
hypeventure[m]
I didn't down much with this meta thing until now so...
2024-12-18 35352, 2024
hypeventure[m]
First time here, finally
2024-12-18 35358, 2024
monkey[m]
Welcome :)
2024-12-18 35341, 2024
hypeventure[m]
And whoever did much of update thingy to my artist page here on musicbrainz, you're welcome
2024-12-18 35315, 2024
mayhem[m]
time to load 100487 releases from the IA.
2024-12-18 35319, 2024
hypeventure[m]
Maybe with enough journalist or so then someone out there will put me to Wikipedia lol...
2024-12-18 35328, 2024
hypeventure[m]
mayhem[m]: What the IA?
2024-12-18 35335, 2024
mayhem[m]
internet archive.
2024-12-18 35343, 2024
hypeventure[m]
s/you're/thanks/, s/welcome/too/
2024-12-18 35353, 2024
hypeventure[m]
mayhem[m]: Got it
2024-12-18 35353, 2024
hypeventure[m]
I recently made an entry for an upcoming music studio that I worked with. They are indies, but for the upcoming track I'm doing, it's with the founder.
hypeventure[m]: No, you have to add a new place instead.
2024-12-18 35319, 2024
hypeventure[m]
yvanzo[m]: Noted.
2024-12-18 35330, 2024
hypeventure[m]
yvanzo[m]: I will head to there.
2024-12-18 35342, 2024
ansh[m]
monkey: While working on YIM, are you reusing the data from 2023?
2024-12-18 35350, 2024
monkey[m]
Currently yes, I saved my YIM23 data into a fakeData.json file, import it in 2024, then use it in YearInMusicWrapper: const { user = fallbackUser, data: yearInMusicData } = fakeData || {};
2024-12-18 35321, 2024
ansh[m]
makes sense. I'll import the previous year's data in my DB, so that it's easier to manipulate the data to get it in the format the graph needs.
2024-12-18 35351, 2024
monkey[m]
Right, I didn't look at the genre data at all, but makes sense to copy in your DB for your use case
2024-12-18 35322, 2024
minimal joined the channel
2024-12-18 35309, 2024
mayhem[m]
meeting time?
2024-12-18 35327, 2024
lucifer[m]
yup
2024-12-18 35349, 2024
lucifer[m]
reosarevok: yvanzo bitmap around?
2024-12-18 35339, 2024
lucifer[m]
hi! so i mostly wanted to discuss how to progress with phase 2 of the oauth stuff. migrating users from MB db to MeB db.
2024-12-18 35325, 2024
yvanzo[m]
Can you please recall where phase 1/2 are described?
2024-12-18 35341, 2024
lucifer[m]
the MeB side of things is coming along nicely and i will put it up on test.meb for testing in a week or two and we will have a fully dfunctiong login/registration system.
2024-12-18 35343, 2024
mayhem[m]
I guess step 1 of phase 2 is to implement user profiles on meb.org no?
2024-12-18 35307, 2024
mayhem[m]
lucifer[m]: so, what I just described as step 1 then?
2024-12-18 35316, 2024
lucifer[m]
the phase 1 was a new oauth provider, and phase 2 is user registeration.
2024-12-18 35318, 2024
lucifer[m]
mayhem: yes, those are ready.
2024-12-18 35326, 2024
mayhem[m]
great!
2024-12-18 35342, 2024
lucifer[m]
well, the main thing pending is making the admin portal for supporters work and email verification.
2024-12-18 35349, 2024
mayhem[m]
I would think that we need to work out some DB migration scripts next (in parallel to testing the UI, of course)
2024-12-18 35303, 2024
lucifer[m]
yes, that too is on my list.
2024-12-18 35314, 2024
bitmap[m]
tangentially related, but we very recently added some restrictions to the MB registration form to combat spammers/vandals, these likely need to be ported over
2024-12-18 35316, 2024
lucifer[m]
* @yvanzo: the phase
2024-12-18 35326, 2024
lucifer[m]
makes sense
2024-12-18 35304, 2024
lucifer[m]
so once we have a user login and registration system in MeB ready, what should be the next steps to migrate?
2024-12-18 35337, 2024
lucifer[m]
1. should we decide a date to switch off registration in MB and point to the MeB page?
2024-12-18 35337, 2024
mayhem[m]
assuming its fully tested, then just migration itself, no?
2024-12-18 35341, 2024
yvanzo[m]
I also identified an issue with the lack fo rate limit for SSO (currently triggered by Weblate).
2024-12-18 35359, 2024
yvanzo[m]
s/fo/of/
2024-12-18 35315, 2024
lucifer[m]
2. keep both registration pages running in parallel, and sync between MeB and MB users table somehow.
2024-12-18 35321, 2024
mayhem[m]
lucifer: are you aiming for a no-downtime solution?
2024-12-18 35335, 2024
lucifer[m]
that is up for discussion as well
2024-12-18 35308, 2024
lucifer[m]
also, would this need to be done as a schema change on MB side?
2024-12-18 35319, 2024
mayhem[m]
because I always thought that we would take down time, dump tables from MB, import tables into MeB. update both MB and MeB and continue (others also need minor update)
2024-12-18 35344, 2024
mayhem[m]
downtime -- might be limited to "no logins, no new accounts"
2024-12-18 35352, 2024
mayhem[m]
then log everyone out, and force re-login via Meb.
2024-12-18 35313, 2024
bitmap[m]
lucifer[m]: this would be my personal preference until we are sure the MeB registration method is fully functioning
2024-12-18 35315, 2024
mayhem[m]
lucifer[m]: I sure hope not.
2024-12-18 35319, 2024
lucifer[m]
eventually, the aim is to have MB function as an oauth app. registration happens on MeB and users login to MB using their MeB accounts.
2024-12-18 35341, 2024
mayhem[m]
do we suspect that we actually need to keep two registration systems running at the same time?
2024-12-18 35345, 2024
bitmap[m]
there is a schema change needed in that we are moving some columns of the MB editor table to MeB. though perhaps we can sidestep the schema change by nulling/blanking the unused columns in MB
2024-12-18 35348, 2024
lucifer[m]
bitmap[m]: i do think this is the safer option because some issues always creep into the first few prod releases
2024-12-18 35351, 2024
mayhem[m]
are we that uncertain about the migration?
2024-12-18 35320, 2024
mayhem[m]
bitmap[m]: sidestep, please.
2024-12-18 35323, 2024
yvanzo[m]
I sure hope it is a no-downtime approach that can allow reverting to MB if something goes wrong.
2024-12-18 35354, 2024
mayhem[m]
well, it sounds like people prefer the dual systems approach. sounds safer for sure.
2024-12-18 35358, 2024
lucifer[m]
uncertain only in issues that are found out after we deploy and migrate. we can test thorougly but if something major comes up having both up might be useful.
2024-12-18 35313, 2024
mayhem[m]
ok, fair enough.
2024-12-18 35315, 2024
reosarevok[m]
Yeah, users is a fairly important thing to not mess up :)
2024-12-18 35327, 2024
reosarevok[m]
I mean we can drop that rob guy but
2024-12-18 35334, 2024
lucifer[m]
makes sense so that is definitely more work as we need to sync users tables in two databases in sync.
2024-12-18 35302, 2024
lucifer[m]
so the next thing is to discuss some details of the dual system.
2024-12-18 35303, 2024
mayhem[m]
1. Create sync between MB and MeB. 2. Deploy account creation & sync 3. Debug. 4. Copy over the old accounts 5. turn off reg on MB
2024-12-18 35306, 2024
mayhem[m]
is that the process then?
2024-12-18 35311, 2024
lucifer[m]
yes.
2024-12-18 35304, 2024
mayhem[m]
for the dual system, we're not going to allow new users to be created both on MB and MeB, right?
2024-12-18 35312, 2024
mayhem[m]
if we don't, then the sync is easy.
2024-12-18 35319, 2024
mayhem[m]
copy over new rows from MeB to MB.
2024-12-18 35320, 2024
lucifer[m]
yeah that's the tough part.
2024-12-18 35332, 2024
lucifer[m]
iiuc, the intention is to allow both.
2024-12-18 35343, 2024
mayhem[m]
is that necessary?
2024-12-18 35319, 2024
mayhem[m]
it rather complicates things. We're going from easy to "shit, mutiple write masters" which is hard
2024-12-18 35324, 2024
reosarevok[m]
It might be fine to have a dbdefs way of turning off registration in MB, so we can easily turn it on again if something breaks
2024-12-18 35357, 2024
reosarevok[m]
(by which time, maybe it's fine to just ignore the meb registrations, unless what broke is that they didn't get copied back)
2024-12-18 35300, 2024
lucifer[m]
i think one solution could be to have MeB registration page create a user in both MB and MeB db.
2024-12-18 35320, 2024
mayhem[m]
how do you handle a failure in one of the two DBs?
2024-12-18 35342, 2024
reosarevok[m]
"Sorry but we cannot create new accounts right now because something is on fire"? :P
2024-12-18 35349, 2024
lucifer[m]
not sure what you mean
2024-12-18 35317, 2024
lucifer[m]
but hopefully after a few weeks of testing we just copy the rows and keep it all in just MeB db.
2024-12-18 35341, 2024
mayhem[m]
for creating an account, you have two open DB connections. you create the account in a transaction for each connection. one connections succeeds, the other fails and rolls back. now what?
2024-12-18 35325, 2024
lucifer[m]
ah okay that yeah it just fails.
2024-12-18 35344, 2024
lucifer[m]
can write to MB db first so that is still the ground truth.
2024-12-18 35347, 2024
mayhem[m]
but now you have one account created, the other one not. now you need to clean that up. uuuugly
2024-12-18 35320, 2024
mayhem[m]
I would prefer only one place to create accounts, while making it easy to switch between the two in case shit happens
2024-12-18 35330, 2024
bitmap[m]
you could delay committing the first transaction until the inner one succeeds no?
2024-12-18 35333, 2024
lucifer[m]
right, so one thing that i'll have to build in the login system is to support logins from both databases.
2024-12-18 35308, 2024
lucifer[m]
so that we'll be checking for an account to exist in either one and log in with that.
2024-12-18 35336, 2024
lucifer[m]
with the MB db taking precedence
2024-12-18 35348, 2024
lucifer[m]
so as long as the account is in MB, everything works fine.
2024-12-18 35353, 2024
mayhem[m]
ick. I really don't like it.
2024-12-18 35305, 2024
mayhem[m]
but if you think that is the best way forward, then so be it.
2024-12-18 35350, 2024
lucifer[m]
i can't think of a better way to make the parallel system work at the moment.
2024-12-18 35357, 2024
yvanzo[m]
The migration could be in several steps: oauth/signing in support through MeB.o, only at another date registration support through MeB.o.
2024-12-18 35316, 2024
yvanzo[m]
So you can first test synchronization.
2024-12-18 35332, 2024
mayhem[m]
lucifer[m]: which is why I don't like the parallel system. we're trying to make things easier on one side, while making things MUCH harder on the other.
2024-12-18 35337, 2024
lucifer[m]
right but that means supporting registrations from both systems.
2024-12-18 35300, 2024
yvanzo[m]
lucifer[m]: Not if you do switch it at a later date.
2024-12-18 35304, 2024
lucifer[m]
s/./,, yvanzo /
2024-12-18 35332, 2024
lucifer[m]
you can't test oauth signing in MB through MeB.o without that no?
2024-12-18 35348, 2024
reosarevok[m]
I guess the suggestion is "register from MB, but then sync it to the MeB DB and log in using that later"?
2024-12-18 35305, 2024
yvanzo[m]
yes
2024-12-18 35307, 2024
reosarevok[m]
So you only register from MB at first, but already move the place for logging in to MeB?
2024-12-18 35311, 2024
lucifer[m]
that could work better i think.
2024-12-18 35324, 2024
yvanzo[m]
yes
2024-12-18 35328, 2024
reosarevok[m]
And then once you're sure that works, you start creating accounts on MeB instead, right?
