#metabrainz

Thanks !

thanks for running the meeting!

whatchagot, lucifer ?

mayhem or rdswift: Since I'm traveling next Monday, maybe one of you two can run the meeting? :)

I might be able to attend, but I wouldn't dare say I can run it : )

bitmap: chaban did some more digging into why external links wont load: https://tickets.metabrainz.org/browse/MBS-12996

MBS-12996: 'External Links' fails to load when creating new artist

I have a dinner I am supposed to be at, but I think I can manage, since the dinner starts right at the end of themeeting.

I can run the meeting, but might need to pass the baton if it runs longer than 45 minutes.

mayhem: about the listens schema migration, i ran the script initially and made a few fixes. it seems fine rn but we should probably be careful with it.

were the diskspace alerts related to that>

?

so question is how careful and how do we make sure that its actually correct?

yes

run it, then point test.lb to the new table?

there's an old volume from last year's timescale migration which can be removed to free up 0.5TB. i'll do that.

ok

also, inserting the rickroll into all daily jams needs to happen next week.

I'll crank out a PR for it.

well yes, but that needs a copy of listen metadata and other tables.

Pratha-Fish: fyi the mechanize library doesn't execute javascript, which is probably why it doesn't see those interactive fields. maybe there is a way to inject those fields yourself

that refer values based on the new table and not the old one. there is also a datatype difference in listened_at column in old and new schema.

i'll create a copy of those tables for the time being then.

bitmap: Hi, thanks a lot for the ping.

I'll be happy to try out other methods too. Can you recommend me a place to get started? I remember something about mechanize not supporting js a while ago, and some people seemed to recommend selenium instead

do we want to go one step further and insert to both tables in prod for a while?

what if we rent a decent server for a week and isolate the work there?

But ig that would be too big of a shift

aerozol: thanks, I wasn't able to reproduce it earlier but I'll be sure to check again

inserting to both tables seems like a good idea, but may be a bit of work.

zas: I forgot we planned to restart trille today, is it too late for you? Otherwise which time would work for you tomorrow?

that a new server idea also sounds good to me.

yeah i guess it between either more work for zas or me. :)

Pratha-Fish: yes selenium is an option, but maybe there's a way to inject the fields yourself with the mechanize api. otherwise you can make POST request manually with the requests library or something

if we rented a new server, we could make a complete clone, convert the data and then test it properly.

renting another server is not much work.

cool sounds good to me

with all the ansible stuff that zas and atj have been doing I would expect this work to be less than an hour.

ok, when do you want to do this>

?

anytime this week sounds good

bitmap: I see. Let me check if there's still a way to use mechanize in this case.. Or I'll just try to emulate a request instead

this week gets more dicey for me. traveling wed - fri.

but I can get you a server right now.

i see, well next week then i guess

do you need me to do this work? if so, then yes, next week. if not, let me get you a server.

pick one!

P.S. bitmap when do you clock out for today? I'll try my best to wrap up all my queries before that :)

2 x 4.0 TB Enterprise HDD in a mirror configuration, so 8TB?

clock out??

yeah that sounds about right

64GB ram enough>

#clockbrainz

zas: ping

how much disk space do you need lucifer ?

64 should be enough.

2 TB at least would be good.

4TB should be good then?

yup

then we can use a standard raid-1 setup.

Pratha-Fish: I'll be around for a while, might be afk for a bit to eat but I will return :)

cool

That's cool :)

I am thinking of pulling an all nighter today, so I might be around for the next ~5 hours

lucifer: do you want the server to be setup to be a MeB machine?

or do you just want the server and set it up as you like?

mayhem: as long as i can copy data from gaga to it easily. anyway is fine.

because the script will read listens from table on gaga and write to the database on this new machine

so virtual network setup? or are you ok with scp?

or SSH tunnels in that case?

don't think scp will work.

tunnels could work yeah

fine with tunnels if vlan is more work

but it sounds like you prefer the viurtual network, yes>

reosarevok: I reckon I need some sort of special access in Discourse to get user information by external id as described in MBS-11937 ?

https://community.metabrainz.org/u/by-external/... > "You are not permitted to view the requested resource."

MBS-11937: Link to Discourse profile from MB profile https://tickets.metabrainz.org/browse/MBS-11937

yup vlan sounds more robust

let me get zas/atj to chime in.

ok.

Or is it some other url I should be hitting?

cool

reosarevok: I’m always available to help with weekly meeting, email support, account admin or anything else if needed, especially as you take some weeks off.

This one does get the info I need, but tied to username: https://community.metabrainz.org/u/mr_monkey.json

won't you just use ssh over the vlan anyway?

mayhem: Will the new datasets UI require any change to the MB mirror setup which downloads dumps from FTP?

atj: why would that be necessary?

is it db connections rather than file transfer?

yvanzo: no. I think the changes will mostly be limited to wikidoc pages about where to download the data.

atj: db connections.

i don't think the pg docker containers are listening on the vlan yet?

lucifer: ^^

nope they arent

that the was the next thing i was going to talk about, we need a short downtime to restart database to listen on vlan as well

ah, ok, so atj , they will soon. :)

lucifer: prep the PRs needed for that and then we can do that change... tomorrow evening?

let's plan that for tomorrow since you'll travelling wed-fri

yup i'll do that

ok, sounds good.

I'm in a cooking class in the first half of the day, and I suppose I should be finishing in about 26 hours from now.

I hope that timing works.

so we start at 26 hours from now?

can we be flexible about this. I am unsure if I will be exactly available then.

this is a bizarre conversation

sure

how is this different from normal, atj ?

lucifer: ok, no problem then.

i guess i've not been fully indoctrinated yet ;)

mayhem: Ok, I would suggest to add "custom" to MB DB dump’s description - "XZ compressed custom PostgreSQL table dumps" - since it’s neither created with "pg_dump" nor directly loadable with "psql" as people frequently expect it to be, it actually requires MB Server (or alternatively mbdata mirroring scripts) to load it properly.

pratha-fish: jasje: jivte: Are you on the forums? With different usernames? Let me know and I’ll try remember to tag you in the meeting notes (https://community.metabrainz.org/t/notes-from-m...)

aerozol: Hi, I am on the forums with the username "snaekboi"

:D

aerozol: My username is jivteshs20

yvanzo: ah, interesting. can you please summarize that in an email to me? Otherwise I will forget.

Thanks tīmu

zas: tomorrow 4pm UTC would work for me resetting trille.

yvanzo: ok

zas/atj: how much work is it to take a new server from hetzner and get the VLAN setup?

lucifer needs a machine for a few days only, so one of the server auction servers would work great, but we need VLAN setup

reminisces about when akshaaatt started and mayhem told him to use a username with less a’s and t’s. Guess you won that one akshaaatt :P (or am I misremembering?)

mayhem: I completed the setup for data-set-hostler,... (full message at <https://libera.ems.host/_matrix/media/v3/downlo...>)

pixelpenguin: yay!

mayhem: to get vlan up it is a matter of minutes (we need to add it on hetzner interface, and run ansible with proper config for this server)

great, let me order the server then

Or maybe I could do aksha9273939t91739 as a username, aerozol?

I am happy I have the same username on every platform ever😂

Doesn’t look secure enough, needs some symbols I think

XD

zas: server ordered. and you or atj do the minimal user and VLAN setup on that machine once we get credentials?

lets call it quest. atj's choice.

gsoc page finally fixed, lucifer. :)

awesome

ok, really. I need to find food. back in a while.

aerozol: mine is Jasjeet

Thanks!

yippies

jasje: what’s with the replacement about sharedpref?

akshaaatt: https://developer.android.com/topic/libraries/a...

I’m looking at it until we have reasons to migrate

basically it provides sharedPrefs in form of flows

Not looking*

since normal variables used to trigger changes in XML