personally, I think that a deleted listens table is one more moving part in the system and that if we can get away without doing it, that would be better (until someone complains I guess)
lucifer
alastairp: we already have that table.
alastairp
oh, well then. that changes things
:)
zas
Yes, we do. One limitation is arising: we cannot use 10.2.2.x network, so it comes as an additional network, but all nodes will be connected through it.
right, I thought you were asking "should we start storing deleted listens so that we can improve stats?" rather than "should we periodically copy our existing deleted listens to spark so that we can improve stats?"
mayhem
sorry
alastairp
lucifer: yeah, just finishing up deleted listens first :)
lucifer
alastairp: ah yes makes sense
alastairp
lucifer: why is track_name gone?
mayhem
it exists in data.
lucifer
it'll be part of jsonb data
BrainzGit
[listenbrainz-android] 14dependabot[bot] opened pull request #26 (03main…dependabot/gradle/androidx.compose.material3-material3-1.1.0-alpha04): Bump material3 from 1.1.0-alpha03 to 1.1.0-alpha04 https://github.com/metabrainz/listenbrainz-andr...
[listenbrainz-android] 14dependabot[bot] opened pull request #27 (03main…dependabot/gradle/androidx.compose.material3-material3-window-size-class-1.1.0-alpha04): Bump material3-window-size-class from 1.1.0-alpha03 to 1.1.0-alpha04 https://github.com/metabrainz/listenbrainz-andr...
[listenbrainz-android] 14dependabot[bot] opened pull request #28 (03main…dependabot/gradle/com.google.android.material-material-1.8.0-rc01): Bump material from 1.8.0-beta01 to 1.8.0-rc01 https://github.com/metabrainz/listenbrainz-andr...
mayhem
this looks ok to me lucifer .
alastairp
and recording_msid is the reverse - pull it out of `data` because we use it often?
lucifer
i intend to use recording msid instead of track name for unique index henceforth.
yes, the schema looks fine to me too. my intuition is that the listened_at and tz_offset is correct for the types of computations that we need to do, but it'd be good to try and actually get the required data from a test db before we go ahead with the full migration
lucifer
makes sense
alastairp
when you do `at time zone 'x'` in an sql query, do you know in what formats x can be?
does it need to be 'Region/City'?
or would we just do `SELECT listened_at + tz_offset as local_time` ?
lucifer
the latter
alastairp
we need to check what timezone pysopg2 will put into the resulting datetime object in this case
lucifer
should be utc i think but need to check.
alastairp
specifically, we should check if it's utc or server time
because `SELECT listened_at + tz_offset at time zone 'UTC' as local_time` is wrong too
anwyay, we can try it out and see what the results are
lucifer
yeah makes sense.
alastairp
I have a feeling that it might be better to extract listened_at always as UTC, and update the listen object with a datetime at the correct timezone (using `.astimezone()`) in python - just to avoid getting an object out of the database with an attached tz, but then it actually being the date at a different tz
but as I said, we can test
lucifer
i don't think we can get a correct timezone without storing timezone instead of offset.
alastairp
yes, I suspect you're right
lucifer
or i guess we could make a table to map INTERVAL 5h 30 mins to UTC +530 time zone and use that
alastairp
but I think that's fine, I have no problem with trying it out and seeing what the code looks like
atj
what about DST changing offsets?
lucifer
the client would submit an offset so we are hoping that the client is applying DST on its end.
atj
ah ok
jivte_ has quit
alastairp
atj: this is why we settled on storing +0100 instead of 'BST' or 'Europe/London'
atj
yeah, i see the logic, as the api client will know the offset at the time of the listen
alastairp
yep
(also, I tried to work out how to automatically calculate 'Europe/Madrid' from python without user input... not easy
monkey
Nothing is ever easy when it comes to time, is it?
alastairp
monkey: let me take some time and get back to you on that...
monkey
And I guess we should be thankful that we don't have to handle relativistic time (yet)
alastairp
well, we allow people to import listens from the past
lucifer
what about listens from the future
atj
"how to automatically calculate 'Europe/Madrid' from python" you mean going from offset -> zone name?
antlarr has quit
alastairp
atj: yes
jivte_ joined the channel
antlarr joined the channel
atj
not sure that's possible, given each offset may match multiple timezones?
alastairp
atj: right, and another motivation to only use UTC hour offsets instead of named time zones
Sophist_UK has quit
antlarr has quit
antlarr joined the channel
antlarr has quit
antlarr joined the channel
vibhoo_24 joined the channel
bitmap
reosarevok: yeah, that doc is out of date, we no longer pass $c as a prop (you have to use react context or getCatalystContext if not inside a component)
reosarevok
Thought so. I'll update it
antlarr has quit
antlarr joined the channel
bitmap
re MBS-8668 do you have an example of which error no longer triggers?
note: we are deploying vswitch changes to all nodes right now, a new IP in 10.10.10.x network will be added
antlarr has quit
antlarr joined the channel
agatzk has quit
antlarr has quit
TOPIC: MetaBrainz Community and Development channel | MusicBrainz non-development: #musicbrainz | BookBrainz: #bookbrainz | Channel is logged; see https://musicbrainz.org/doc/IRC for details | Agenda: Reviews
antlarr joined the channel
atj
vswitch changes applied and nothing appears to be on fire
zas
(yet) ;)
alastairp
I can change that if you want?
jasje_ joined the channel
antlarr has quit
antlarr joined the channel
jasje_
akshaaatt: in EntityTestUtils, what is the loadResourceAsString function?
akshaaatt: I'll be afk for a while, won't be active for a week or so. I'll try to wrap up work on tickets for aerozol.
antlarr has quit
agatzk joined the channel
antlarr joined the channel
serial-ata joined the channel
BrainzGit
[listenbrainz-server] 14alastair opened pull request #2338 (03master…canonical-release-connection): Generate canonical release redirect only in the mb schema https://github.com/metabrainz/listenbrainz-serv...
alastairp
lucifer: ^ I think this is the only change we need for the mapper bug from the other month
BrainzGit
[listenbrainz-server] 14alastair closed pull request #2288 (03master…canonical-data-mb-conn): Remove lb_conn option from canonical musicbrainz data dump https://github.com/metabrainz/listenbrainz-serv...
People up: atj, bitmap, lucifer, monkey, reosarevok, alastairp, yvanzo, mayhem, zas, akshaaatt, CatQuest, Pratha-Fish, yellowhatpro, riksucks, Freso – anyone else who wish to give review, let me know ASAP. :)
atj: Go!
Sophist-UK joined the channel
atj
hi
CatQuest
fish
Pratha-Fish
🐟
aerozol
Mōrena! I’m here too Freso :)
CatQuest
:D
vibhoo_24
Freso: count me in
Pratha-Fish
^_^
atj
last week I worked with zas to test deploying a Hetzner vSwitch to all servers so they can communicate securely
as part of that work i simplified the netplan and shorewall configuration in ansible so it should be a bit easier to understand
we just added the vSwitch VLAN to all servers today and no issues have been reported (so far)
aerozol: go
monkey
vWell done
aerozol
Kia ora! I had a lovely break and finally finished Hades (the game)
mayhem
atj: thanks for working on that, thats going to be hugely helpful for us.
aerozol
Apart from that I finally tidied some wiki pages (esp the userscripts one), kept on top of discussions, and have been doing lots of looking at other YIM-type stuff to inform the next one. Hoping to get onto the design early early