yvanzo: bitmap , It just hit me that if we are creating triggers, then we are expecting it to log the `internet_archive_urls` table right?
But since we have 2 tables, edit_data and edit_note to apply the trigger on, and we will have to tranform the rows, how we are gonna do that using trigger.
I suppose the advantage was, that it would still be logging to the `internet_archive_urls`, but we still need transforming the rows from the 2 tables before that right?
* yvanzo: bitmap , It just hit me that if we are creating triggers, then we are expecting it to log the `internet_archive_urls` table right?
But since we have 2 tables, edit\_data and edit\_note to apply the trigger on, and we will have to tranform the rows, how we are gonna do that using trigger.
I suppose the advantage was, that it would still be logging to the `internet_archive_urls` when the container is down, but we still need transforming the rows from the 2 tables before that right?
dseomn_ joined the channel
dseomn has quit
dseomn_ is now known as dseomn
mara42 has quit
mara42 joined the channel
aerozol[m]
Jade: sorry about the late reply, my week has been absolute chaos. Are you happy to meet and have a chat about the project on any of: Tuesday evening, Wednesday morning, Thursday evening, Friday morning
This assumes you are in Europe somewhere! I am in New Zealand so I have shifted my available times forward 12 hours, if you are somewhere else I can work out when I’m available in your times :D
FYI I am happy to video chat, but if you are more comfortable we can just have a text meeting
pranav: Liking the video of load more in action! Don’t forget to replace the 100% blacks - and let me know if you think the colours need tweaking
pranav[m]
aerozol: yup will implement the colors today, wanted to get all elements first
aerozol[m]
All good, no rush, just making sure you hadn’t forgotten. And curious to see what it looks like!
yvanzo[m]
Hi yellowhatpro yes the transformation is still needed and that is the focus of this week.
yellowhatpro[m]
Right, so that has to be done using postgres only right?
And it all has to be defined in the trigger itself
yvanzo[m]
Not necessarily, triggers can just store table name and row id.
yellowhatpro[m]
so does that mean, the transformations, or getting the URLs from the tables, have to be done somewhere between listen/notify
given we have the row ids and table names
yvanzo[m]
what with listen/notify?
yellowhatpro[m]
so the idea was to run notify on the internet_archive_urls table, and fetch x URLs, and send it to the listener, which will archive the URLs
yvanzo[m]
it means that an intermediate table would be needed before getting to the *_urls table indeed.
yellowhatpro[m]
yeah right
I was just thinking, is there any way we can just rip the URLs from edit* tables, while we are dumping them in internet_archive_urls?
But it seems a bit tricky, given the structure of edit_data, so it seems an intermediate table will be needed.
yvanzo[m]
It would be tricky to implement URLs extraction using Postgres functions, and there would be a number of drawbacks: (1) It would run in our main Postgres instance and if something goes wrong it could be more tedious to turn off. (2) It wouldn’t be able to process external sources of data in the future.
yellowhatpro[m]
But if we create a new table where we dump row ids and table names, we still need to somehow read this table and then make transformations, and then send the result to `internet_archive_urls` table.
Doesn
* But if we create a new table where we dump row ids and table names, we still need to somehow read this table and then make transformations, and then send the result to `internet_archive_urls` table.
Doesn't it seem like the polling task with one extra step?
yvanzo[m]
Yes, and with the ability to handle non-linear rows.
yellowhatpro[m]
ohh cool. (what's non-linear rows btw in this context ?? sorry for nub doubt 😅)
yvanzo[m]
When an edit is added, it’s just a new row. When a row is updated, it’s row id stays the same. For example, if we want to archive external links from editor’s bio.
Polling would just consider new rows, not updated rows.
Instead of polling, I would check if there is anything else we can do though
Kladky joined the channel
Jade[m]
[@aerozol](https://matrix.to/#/@aerozol:matrix.org) Don't worry about it! 😄
Yeah, I'm in the UK - so 11 hours behind you. Tuesday evening sounds good to me?
And I'm happy to video chat - just send me the link
yvanzo[m]
yellowhatpro: Don’t get too distracted with moving to triggers (in the long term), transformation(s) can be implemented/tested/documented separately already.
discordbrainz
<12Aerozol> @jade: Sorry, I mixed up the times - I can do your Monday evening, anytime after 9pm, or Tuesday morning, 10-12am (I have a few things on sorry)
reosarevok: yup, I can lengthen variable names if you are fine with that too.
reosarevok[m]
Go for it :)
mara42 joined the channel
lucifer[m]
<rimskii[m]> "also lucifer I almost finished..." <- awesome 💯
mara42 has quit
mara42 joined the channel
yvanzo[m]
<reosarevok[m]> "Go for it :)..." <- Pushed. You can’t review your own PR but you can give a feedback still. :)
mara42 has quit
mara42 joined the channel
pite joined the channel
pranav[m] uploaded an image: (1217KiB) < https://matrix.chatbrainz.org/_matrix/media/v3/download/matrix.org/AdGVOBoQHkFpuKlYxaJtINca/Screenshot%202024-06-14%20at%204.59.04%E2%80%AFPM.png >
pranav[m]
aerozol[m] (IRC): here following means from user perspective or user perspective? For eg -> If I am viewing ur user page, shud i see follow or following if I have followed them or not or you?
<yellowhatpro[m]> "so it's pretty much the same ig,..." <- > <@yellowhatpro:matrix.org> so it's pretty much the same ig, just a step added to the polling task.... (full message at <https://matrix.chatbrainz.org/_matrix/media/v3/...>)
yellowhatpro[m]
yeah right, extracting URL, filtering different URLs would be pain indeed.
I'll just create a different branch for it then and make it a side-quest
zas has quit
zas joined the channel
mara42 has quit
bitmap[m]
sure, I'd suggest not introducing yet another intermediate table just so you can use triggers for edit_note though. polling is the simpler route in that case
mara42 joined the channel
pranav[m] uploaded an image: (1244KiB) < https://matrix.chatbrainz.org/_matrix/media/v3/download/matrix.org/WtXBDwECqwLBJrzAUVhCfOJg/Screenshot%202024-06-14%20at%207.43.08%E2%80%AFPM.png >
pranav[m]
aerozol: Just one suggestion, could we use the existing similar user component which looks as above instead of the new one? Or should I change it everywhere?
yvanzo[m]
bitmap: Can test.mb.o be updated after beta (+ MBS-13579) again or does it require any special attention when updating?
(Or is the PR addressing this ticket ready to merge?)
bitmap[m]
it's not ready but I haven't had time to fix it yet, you can feel free to overwrite test.mb
yvanzo[m]
Thanks, will do.
pranav[m]
lucifer: Could u lemme know the web socker URL i need to connect to for getting the listening now? I am server rejected WebSocket connection: HTTP 400.. I must be using the wrong url
lucifer[m]
pranav: how are you trying to connect?
yvanzo[m]
I’m keeping your current patch for that ticket on top of beta for the test branch.
