#metabrainz

[bookbrainz-site] 14insane-22 opened pull request #1083 (03master…unifiedForm-isbn): BB-764: Unified form ISBNs bug https://github.com/metabrainz/bookbrainz-site/p...

@atj right, will have a check for such domains

munishk: i tried and it did work locally for me. can you push what changes you made?

huhridge: IA search responses differ a lot from spotify and youtube, i don't think you would be able to do such seeding. i am not even sure if IA metadata has unique artist, recording ids so you will have to play with their API responses and investigate it

<07munishk> I'll push the changes

rimskii: i realised i had never replied to your proposal. it looks good to me. choose 175 hours when you submit it to the gsoc portal for it since you are doing 2 integrations.

lucifer: yeah that was what i was thinking, a lot of garbage gets returned when it's searched

so how should i proceed?

just do the whole 78 rpm collection?

huhridge: 78 rpm is a good start but there are probably more audio collections in IA.

lucifer: should i just write a crawler for a given collection then?

Sure

there are duplicates and stuff, so i'm sure it's not that straightforward, but it seems like a start

huhridge: okay!

lucifer: there are items on IA that are samples, that is only 30 seconds are available for playing, i should remove these right?

<07munishk> lucifer: pushed changes : https://github.com/munish0838/listenbrainz-serv...

Hi @bitmap, regarding the pub sub solution we discussed earlier, even if we use SQL queries, we still need a way to pass the query result to the URL-saving task right? (unless we are doing the network calls in two different tasks, which I think will be a bit more to handle, since now with rate limiting, we might have to check for concurrent calls from 2 different tasks).

I read more on this, and the best I could come up with was a message passing solution. Also, since `sqlx` crate has methods for listening to events, I think it will be easier to handle.

Also the query you specified can be used while we notify the listener

I've also made the necessary changes in the proposal, and have made the necessary changes.

I've also made the necessary changes in the proposal*

I will keep checking for alternates, but I think the best I could think of was this 😅

as of now*

yellowhatpro: sorry, I'm not sure I completely follow the issue. does rust not allow you to access variables from the scope in which a task is spawned, from inside the task? or doesn't rust have channels to communicate between tasks?

it sounds like you are saying that there will only be a single URL-saving task, but I was assuming you would spawn a separate task for each URL right after you query the database

hello i was looking into LB-1557, and it seems to work in the dev env, but not in production

LB-1557: Spaces in username render as %20 in navigation bar https://tickets.metabrainz.org/browse/LB-1557

yeah rust does have channel for message passing. But isn't that similar to what postgres offers us? I found it similar to what listen/notify does

like it's rendering fine in the dev env, but in production the username is rendering with %20A

@bitmap The way I am planning to use URL-saving task, is to spawn a new thread /green thread for it, and let it listen to incoming URLs

Otherwise I would have to spawn it on each URL while polling

does rust not allow you to access variables from the scope in which a task is spawned, from inside the task?

- Regarding this, I am assuming you are saying if I can't access variables declared in say main function, and I cant use them in the spawned tasks. Yeah we can do so using some Atomic ref counters (Arcs), or Mutex.

I had a good look at rustbook 's concurrency topic today which says: https://usercontent.irccloud-cdn.com/file/LZRQ1...

if you feel that LISTEN/NOTIFY is the simplest solution then you should go for it

I was thinking it would be easier to just spawn a new task every 5 (?) seconds in accordance with the rate limit, and that the task would have access to the URL that was just queried

<05rustynova> If you want Thread A to own the variable, but Thread B to have it too, then Arc<T> allow unmutable access, and Arc<Mutex<T>> allow mutable access. Both Thread A and Thread B will own the variable, and will discard it when they both drop it. However, this isn't always the best case. If you want to send a variable to Thread B, and you won't use it in Thread A anymore, there "mpsc" that allow sending variables to threads

thanks rustynova ;)

That was my initial plan, but I encountered the check_and_retry use case , which basically checks if a url is saved or not, and if it is not, then it will try saving it. Now this has to run differently, so that is why I wanted a message passing solution

@rustynova yeah mpsc creates a channel so that we can send/recv messages. But I found sqlx has same method for notify/listen, which I found same as mpsc, as they both are for async communication

<05rustynova> It's probably better to use the mysqlx solution. But I never dug into that enough to recomend anything.

that's the plan! Yeah, the sqlx version uses tokio-runtime, so maybe some benefit there. But we do have other channels based on tokio so we never know .

<lucifer> "rimskii: i realised i had..." <- sure! thank you so much, will def do that !!