because it kept on saying that they had really high similarity
ruaok
can we frame this in terms of "this was done, we solved the theoretical aspects, but now we need to scale it"?
alastairp
absolutely
ruaok
that sounds like a perfect gsoc project, no?
alastairp
scalability is a potential issue
ruaok
it is *the* issue, no?
alastairp
we had no machine here which was fast enough to really get an idea about how difficult the issue was
we were running with about 25% of the database, and it was taking ages to do stuff (but our infrastructure isn't as powerful as the hetzner dedicated servers)
ruaok
I suspect that we'll need to solve this using spark.
rsh7
iliekcomputers: got time to rebase the integration branch?
alastairp
right, so it depends on what our goals with this are, and how we want to query it
if we can wait a few seconds or tens of seconds, BQ or spark are ideal candidates
ruaok
or we rent a stupidly big cloud instance and run it on that periodically.
my goals is to feed more data into training recommendation engines.
so batches are fine.
alastairp
and it's possible that spark may even be better, because we're not constrained by cube's requirement that the distance metrics are linear
right, so, it also depends on what the usecase is
clustering? or selection of similarity from a single example instance?
ruaok
this is the usecase I want to solve.
I'm unsure of how to get there, but I am sure that I want a track similarity mapping in the end
alastairp
I'm not sure if it's responsible to select every item in the database and independently caculate its similarity with every other track
alastairp: another favor please... can you recommend some papers that combine user behavioral data with acoustic data in order to build recommendation engines?
didn't dimi do his PhD on that?
alastairp
mmm, good question
yeah, I think you're right
Gabriel did stuff on behaviour data I think, I'm not sure if he included acoustic data
ruaok
that is the next thing we need to understand.
yea, we have the CF in spark to do that. but dimi taught me that we need AB as well.
reosarevok registered for the "do you know Estonian laws" exam
reosarevok
One step closer to being a citizen, yay
CatQuest
reosarevok: even more school though.
perpetual student you
reosarevok
Nah
djinni` joined the channel
CatQuest
but yay!
reosarevok
You basically just go there and are given the Constitution and stuff and just need to show you can figure it out :p
So I don't think it actually requires any studying
Might like read it once ahead of time just in case, but
CatQuest
yea beter be safe thna srry :P
better sorry*
gr0uch0mars joined the channel
amCap1712
hi gr0uch0mars
Can you explain the way you referred in the PR comment to organize code
gr0uch0mars
hi. yes I was going to look for a good post to link here, but meanwhile I referred to organizing certain files into features
like all files of the presentation-layer of Artist together: viewModel, activity, adapters, fragments…
that way, it's easier to have a quick preview of what does the app offers (something related to Artist), and there's “only” one place if you have to touch the code
Here is a link about Clean Architecture (way beyond simply “grouping features”) that, although difficult to implement in its totality, it worth reading about: https://fernandocejas.com/2018/05/07/architecti...
amCap1712: take a look at the post and share your thoughts. Working with a good architecture is as important as making code work (although not urgent)
amCap1712
thanks gr0uch0mars
ahmedkrmn has quit
gr0uch0mars
amCap1712: other question I was thinking yesterday. Is there a design for the app UI? Or can we work on an improved design?
amCap1712
gr0uch0mars: we can work on improved design
gr0uch0mars
great. Let me think of some ideas and I'll share them. Meanwhile we can work on presenting the data retrieved from the API in an “ordered” manner, like you are doing for Artists
Freso
Hm. Does it make sense to continue to list AcousticBrainz on /supporters with it also being listed on /projects?
amCap1712
ok great gr0uch0mars
Freso
It feels a bit like "oh, hey, we support ourselves!", no?
UPF is already listed on their own.
alastairp
Freso: I think that "supporters" is directly linked to "has an API key to download the database"
which is why AB is on supporters
Freso
Could be. Just looks a bit odd and self‐congratulatory to me is all. :)
alastairp
maybe we can hide the account from the page if you really want, but I'm not sure if it's needed