Gentlecat: But using classes made things easier to handle in the code.
2016-07-20 20201, 2016
armalcolite
alastairp: not sure what do u mean? How are you exactly importing the data?
2016-07-20 20218, 2016
armalcolite
Gentlecat: I have to constantly access the various values of tokens in the code (ex, token_id, api_key from which token was created, user_id who is attached to the token)
2016-07-20 20257, 2016
alastairp
armalcolite: to extend what Gentlecat is saying, in acousticbrainz and listenbrainz our policy has been to pass around objects as dictionaries
2016-07-20 20207, 2016
alastairp
rather than creating an explicit class to carry the data
2016-07-20 20212, 2016
Gentlecat
yeah, I see what you mean
2016-07-20 20220, 2016
alastairp
it's true that classes help encapsulate data, but we haven't been doing that
2016-07-20 20227, 2016
Gentlecat
I've been actually thinking about this
2016-07-20 20252, 2016
Gentlecat
there are some benefits, but it's easy to make it unnecessarily complex
2016-07-20 20258, 2016
alastairp
I suspect Gentlecat's comment is pointing out that we generally don't do this, and so adding a second way of working with data can make it complicated to understand what's going on
2016-07-20 20209, 2016
armalcolite
and, making the classes static reduced the number of calls that i have to make to achieve something
2016-07-20 20217, 2016
Gentlecat
it's easier to understand when you just see what comes in and what comes out
2016-07-20 20227, 2016
Gentlecat
kind of functional-ish style
2016-07-20 20233, 2016
alastairp
although, to add to this, we didn't have have significant database access before armalcolite added his stuff (compared to e.g. acousticbrainz)
2016-07-20 20257, 2016
armalcolite
Though the logic in the db codes is pretty naive
2016-07-20 20211, 2016
alastairp
if we get to this stage, I wonder if going full-sqlalchemy ORM is a way to go
2016-07-20 20214, 2016
armalcolite
they have no authentication, so complexity is not much
2016-07-20 20216, 2016
Gentlecat
(I have CB to compare with, though it's deep in ORM hell)
2016-07-20 20220, 2016
alastairp
ah.
2016-07-20 20230, 2016
alastairp
I guess you have counter examples there then :)
2016-07-20 20241, 2016
alastairp
interesting
2016-07-20 20243, 2016
Gentlecat
it takes me a lot of time to understand some parts now that I'm coming back to it
2016-07-20 20247, 2016
alastairp
right
2016-07-20 20252, 2016
Gentlecat
nothing beats the good old SQL query
2016-07-20 20258, 2016
alastairp
I agree, I prefer to write and understand sql
2016-07-20 20214, 2016
alastairp
though there are some ways of using the ORM which are almost a direct translation of the sql
2016-07-20 20234, 2016
Gentlecat
yeah, you have to be very careful though
2016-07-20 20235, 2016
armalcolite
\me agrees with Gentlecat comlpetely
2016-07-20 20242, 2016
alastairp
armalcolite: regarding your question about importing
I have a quick question which i'm hoping for some help on
2016-07-20 20218, 2016
alastairp
mellowdoubt: hi! ask away!
2016-07-20 20236, 2016
alastairp
if it's specifically related to adding or editing data on the musicbrainz website you might have more luck in #musicbrainz
2016-07-20 20250, 2016
armalcolite
yes, i extract the epoch to compare on the show listens page.
2016-07-20 20257, 2016
alastairp
armalcolite: did you copy this query directly from the casandra code?
2016-07-20 20207, 2016
armalcolite
yes
2016-07-20 20211, 2016
mellowdoubt
if an artist uses an ampersand (&) symbol in there name as opposed to calling themselves x and x should that be changed on the database
2016-07-20 20228, 2016
armalcolite
This shows the listens in the range specified on the listens page.
2016-07-20 20243, 2016
mellowdoubt
my specific example is the band 'jessica and the fletchers'
2016-07-20 20249, 2016
alastairp
psycopg2 has some nice features where if you select a timestamp object, it will create a python datetime object with the correct timezone
2016-07-20 20220, 2016
armalcolite
yes, but the show listens logic needed to be changed then.
2016-07-20 20240, 2016
alastairp
so, just because the cassandra code selected unix timestamps, and converted them in python, it doesn't mean we should do the same in postgres
2016-07-20 20246, 2016
alastairp
sure! that's part of the migration
2016-07-20 20204, 2016
mellowdoubt
on most releases they tend to use the '&' so should it be changed to this or should the existing name be left alone?
2016-07-20 20252, 2016
alastairp
mellowdoubt: reosarevok may be able to help you with that, but I think you'll get better help in the #musicbrainz channel. In here we are mostly talking about developement, rather than use of the database
2016-07-20 20216, 2016
alastairp
sorry I didn't manage to review the code before we merged it, I may have noticed it earlier, but I think we should make this change
2016-07-20 20226, 2016
mellowdoubt
ah, sorry <alastairp> i'll try and switch channels :)
2016-07-20 20239, 2016
alastairp
no worries! many people are in both channels, so you'll get a response anyway
2016-07-20 20245, 2016
alastairp
but perhaps the other one will get you one faster
2016-07-20 20201, 2016
mellowdoubt has left the channel
2016-07-20 20208, 2016
alastairp
I'm thinking back to our original API specification. I see we use timestamps in query parameters and return values. I can't remember why we decided to do that instead of formatted dates
2016-07-20 20219, 2016
alastairp
and if it was a design decision or a result of what cassandra forced us to do
2016-07-20 20239, 2016
armalcolite
actually, from what i remember
2016-07-20 20256, 2016
armalcolite
cassandra used partitions and you used this info to separate listens into partitions.
2016-07-20 20207, 2016
armalcolite
(i am not completely sure)
2016-07-20 20242, 2016
alastairp
yes, but internally I don't know if we stored the listen date just as a number (representing the timestamp) or as a date object with a semantic meaning
2016-07-20 20214, 2016
alastairp
our tools let us treat listen dates as real objects, so I believe we should take advantage of this
2016-07-20 20228, 2016
armalcolite
we used to store only the timestamps everywhere.
2016-07-20 20250, 2016
armalcolite
hence, i needed to introduce 'extract(epoch from ts)' to mimic this feature.
2016-07-20 20200, 2016
armalcolite
while adding postgres
2016-07-20 20221, 2016
armalcolite
using timestamps in the query params is much better than using formatted dates?
2016-07-20 20232, 2016
alastairp
yeah, I understand. it was a reasonable choice to make when replicating the behaviour of cassandra
2016-07-20 20247, 2016
alastairp
I don't have a strong opinion on either option
2016-07-20 20225, 2016
armalcolite
the problem with the current code is that fromtimestamp() returns localtime
2016-07-20 20239, 2016
alastairp
right
2016-07-20 20247, 2016
armalcolite
if we can just make that part to return utc() then our problem is solved.
2016-07-20 20249, 2016
alastairp
utcfromtimestamp doesn't :)
2016-07-20 20207, 2016
armalcolite
awesome!
2016-07-20 20206, 2016
alastairp
however, to me this still sounds like a solution to a problem that we shouldn't have in the first place
2016-07-20 20220, 2016
alastairp
my preference would be for Listenstore to return datetime objects
2016-07-20 20235, 2016
alastairp
we can convert these to timestamps when we need to display them
2016-07-20 20252, 2016
armalcolite
i thought of that solution earlier.
2016-07-20 20257, 2016
armalcolite
my idea was:
2016-07-20 20210, 2016
armalcolite
Using the built-in DB feature to extract the timestamp would be much more fast than iterating and than converting timestamp for all of them.
2016-07-20 20247, 2016
armalcolite
but i now think it is just a handful of listens and it wont make much difference.
2016-07-20 20259, 2016
alastairp
I don't understand what you're saying
2016-07-20 20233, 2016
alastairp
don't consider it in terms of the computation time required to perform an action. no matter what we do we are not going to run into scalability problems with this example
2016-07-20 20256, 2016
diana_olhovyk_ has quit
2016-07-20 20200, 2016
alastairp
instead we should use the tools which make our task easier for us
2016-07-20 20206, 2016
alastairp
time is *hard*
2016-07-20 20209, 2016
alastairp
it is very hard
2016-07-20 20221, 2016
armalcolite
I meant using extract(epoch from ts) would be faster than using datetime conversion to get timestamp.
2016-07-20 20246, 2016
alastairp
in fact, we've already shown that if we don't use the tools available to us, we cause problems in our code
2016-07-20 20210, 2016
armalcolite
sure! i'll keep this in mind.
2016-07-20 20214, 2016
alastairp
so please use the timestamps from postgres
2016-07-20 20229, 2016
alastairp
the dates, I mean
2016-07-20 20236, 2016
armalcolite
ah, ok.
2016-07-20 20254, 2016
armalcolite
i'll do it now.
2016-07-20 20214, 2016
alastairp
the import of now listening songs is probably more important
2016-07-20 20248, 2016
alastairp
please make a ticket for it too, so that we can follow what you're working on
