#metabrainz

see the first one, @attr: {nowplaying: "true"}

there is no timestamp for this song

keep what together? methods?

yes!

I guess ruaok and I always scrobble data, and you didn't when you were testing

you should skip that one in the import

oh

i have a ticket for this.

Gentlecat: Yes

I would say that this is more important than almost anything - as we can't test with it like this currently

but that's what modules are for

https://github.com/metabrainz/listenbrainz-serv... for example

Gentlecat: earlier i had all of them in the single file, but just yesterday (on alastairp's) suggestion i made them modular.

just to clarify, I'm talking about these modules https://docs.python.org/3/tutorial/modules.html

Gentlecat: But using classes made things easier to handle in the code.

alastairp: not sure what do u mean? How are you exactly importing the data?

armalcolite: to extend what Gentlecat is saying, in acousticbrainz and listenbrainz our policy has been to pass around objects as dictionaries

rather than creating an explicit class to carry the data

yeah, I see what you mean

it's true that classes help encapsulate data, but we haven't been doing that

I've been actually thinking about this

there are some benefits, but it's easy to make it unnecessarily complex

I suspect Gentlecat's comment is pointing out that we generally don't do this, and so adding a second way of working with data can make it complicated to understand what's going on

and, making the classes static reduced the number of calls that i have to make to achieve something

it's easier to understand when you just see what comes in and what comes out

kind of functional-ish style

although, to add to this, we didn't have have significant database access before armalcolite added his stuff (compared to e.g. acousticbrainz)

Though the logic in the db codes is pretty naive

if we get to this stage, I wonder if going full-sqlalchemy ORM is a way to go

they have no authentication, so complexity is not much

(I have CB to compare with, though it's deep in ORM hell)

ah.

I guess you have counter examples there then :)

interesting

it takes me a lot of time to understand some parts now that I'm coming back to it

right

nothing beats the good old SQL query

I agree, I prefer to write and understand sql

though there are some ways of using the ORM which are almost a direct translation of the sql

yeah, you have to be very careful though

\me agrees with Gentlecat comlpetely

armalcolite: regarding your question about importing

and I wasn't at the time

I put my username in http://localhost:8080/user/import?lastfm_userna... and click "import"

Gentlecat: yeah. I think everyone learns something the first time they naievly use an orm :D

me too

This is what i did.

with my username or with yours?

oh, with mine.

I remember this error

alastairp: http://tickets.musicbrainz.org/browse/LB-89

armalcolite: can you treat that as an urgent ticket?

we definitely cannot release without this part working - and it was affecting ruaok's testing too

ok. (But i have some open code for api_compat that i need to finish first)

You can use other channel for testing, for now.

yeah, sure. don't drop everything. next time you're moving

what other channel? I need to get data into my db to test

nasasie

it has a lot of listens!

right, but that's only going to work as long as CatCat isn't listening to anything at the moment!

I managed to import by stopping my music and then importing

yeah. :P

But CatCat is not that frequent song lover.

Everytime i used it, he was not listening. :P

[listenbrainz-server] pinkeshbadjatiya opened pull request #90: Add support for getting scrobbles from api_compat via ws.audioscrobbler.com (master...update-ngnix-config-for-ip-mapping) https://github.com/metabrainz/listenbrainz-serv...

armalcolite: I'm looking at https://github.com/metabrainz/listenbrainz-serv...

I think there's another more annoying bug here which isn't quite fixed

I remember Gentlecat and I spent a *long* time trying to get it to work correctly

Yippee, build fixed!

Project listenbrainz-server build #111: FIXED in 1 min 12 sec: https://ci.metabrainz.org/job/listenbrainz-serv...

alastairp: oh, can u share more info.

I'm looking at it now

one sec

was related to the way time zone was specified iirc

I have a row in the database with a date of 2016-07-20 14:11:40+00

which is correct, that's when I listened to the track

however, in the template it has been rendered as 2016-07-20T16:11:40Z

which is the time in my local timezone (or perhaps the timezone of the database server. I need to check this), with a Z added at the end

https://github.com/metabrainz/listenbrainz-serv... looks suspicious to me

what's the type of timestamp that comes out of psycopg?

datetime?

yes

hello! this is my first time here :)

then what does datetime.fromtimestamp(int(listen.timestamp)) do?

not quite: https://github.com/metabrainz/listenbrainz-serv...

I have a quick question which i'm hoping for some help on

mellowdoubt: hi! ask away!

if it's specifically related to adding or editing data on the musicbrainz website you might have more luck in #musicbrainz

yes, i extract the epoch to compare on the show listens page.

armalcolite: did you copy this query directly from the casandra code?

yes

if an artist uses an ampersand (&) symbol in there name as opposed to calling themselves x and x should that be changed on the database

This shows the listens in the range specified on the listens page.

my specific example is the band 'jessica and the fletchers'

psycopg2 has some nice features where if you select a timestamp object, it will create a python datetime object with the correct timezone

yes, but the show listens logic needed to be changed then.

so, just because the cassandra code selected unix timestamps, and converted them in python, it doesn't mean we should do the same in postgres

sure! that's part of the migration

on most releases they tend to use the '&' so should it be changed to this or should the existing name be left alone?

mellowdoubt: reosarevok may be able to help you with that, but I think you'll get better help in the #musicbrainz channel. In here we are mostly talking about developement, rather than use of the database

sorry I didn't manage to review the code before we merged it, I may have noticed it earlier, but I think we should make this change

ah, sorry <alastairp> i'll try and switch channels :)

no worries! many people are in both channels, so you'll get a response anyway

but perhaps the other one will get you one faster

I'm thinking back to our original API specification. I see we use timestamps in query parameters and return values. I can't remember why we decided to do that instead of formatted dates

and if it was a design decision or a result of what cassandra forced us to do

actually, from what i remember

cassandra used partitions and you used this info to separate listens into partitions.

(i am not completely sure)

yes, but internally I don't know if we stored the listen date just as a number (representing the timestamp) or as a date object with a semantic meaning

our tools let us treat listen dates as real objects, so I believe we should take advantage of this

we used to store only the timestamps everywhere.

hence, i needed to introduce 'extract(epoch from ts)' to mimic this feature.

while adding postgres

using timestamps in the query params is much better than using formatted dates?

yeah, I understand. it was a reasonable choice to make when replicating the behaviour of cassandra

I don't have a strong opinion on either option

the problem with the current code is that fromtimestamp() returns localtime

right

if we can just make that part to return utc() then our problem is solved.

utcfromtimestamp doesn't :)

awesome!

however, to me this still sounds like a solution to a problem that we shouldn't have in the first place

my preference would be for Listenstore to return datetime objects

we can convert these to timestamps when we need to display them

i thought of that solution earlier.

my idea was:

Using the built-in DB feature to extract the timestamp would be much more fast than iterating and than converting timestamp for all of them.

but i now think it is just a handful of listens and it wont make much difference.

I don't understand what you're saying

don't consider it in terms of the computation time required to perform an action. no matter what we do we are not going to run into scalability problems with this example

instead we should use the tools which make our task easier for us

time is *hard*

it is very hard

I meant using extract(epoch from ts) would be faster than using datetime conversion to get timestamp.

in fact, we've already shown that if we don't use the tools available to us, we cause problems in our code

sure! i'll keep this in mind.

so please use the timestamps from postgres

the dates, I mean

ah, ok.

i'll do it now.

the import of now listening songs is probably more important

please make a ticket for it too, so that we can follow what you're working on