#metabrainz

[@holycow23:matrix.org](https://matrix.to/#/@holycow23:matrix.org) what postgres queries and which metadata files?

lucifer[m]: Let's say I want to write a postgres query to fetch the listens with their year of release so I need to use the metadata from the HDFS query and for listens I am using the function as defined in the gist

[@holycow23:matrix.org](https://matrix.to/#/@holycow23:matrix.org) you won't need to write a postgres query, it would be a spark sql query. the distinction is also important because for some things the syntax of spark sql is different from postgres. for the hdfs metadata you can read the dataframe as in the example gist using dataframe api or spark sql query.

You shouldn't need to check the MB dumps in any case. Any metadata that you need should come from MB db. The format of MB dumps is different from MB DB so if you use the first it would create issues.

I'll check the release year YIM queries in a while and confirm if all the data you need for that one is available or not.

I have added release_metadata_cache to the sample dumps already. (Might need to update the codebase/container to import it successfully though).

<lucifer[m]> "You shouldn't need to check..." <- Where can I see the formt of the MD db?

Alternatively you can connect to the database on wolf using ssh port forwarding and browse the database with your choice of tool.

But are all the tables present cause I could locate... (full message at <https://matrix.chatbrainz.org/_matrix/media/v3/...>)

* But are all the tables present cause I could locate... (full message at <https://matrix.chatbrainz.org/_matrix/media/v3/...>)

This is not MB db.

This is the list of metadata files imported into spark

sorry this is the metadata part right?

Yeah my bad

So, I gotta use this right?

No need for MB db?

Yes this data already exists and you can use it as needed

If there is some metadata that doesn't in these files then you would need to write queries to create these files by reading data from MB db

I'll update the setup on wolf later today to add release_metadata_cache to the table.

*to spark.

lucifer[m]: I didn't get this?

There is one more metadata file available in production that is missing from your local setup because I only added it to sample dumps last week.

I'll update your spark setup to add that file.

reosarevok: a while back we talked about the continued need to mass mail auto-editors for election notifications, even in a post SSO implementation future...... (full message at <https://matrix.chatbrainz.org/_matrix/media/v3/...>)

not urgent, just a few thoughts i had while heading towards bed :)

lucifer: I wrote this small script... (full message at <https://matrix.chatbrainz.org/_matrix/media/v3/...>)

* lucifer: I wrote this small script... (full message at <https://matrix.chatbrainz.org/_matrix/media/v3/...>)

<julian45[m]> "reosarevok: a while back we..." <- > <@julian45:julian45.net> reosarevok: a while back we talked about the continued need to mass mail auto-editors for election notifications, even in a post SSO implementation future...... (full message at <https://matrix.chatbrainz.org/_matrix/media/v3/...>)