#metabrainz

also, did you see my message above from earlier regarding APIBadRequests?

ah, I glanced at it

this is the one where the spotify importer is raising it and causing it to trigger to sentry?

yes

in that case I think that the spotify importer should catch the exception and silently continue

yeah makes sense

oh, I just had a thought, did you add the ability to ignore specific exceptions in sentry?

I can't remember if we finished that discussion

no, i haven't yet.

ok, no problem. it should be no different with sentry-python compared to what it was previously doing, but that's a future improvement

do we have a ticket for it? I'll make one if not

afair, we decided to pass exception during initialization to sentry.

no, we don't have a ticket for that.

yes, that's what I recall too

also, we wanted to see how these new integrations work out and if we don't need to add that at all

i think we won't be needing it soon because the APIBadRequests are now probably figured out.

yeah, if the badreuests are due to the spotify importer then maybe it's doing the right thing

regarding the null entry in postgres... we had similar issues in AB because we store data in json

let me find how we fix it - I don't know if we look for null, or catch a specific exception from the db

UntranslaeCharacterError I think it is.

my original thinking was to catch it and sanitize the input and retry. but catching and raising a 400 is fine too.

alastairp, sentry-sdk recently attained 1.0.0. when I had tested it, it was 0.20.*. which one should i upgrade to ?

oh whoops. didn't think of that

let's just use the old version

👍

_lucifer: what's the python-sentry config item? ignore_errors or before_send?

I'm finding conflicting information

ignore_errors

you could use before_send to emulate ignore_errors behaviour

BU-43

BU-43: Add a way for apps to tell sentry to not report on specific exceptions https://tickets.metabrainz.org/browse/BU-43

the issue I found was that a user was doing that and wanted a built in method for that. the sentry devs had replied that there is ignroe_errors but it was probably not documented.

-> lunch, back in ~15m

👍

[listenbrainz-server] amCap1712 opened pull request #1367 (master…sentry-upgrade): Upgrade brainzutils and sentry_sdk https://github.com/metabrainz/listenbrainz-serv...

_lucifer: I added one change to your PR : https://github.com/metabrainz/listenbrainz-serv...

if that looks ok to you, I'll merge and test.

yup, lgtm.

kewl.

[listenbrainz-server] mayhem merged pull request #1366 (master…listen-threshold): LB-851: Calculate similar users only for users who have a minimum number of listens https://github.com/metabrainz/listenbrainz-serv...

!m _lucifer

You're doing good work, _lucifer!

_lucifer: on leader you were smart to save the spark_config.py, but we didn't save spark_config.sh . :(

lets hope that there were no changes from the default version. heh.

oh! i forgot about. 🤞

if it works fine now, i'll add spark_config.sh as well to docker-server-configs

ok, trying now.

[metabrainz.org] amCap1712 opened pull request #359 (master…sentry-upgrade): Upgrade brainzutils and sentry-sdk https://github.com/metabrainz/metabrainz.org/pu...

hello, I'm back now

_lucifer: if you're working on LB, I could do AB

hello formerly alastairp. nice to meet you back!

ruaok big on the dad jokes these days

alastairp, i am done with LB, MeB. doing CB now. sure, thanks!

ah, I see the LB one. so many brainzgit reports from you I can't keep track of them

_lucifer: see sentry for a hiccup on the new thresholding.

ruaok, i see. its confusing the field with the method. i'll change the field name to something else.

yep, seems like it.

alastairp: do we have internal API endpoints in LB?

what do you mean by internal? for the website to use, or this concept that I wanted to introduce for the stats?

i think using as a string could work though. should i just hotfix in leader? if it works then i'll open a PR.

something not officially supported, but could be used by javascript in our pages.

_lucifer: go for the fix.

hotfix.

in some places we've just added a view in the html blueprint that returns json

there's an example of a spotify token refresh endpoint somewhere which the player uses

thanks, perfect.

but we don't have a standardised way of doing this

the "get people who x follows" was one of these but then we moved it into API_PREFIX

[critiquebrainz] amCap1712 opened pull request #354 (master…sentry-upgrade): Upgrade brainzutils and sentry-sdk https://github.com/metabrainz/critiquebrainz/pu...

_lucifer: hi, just to confirm - when you were originally working on this task you performed the upgrade on all downstream apps and tried to send an error?

alastairp, yes.

but i do not remember whether I tested `capture_message` specifically.

I found my old branch. I tested that as well.

🎉

_lucifer: can I try regenerating the dataframes now?

ruaok, no i figured the hotfix won't work because the workers won't get the updated code. i am working on a PR as we speak.

ah, ok.

thanks.

[acousticbrainz-server] alastair opened pull request #394 (master…sentry-upgrade): Upgrade brainzutils and sentry-sdk https://github.com/metabrainz/acousticbrainz-se...

[listenbrainz-server] amCap1712 opened pull request #1368 (master…listen-threshold): Update filter condition to tell spark we want the field and fix a regression https://github.com/metabrainz/listenbrainz-serv...

in case this fix doesn't work, i'll rename the count field to soemthing but that'll probably need changes elsewhere as well.

_lucifer: I noticed that in docker-server-configs we are using the old deprecated dsn with a secret for most services

yes right.

I'll open a PR to update that to the public DSN for these apps

cool thanks!

huh, we also have bono-ds-hoster. I forgot to test that.

hmm, but I may have set that up manually instead of using BU

I remember that one of these services I did manually

yeah right that uses sentry-sdk directly

its already using new sentry

great

just jumping ahead a moment here - in order to merge the cache updates in BU we need to merge the CB release which does manual cache expiration?

yes https://github.com/metabrainz/critiquebrainz/pu...

great

once I do docker-server-configs I'll look at that

👍

want to try and release that today too?

it'd be great to get it out of the way!

should we go for cache or startup

cache

sure, let's do that then

well, depending on how long you're around for maybe we could do both... let's see

we also a cache PR in LB

there's always tomorrow :)

yup

the ratelimiting? yeah

yes

that's not related to this perse though.

OK, great. let's keep that in mind for next steps, but let's not get distracted by it

👍

I see a dsn configuration for AB beta that I can't find in sentry

I wonder if we deleted it

_lucifer: https://docs.sentry.io/product/sentry-basics/en...

I wonder if we should use environments instead of different projects for beta/test/prod

[acousticbrainz-server] alastair merged pull request #394 (master…sentry-upgrade): Upgrade brainzutils and sentry-sdk https://github.com/metabrainz/acousticbrainz-se...

alastairp, i think using environments will be better. that would let us group errors across environments. that's not possible with projects i think.

_lucifer: ah, I think I found a bug in our sentry-sdk upgrade

so it seems that there was some way of passing generic information to raven (including environment), by using SENTRY_CONFIG. You're not using **options in the call to sentry_sdk.init

ah ok, remove SENTRY_CONFIG and just pass **options to init?

I'm not sure. give me a moment to think it through

_lucifer: one failing test on the spark PR.

_lucifer: I think you're right, we can just pass it straight into init(). I opened BU-44 to cover this

BU-44: Further improvements to sentry integration https://tickets.metabrainz.org/browse/BU-44

_lucifer: I'm digging further into sentry config in consul, and have written comments on LB and CB

while we're touching this code we should get rid of the out-of-date config options and structure. It looks like AB is fine in this regard

ruaok, i had pushed the fix for that. i see you merged it. thanks!

alastairp, yeah makes sense to cleanup the config.

i have updated LB PR.

!m _lucifer

You're doing good work, _lucifer!

two people dragging you in 2 directions and you still manage to get stuff updated

regarding CB, should we do lazy-loading or usual?

[listenbrainz-server] mayhem merged pull request #1368 (master…listen-threshold): Update filter condition to tell spark we want the field and fix a regression https://github.com/metabrainz/listenbrainz-serv...

what do you mean usual?

indeed, _lucifer the multitasking master!

i mean eager, like we do currently for ravenjs.

sorry, I still don't know what you mean :(