#metabrainz

/

      • AfroThundr|main joined the channel
      • AfroThundr|main has quit
      • AfroThundr|main joined the channel
      • ivnat joined the channel
      • eharris has quit
      • Nyanko-sensei joined the channel
      • D4RK-PH0ENiX has quit
      • eharris joined the channel
      • ivnat has quit
      • Nyanko-sensei has quit
      • D4RK-PH0ENiX joined the channel
      • Matthew_ has quit
      • Matthew_ joined the channel
      • Nyanko-sensei joined the channel
      • D4RK-PH0ENiX has quit
      • amCap1712
        hi all
      • yvanzo, bitmap, CallerNo6: How does MB display the release artist in appears on releases section on recordings page?
      • Also, what will be the webservice query equivalent to it?
      • which returns artist credit for the recording but how to get the artist credits for induvidual releases
      • reosarevok
        amCap1712: add artist-credits to the inc IIRC
      • amCap1712
        thanks reosarevok
      • it worked
      • so that means artist parameter queries artist details for the entity and artist credit for associated entities?
      • Lotheric has quit
      • Lotheric joined the channel
      • Nyanko-sensei has quit
      • D4RK-PH0ENiX joined the channel
      • Freso
        amCap1712: And if possible, you should use https:// always, please. :)
      • amCap1712
        Freso: I use https by default but I have an extension to automatially pretty print json in browser and i don't know but it behaves unusually with https. Thats why the http link.
      • ruaok
        yvanzo: is there a new VM to download yet?
      • amCap1712
      • bitmap, yvanzo, reosarevok : I have posted two snippets above. The first one is from lookup query and the second from search query.
      • In the first result, there is a name and joinphrase field but this field is absent in search result. Is this intentional?
      • reosarevok
        amCap1712: joinphrase might be, name isn't (https://github.com/metabrainz/sir/pull/99 - merged but not yet released I think)
      • amCap1712
        ok thanks reosarevok
      • reosarevok
        bitmap, yvanzo: 08006 DBI connect('dbname=musicbrainz_db;host=10.2.2.24;port=65401','musicbrainz',...) failed: ERROR: no more connections allowed (max_client_conn)
      • wat
      • ruaok
        reosarevok where is that?
      • reosarevok
        ruaok: just got it when voting
      • beta
      • Well, "just" - at 12:22, obviously :d
      • ruaok
        fun.
      • yvanzo
        ruaok: not yet, mostly missing MB Solr (PR open) and SIR (reindex bug)
      • amCap1712, reosarevok: indeed, should be included in the next release of SIR.
      • ruaok
        what is the fastest way for a customer who is using an old VM to keep moving?
      • yvanzo
        ruaok: pulling musicbrainz-docker and rebuilding musicbrainz service.
      • ruaok
        in a clean VM?
      • yvanzo
        That is the easiest, but it implies a down time.
      • in latest VM.
      • Btw, is there anyone from the team willing to join SIR development?
      • It is nice to have volunteers but that might not be enough in the long run.
      • zas
        reosarevok: https://github.com/metabrainz/docker-postgres/b... set this to max conn to 400, but for some reason, there was a peak in established connections at the same time: https://stats.metabrainz.org/d/000000048/hetzne...
      • ruaok
        not really. param and monkey are stretched pretty thin. :(
      • reosarevok
        yvanzo: I think you just became the SIR lead
      • ruaok
        yep. for better or for worse.
      • Mr_Monkey
        Also, I don't speak Python (yet), so I have some ground to cover before being useful on SIR
      • zas
        at the same time, number of established connections has a peak on almost all backends
      • and it matches a drop of established connections on lemmy ... : https://stats.metabrainz.org/d/000000055/hetzne...
      • reosarevok
        My Python is mediocre but I can try to help - my Java is absolutely shit though so anything that requires Java is going to be a mess :p
      • yvanzo
        I don’t either, would be nice to have at least one.
      • zas
        reosarevok: this error matches with a restart of containers on lemmy https://stats.metabrainz.org/d/000000048/hetzne...
      • yvanzo
        I can handle Java (not that I would favor it)
      • zas
        hmmm, actually not sure what happened on lemmy, did someone restart any container on it recently ?
      • ruaok
        recently?
      • no
      • but I have no idea why that proxy container keep flapping.
      • well, the container is ilde and doing nothing. not getting any requests at all.
      • but the service blinks on and off all the time.
      • zas
        it seems related to the error reo got, which isn't good, because it applies to all pg bouncers connections
      • ruaok
        I think with BB on board, we may need to increase the number of active connections we allow to PG.
      • zas
        something changed on lemmy at the same time, i thought it was a container restart, but apparently not
      • ruaok
        if it was, I didn't do it.
      • zas
        ruaok: 400 simultaneous connections is more than enough, but i think something was actually using too many connections during a short instant, it's now back to normal
      • ruaok
        actually, if you could look at the proxy-api container issue, I'd love you for it. it doesn't appear to be a problem with the service, but with traffic getting to the service.
      • zas
        the usual number of connections is around 100
      • ruaok
        zas: pg bouncer is keeping a lot of connections open for pooling.
      • zas
        yes, but values are ok, and safe imho, increasing this number doesn't fix the problem we just had. In fact, it is much safer to have this number not too high. The real question is why, suddenly, the number of connections increased that much (>800) = which service actually did this shit, i suspect lb stuff, because it seems to me everything points at lemmy, but i can be wrong ofc
      • ruaok: where is running this proxy-api container ??
      • ruaok
        lemmy
      • zas
        i see none with this name
      • ruaok
        listenbrainz-api-compat-prod
      • zas
        ah, ok, let me see
      • did you look at its log ?
      • because imho it's the cause of the problem, it was spawning/killing workers at the same time the reo's error happened
      • but i don't know much about this stuff
      • looks suspicious
      • ruaok
        yes, which is why I pointed you to it.
      • and I dont understand it either. for some reason a consul value that we defined just doesn't show up.
      • then on the next restart it does.
      • iliekcomputers: can you please rebuild the compat api container with the latest brainzutils image and redeploy it?
      • that might solve the problem -- upgrade all the stuff to the latest and greatest.
      • CatQuest
        in my experience upgrading shit tends to break shit but ¯\_(ツ)_/¯
      • yokel joined the channel
      • D4RK-PH0ENiX has quit
      • zas
        ruaok: i think i found the issue, uwsgi doesn't check for config files to exist before starting, so it is possible consul-template didn't generate the config file yet, while uwsgi is starting
      • ruaok
        ah yes. we fixed this in the other images.
      • iliekcomputers: ^^
      • zas
        also, processes is set to 100, but i see only 50 workers (dunno why), threads is not set (so it defaults to 1), but each process will create 3 connections to pg on bowie (it uses 3 dbs), and 1 to pg on queen, it means 300 connections, which is likely to exhaust all connections pretty fast on high load. Not sure if i understand this uwsgi stuff well, but since each process is totally separated from others, pg connections aren't
      • shared imho
      • bitmap, yvanzo: do we have a mean to know how many connections are made to pg on bowie from lemmy ?
      • D4RK-PH0ENiX joined the channel
      • yvanzo
        lemmy is listenbrainz+messybrainz, right? 9+10 atm
      • ferbncode
        spellew: The timeline looks good. Let me know if you have something to discuss before the coding period starts. Also, There is an issue up in CB related to the `musicbrainz_db`. You'd probably want to work on it before you start your actual gsoc project. It would be great if you could take a look at it (https://sentry.metabrainz.org/share/issue/a71fd...).
      • CatQuest
        meow `😺`
      • pristine__
        ruaok: Good evening:)
      • presently, all the three scripts are dependent, so for instance if we need to check something in recommend.py (script responsible to generate recommendations) we shall wait till create_dataframes.py and train_models.py run. Should we make them independent?
      • I mean storing the result of every script in hdfs so that the script that follows can retrieve the required data from hdfs and not wait for other scripts to finish.
      • Cyna
        yvanzo, bitmap : the code for aliases component is in .tt right ?. I found it in /root/entity/alias/edit.tt
      • so I guess my first task is to convert it to React right ?
      • Lotheric has quit
      • Lotheric_ joined the channel
      • Lotheric_ is now known as Lotheric
      • SothoTalKer_ joined the channel
      • reosarevok
        Cyna: you don't need to convert that, that's the actual form for editing aliases
      • I mean, eventually we should convert it too, but :)
      • For now, stick to the edit details pages
      • That's most of what you'll need for previews
      • antlarr2 joined the channel
      • heyoni_ joined the channel
      • dolina_ joined the channel
      • loujine_ joined the channel
      • Cyna
        what is the details folder for ?
      • rektide_ joined the channel
      • add_<Entity> and edit_<Entity> are the files that are present
      • What gets rendered on clicking edit 🤔
      • flamingspinach_ joined the channel
      • flamingspinach has quit
      • antlarr has quit
      • moufl has quit
      • dolina has quit
      • navap has quit
      • heyoni has quit
      • loujine has quit
      • AfroThundr|main has quit
      • SothoTalKer has quit
      • rektide has quit
      • moufl joined the channel
      • reosarevok
        The details folder is for the actual previews of the edits
      • The kind of stuff you see when you go to, dunno, https://musicbrainz.org/edit/61302052
      • We'll want to convert those first (those are already being used by the release editor preview system)
      • And then we can look into converting forms and integrating the preview system there
      • kgz has quit
      • Cyna
        I had a different approach... I was planning on using React State feature in every form input
      • navap joined the channel
      • AfroThundr|main joined the channel