#metabrainz

/

      • ruaok
        thank you yvanzo
      • 2019-05-09 12948, 2019

      • ruaok
        40Gb shared buffers deployed. now to wait.
      • 2019-05-09 12952, 2019

      • alastairp
        zas: we're working on some patches that should remove the need for complex sorts from the majority of our queries
      • 2019-05-09 12900, 2019

      • alastairp
        which should reduce disk writes
      • 2019-05-09 12926, 2019

      • alastairp
        however it's true that they might come back as we get more load, so trying to give more resources would be good
      • 2019-05-09 12929, 2019

      • zas
        machine is powerful already, db on SSD would be faster ofc, but since the db is big (and get bigger) it would cost a lot of more
      • 2019-05-09 12956, 2019

      • zas
        and i have the feeling the server doesn't get that many queries atm, and is under performing
      • 2019-05-09 12938, 2019

      • alastairp
        the data returned in a single query is large - 100s of kb to MB depending on what query
      • 2019-05-09 12930, 2019

      • BrainzGit
        [bookbrainz-site] MonkeyDo opened pull request #283 (master…hide-deprecated-relationships): Hide deprecated relationships in relationship editor https://github.com/bookbrainz/bookbrainz-site/pul…
      • 2019-05-09 12935, 2019

      • travis-ci joined the channel
      • 2019-05-09 12935, 2019

      • travis-ci
        Project bookbrainz-site build #2164: passed in 3 min 38 sec: https://travis-ci.org/bookbrainz/bookbrainz-site/…
      • 2019-05-09 12935, 2019

      • travis-ci has left the channel
      • 2019-05-09 12918, 2019

      • zas
        alastairp: did you measure at which rate the db is growing ? i did a rough estimate of 50gb/month (based on disk space, so perhaps wrong)
      • 2019-05-09 12952, 2019

      • alastairp
        no, but we could estimate it based on average data size and rate of growth of number of rows
      • 2019-05-09 12944, 2019

      • zas
        can you do that ? if we need to throw more resources it will be useful. Perhaps time to think about scalability of this project
      • 2019-05-09 12925, 2019

      • zas
        ruaok: last change looks rather positive, still need more stats to be sure though
      • 2019-05-09 12957, 2019

      • ruaok
        Oh, good. I guess the caches are warm now.
      • 2019-05-09 12913, 2019

      • ruaok is spending quality time at correos
      • 2019-05-09 12941, 2019

      • zas
        btw, what's the traffic on eth0 atm ?
      • 2019-05-09 12905, 2019

      • ruaok
        Akf atm. Will look when home again.
      • 2019-05-09 12942, 2019

      • zas
        ftp download
      • 2019-05-09 12921, 2019

      • ruaok
        Sigh. Everytime i collect a parcel, it's not a matter of picking it up, but some great mystery the staff needs to solve.
      • 2019-05-09 12950, 2019

      • alastairp
        Mr_Monkey: still at office?
      • 2019-05-09 12949, 2019

      • bitmap has quit
      • 2019-05-09 12926, 2019

      • bitmap joined the channel
      • 2019-05-09 12955, 2019

      • Mr_Monkey
        alastairp: Yep
      • 2019-05-09 12917, 2019

      • Mr_Monkey
        Where do you want to meet?
      • 2019-05-09 12914, 2019

      • alastairp
        Mr_Monkey: I'm a block away, I'll come upstairs
      • 2019-05-09 12927, 2019

      • Mr_Monkey
        Oh !
      • 2019-05-09 12952, 2019

      • Cyna
        yvanzo: You around for a bit ?
      • 2019-05-09 12945, 2019

      • yvanzo
        yup!
      • 2019-05-09 12908, 2019

      • Cyna
        I've been trying to figure out how to connect my redis container to work as a redis-server for my server container. Previously when i did it manually I used to run the command `redis-server` in a new terminal
      • 2019-05-09 12900, 2019

      • Cyna
        But I cant do the same with compose... This is probably one of the issues Im facing before which my compose-file can be completed
      • 2019-05-09 12949, 2019

      • Cyna
        I tried changing the values in `DBDefs.pm` so now my server pings to redis:6379 but it doesnt do the job yet
      • 2019-05-09 12942, 2019

      • Cyna
      • 2019-05-09 12918, 2019

      • reosarevok
        yvanzo: any idea why labels don't seem to be editable anymore from the ticket in jira?
      • 2019-05-09 12937, 2019

      • reosarevok
        (like, components has click to edit but labels does not)
      • 2019-05-09 12902, 2019

      • yvanzo
        reosarevok: you might not have permissions, I will check
      • 2019-05-09 12925, 2019

      • Cyna
        yvanzo: could you look into my query too please
      • 2019-05-09 12947, 2019

      • yvanzo
        Cyna: what is your query?
      • 2019-05-09 12905, 2019

      • reosarevok
        yvanzo: CatQuest needs them for INST
      • 2019-05-09 12910, 2019

      • reosarevok
        (the permissions)
      • 2019-05-09 12918, 2019

      • reosarevok
        I mean, I could use them too but that's less important
      • 2019-05-09 12938, 2019

      • yvanzo
        reosarevok: I added you to the jira-developers group
      • 2019-05-09 12940, 2019

      • Cyna
        Above reosarevok question
      • 2019-05-09 12953, 2019

      • Cyna
        around 3 to 4 messages
      • 2019-05-09 12921, 2019

      • CatQuest
        sigh
      • 2019-05-09 12939, 2019

      • reosarevok
        yvanzo: please make sure CatQuest can use them too, at least for the INST project :)
      • 2019-05-09 12948, 2019

      • CatQuest
        :)
      • 2019-05-09 12909, 2019

      • ruaok
        zas: you were right, no improvement for frank with more shared_buffers.
      • 2019-05-09 12917, 2019

      • ruaok
        I can try work_mem 256MB, but I dunno.
      • 2019-05-09 12927, 2019

      • yvanzo
        Cyna: but what is your setup?
      • 2019-05-09 12944, 2019

      • yvanzo
        reosarevok: oops, ok, checking that now.
      • 2019-05-09 12949, 2019

      • Cyna
        its the basic musicbrainz server setup
      • 2019-05-09 12917, 2019

      • Cyna
      • 2019-05-09 12932, 2019

      • Cyna
        This is my docker-compose with Dockerfle
      • 2019-05-09 12930, 2019

      • zas
        ruaok: I think there is a small improvement, cpu load decreased a bit, but hard to be sure, traffic is varying a lot
      • 2019-05-09 12946, 2019

      • yvanzo
        Cyna: ok, so far so good, it is very close to https://github.com/metabrainz/musicbrainz-docker/…
      • 2019-05-09 12954, 2019

      • zas
        BTW arriving at Sants in few minutes
      • 2019-05-09 12927, 2019

      • Cyna
        Well I couldnt successfully run musicbrainz-docker
      • 2019-05-09 12933, 2019

      • zas
        Where do we meet?
      • 2019-05-09 12955, 2019

      • Cyna
        This is my own docker compose in which I know what every command does and its the minimal setup required to setup a dev or test environment
      • 2019-05-09 12901, 2019

      • yvanzo
        Cyna: do you really need that networking stuff in compsoe?
      • 2019-05-09 12935, 2019

      • Cyna
        Well to communicate between containers I do need to create a network, dont I >
      • 2019-05-09 12939, 2019

      • Cyna
        *?
      • 2019-05-09 12943, 2019

      • yvanzo
        Yes but there is a default network already.
      • 2019-05-09 12945, 2019

      • Cyna
        I missed adding networks to server container config
      • 2019-05-09 12916, 2019

      • yvanzo
        Cyna: exactly, that might be the bug :)
      • 2019-05-09 12950, 2019

      • Cyna
        I've replaced and started composing it... Bounce back in 20 mins
      • 2019-05-09 12951, 2019

      • yvanzo
        (or you can remove network musicbrainz instead)
      • 2019-05-09 12920, 2019

      • Cyna
        Damn... it takes atleast 10 mins to get all the containers set up
      • 2019-05-09 12932, 2019

      • Cyna
        mainly because of cpamn
      • 2019-05-09 12948, 2019

      • CatQuest has quit
      • 2019-05-09 12908, 2019

      • CatQuest joined the channel
      • 2019-05-09 12908, 2019

      • CatQuest has quit
      • 2019-05-09 12908, 2019

      • CatQuest joined the channel
      • 2019-05-09 12903, 2019

      • ruaok
        2019-05-09 17:35:42.622 GMT @ 2514 LOG: temporary file: path "base/pgsql_tmp/pgsql_tmp2514.2", size 63991560
      • 2019-05-09 12916, 2019

      • ruaok
        64MB temp files. how the hell?
      • 2019-05-09 12904, 2019

      • ruaok
      • 2019-05-09 12917, 2019

      • ruaok
        is that the hl extractor DDoSing us?
      • 2019-05-09 12934, 2019

      • ruaok
        so, that query generates the huge temp files.
      • 2019-05-09 12905, 2019

      • ruaok
        and there were quite a few of those queries -- we should increase the delay between queries so that we do this less often
      • 2019-05-09 12951, 2019

      • yvanzo
        CatQuest: I added you to INST developers, can you please check editing labels on INST issues?
      • 2019-05-09 12927, 2019

      • Cyna
        it worked
      • 2019-05-09 12931, 2019

      • Cyna
        Srta
      • 2019-05-09 12934, 2019

      • Cyna
        *sorta
      • 2019-05-09 12924, 2019

      • ruaok
        alastairp: iliekcomputers: we should deploy the rate limiter on AB ASAP.
      • 2019-05-09 12923, 2019

      • ruaok
        after dinner zas and I will have to take a closer look at this traffic. someone is hitting us too hard.
      • 2019-05-09 12932, 2019

      • Cyna
        I've finally created a first working build
      • 2019-05-09 12918, 2019

      • yvanzo
        Congrats I guess! :)
      • 2019-05-09 12901, 2019

      • Gazooo has quit
      • 2019-05-09 12949, 2019

      • Dimtree has quit
      • 2019-05-09 12900, 2019

      • chhavi_ has quit
      • 2019-05-09 12946, 2019

      • ruaok
        zas: frank is still fine. I'm turning on the hl calc again to see if the problem returns
      • 2019-05-09 12948, 2019

      • ruaok
        of course, now it has a backlog it has to churn through first.
      • 2019-05-09 12917, 2019

      • ruaok
        and the load comes back, right on queue after it runs out of items to examine.
      • 2019-05-09 12945, 2019

      • ruaok
        iliekcomputers: alastairp: here is my assessment of what is currently wrong with AB: https://gist.github.com/mayhem/a3916f616d670e524f…
      • 2019-05-09 12955, 2019

      • ruaok
        I'll start working on fixes for both in the morning.
      • 2019-05-09 12921, 2019

      • yvanzo has quit
      • 2019-05-09 12927, 2019

      • ephemer0l is now known as SeaDough
      • 2019-05-09 12901, 2019

      • Lotheric has quit
      • 2019-05-09 12932, 2019

      • SeaDough is now known as LandBread
      • 2019-05-09 12946, 2019

      • LandBread is now known as SeaDough
      • 2019-05-09 12957, 2019

      • SeaDough is now known as GeneralDiscourse
      • 2019-05-09 12900, 2019

      • Lotheric joined the channel
      • 2019-05-09 12955, 2019

      • punkscience joined the channel
      • 2019-05-09 12958, 2019

      • punkscience
        Hey there. Are there any 'getting started' docs on how to contribute to the listenbrainz server issues? Looking for the 101 -- how to launch the server, any guidance on the workflow for submitting changes etc.
      • 2019-05-09 12912, 2019

      • punkscience
      • 2019-05-09 12921, 2019

      • punkscience
        Seems relevant!
      • 2019-05-09 12920, 2019

      • punkscience
        So I am installing the dev environment on a Mac. I ran this line:
      • 2019-05-09 12922, 2019

      • punkscience
        docker-compose -f docker/docker-compose.yml -p listenbrainz run --
      • 2019-05-09 12941, 2019

      • punkscience
        The error message says ERROR: No such service: --
      • 2019-05-09 12916, 2019

      • punkscience
        Whatever '--' means is probably something supported by linux but not macOS.
      • 2019-05-09 12932, 2019

      • punkscience
        Any idea what the translation might be?
      • 2019-05-09 12905, 2019

      • aidanlw17
        Hey punkscience! I don't work on listenbrainz but I just followed through the installation process, and I think it should work on mac - it did on mine, at least
      • 2019-05-09 12935, 2019

      • aidanlw17
        The problem you're having there is that the command continues onto the next line in that blog post
      • 2019-05-09 12905, 2019

      • aidanlw17
        So the full command is "docker-compose -f docker/docker-compose.yml -p listenbrainz run --rm web python3 manage.py init_db --create-db", for example
      • 2019-05-09 12932, 2019

      • aidanlw17
        If you haven't found this already, I think a more comprehensive guide would be the listenbrainz-server docs
      • 2019-05-09 12943, 2019

      • aidanlw17
      • 2019-05-09 12941, 2019

      • punkscience has quit