#metabrainz

/

      • ruaok
        https://github.com/metabrainz/listenbrainz-server…

        • 2017-12-22 35639, 2017

      • github joined the channel

        • 2017-12-22 35639, 2017

      • github
        [listenbrainz-server] mayhem opened pull request #323: Deploy to beta improvements (master...lb-beta-deploy) https://git.io/vb9XI

        • 2017-12-22 35639, 2017

      • github has left the channel

        • 2017-12-22 35642, 2017

      • ruaok
        iliekcomputers: how so?

        • 2017-12-22 35605, 2017

      • naiveai has quit

        • 2017-12-22 35651, 2017

      • iliekcomputers
        ruaok: we do import default_config, try: import custom_config in a few places in the db code and the scripts.

        • 2017-12-22 35613, 2017

      • iliekcomputers
        Creating a flask app and using its configuration would be a better way to manage this

        • 2017-12-22 35637, 2017

      • iliekcomputers
        https://github.com/metabrainz/listenbrainz-server…

        • 2017-12-22 35649, 2017

      • ruaok
        so, should the consul based config be copied to custom_config.py ?

        • 2017-12-22 35659, 2017

      • ruaok
        now that I think of it, that makes a lot of sense, but is harder to detect.

        • 2017-12-22 35653, 2017

      • ruaok
        ah, no. custom_config.py does not exist, on the .default file.

        • 2017-12-22 35606, 2017

      • ruaok
        so, that makes sense. does the manage.py also read the custom_config.py file?

        • 2017-12-22 35604, 2017

      • iliekcomputers
        ruaok: yes

        • 2017-12-22 35619, 2017

      • ruaok
        perfect. let me update the PR then.

        • 2017-12-22 35601, 2017

      • iliekcomputers
        https://github.com/metabrainz/listenbrainz-server…

        • 2017-12-22 35654, 2017

      • iliekcomputers
        Instead of this, just using the flask app configuration would be better (and remove the need for copying the consul configuration to custom_config)

        • 2017-12-22 35619, 2017

      • iliekcomputers
        For now, copying should work though.

        • 2017-12-22 35630, 2017

      • ruaok
        not sure what you mean by "flask app configuration" . isn't that what we're doing?

        • 2017-12-22 35651, 2017

      • iliekcomputers
        Instead of importing the files, we create an app (which loads the config files) and then use app.config for the values we need.

        • 2017-12-22 35619, 2017

      • iliekcomputers
        Currently we just import default config, followed by custom_config

        • 2017-12-22 35611, 2017

      • ruaok
        ok, I don't see the immediate value in doing that. lets do that later if you really feel it is necessary.

        • 2017-12-22 35618, 2017

      • github joined the channel

        • 2017-12-22 35618, 2017

      • github
        [sir] samj1912 closed pull request #71: [WIP] Refactor SIR to handles large queries (master...large) https://git.io/vbSjN

        • 2017-12-22 35618, 2017

      • github has left the channel

        • 2017-12-22 35655, 2017

      • iliekcomputers
        ruaok: example: https://github.com/metabrainz/listenbrainz-server…

        • 2017-12-22 35603, 2017

      • iliekcomputers
        ruaok: yeah, for later

        • 2017-12-22 35615, 2017

      • D4RK-PH0ENiX joined the channel

        • 2017-12-22 35613, 2017

      • Ashish_
        For that task of adding photograph of to wikimedia commons any instrument will do right

        • 2017-12-22 35651, 2017

      • heyoni joined the channel

        • 2017-12-22 35640, 2017

      • iliekcomputers
        ruaok: about the copying of dumps out of the container, once you add the listenbrainz_scheduler container to services.sh, just creating a volume there and fixing the command in crontab would work.

        • 2017-12-22 35614, 2017

      • ruaok
        k

        • 2017-12-22 35609, 2017

      • github joined the channel

        • 2017-12-22 35609, 2017

      • github
        [sir] samj1912 closed pull request #70: Fix documentation generation (master...doc-update) https://git.io/vbSK0

        • 2017-12-22 35609, 2017

      • github has left the channel

        • 2017-12-22 35643, 2017

      • samj1912
        Mineo: can you change the repo url for sir documentation to metabrainz/sir instead of mineo/sir?

        • 2017-12-22 35619, 2017

      • sampsyo has quit

        • 2017-12-22 35634, 2017

      • ruaok
        iliekcomputers: PR updated. please have a look.

        • 2017-12-22 35637, 2017

      • ruaok
        dumps are running now

        • 2017-12-22 35642, 2017

      • sampsyo joined the channel

        • 2017-12-22 35610, 2017

      • heyoni has quit

        • 2017-12-22 35623, 2017

      • naiveai joined the channel

        • 2017-12-22 35627, 2017

      • naiveai has quit

        • 2017-12-22 35653, 2017

      • iliekcomputers
        ruaok: lgtm

        • 2017-12-22 35652, 2017

      • ruaok
        not a lot of feedback on how long it will take to dump this data.

        • 2017-12-22 35600, 2017

      • ruaok
        I guess that isn't really needed,but I have no clue right now.

        • 2017-12-22 35645, 2017

      • Ashish_ has quit

        • 2017-12-22 35606, 2017

      • naiveai joined the channel

        • 2017-12-22 35623, 2017

      • iliekcomputers
        Should have added logging for number of users done so far :/

        • 2017-12-22 35643, 2017

      • naiveai is up now

        • 2017-12-22 35657, 2017

      • samj1912
        naiveai: what timezone are you on? O.o

        • 2017-12-22 35620, 2017

      • naiveai
        i'm in the same timezone as you but i'm sleeping as if I was in europe :P

        • 2017-12-22 35637, 2017

      • naiveai
        slept at bout 11am woke up just now

        • 2017-12-22 35641, 2017

      • ruaok
        oh, I see your problem, naiveai. You're still trying to sleep.

        • 2017-12-22 35646, 2017

      • naiveai
        :D

        • 2017-12-22 35651, 2017

      • ruaok
        iliekcomputers and samj1912 gave that up a long time ago

        • 2017-12-22 35653, 2017

      • ruaok
        or so it seems

        • 2017-12-22 35605, 2017

      • samj1912
        lol :P

        • 2017-12-22 35636, 2017

      • samj1912 wont comment on his sleep hours :P

        • 2017-12-22 35644, 2017

      • iliekcomputers sleeps like a log always :D

        • 2017-12-22 35618, 2017

      • samj1912
        and naiveai that's closer to US time than europe :P

        • 2017-12-22 35627, 2017

      • naiveai
        true

        • 2017-12-22 35648, 2017

      • arthelon[m]
        naiveai: don't give up too much sleep for gci haha

        • 2017-12-22 35618, 2017

      • chhavi_
        samj1912: Are we talking sleep hours? Want me to pitch in? :p

        • 2017-12-22 35631, 2017

      • samj1912 runs away

        • 2017-12-22 35603, 2017

      • naiveai
        question: thing works in regular Python script but not in jupyter notebook. what do?

        • 2017-12-22 35631, 2017

      • iliekcomputers
        So the last hour and a half added a lot of stuff to work on for me, opened LB-264, LB-265, LB-266, LB-267 :D

        • 2017-12-22 35631, 2017

      • BrainzBot
        LB-264: Keep track of DB Schema versions https://tickets.metabrainz.org/browse/LB-264

        • 2017-12-22 35631, 2017

      • BrainzBot
        LB-267: Add more feedback to data dump creation https://tickets.metabrainz.org/browse/LB-267

        • 2017-12-22 35631, 2017

      • BrainzBot
        LB-266: Use flask app configuration dict instead of importing config files directly https://tickets.metabrainz.org/browse/LB-266

        • 2017-12-22 35631, 2017

      • BrainzBot
        LB-265: Allow automatic updating from one schema version to another https://tickets.metabrainz.org/browse/LB-265

        • 2017-12-22 35642, 2017

      • samj1912
        naiveai: post a pastebin?

        • 2017-12-22 35635, 2017

      • naiveai
        samj1912: https://pastebin.com/bc39nWWj this gives me "NoneType object is not callable" in jupyter, but not in a script

        • 2017-12-22 35619, 2017

      • samj1912
        archive isn't defined?

        • 2017-12-22 35651, 2017

      • naiveai
        no, it is, it's a dictionary

        • 2017-12-22 35609, 2017

      • samj1912
        I meant in the pastebin

        • 2017-12-22 35609, 2017

      • samj1912
        anyways, that simple script after redefining archive as {} works for me

        • 2017-12-22 35612, 2017

      • samj1912
        as it should

        • 2017-12-22 35626, 2017

      • alastairp
        iliekcomputers: FYI: http://flask.pocoo.org/docs/0.12/patterns/appfact…

        • 2017-12-22 35632, 2017

      • alastairp
        > The downside is that you cannot use the application object in the blueprints at import time. You can however use it from within a request. How do you get access to the application with the config? Use current_app:

        • 2017-12-22 35640, 2017

      • drsaunders joined the channel

        • 2017-12-22 35655, 2017

      • alastairp
        I saw that in some of your patches you build app and then store app.config to use in a later place

        • 2017-12-22 35607, 2017

      • alastairp
        you can use current_app because it's lazy

        • 2017-12-22 35608, 2017

      • alastairp
        oh, now that I think about it though, you probably need an active application context, which you'll only have if you integrate it into the flask cli

        • 2017-12-22 35642, 2017

      • xps2 joined the channel

        • 2017-12-22 35636, 2017

      • travis-ci joined the channel

        • 2017-12-22 35637, 2017

      • travis-ci
        metabrainz/picard#2863 (master - ec07112 : Sambhav Kothari): The build passed.

        • 2017-12-22 35637, 2017

      • travis-ci
        Change view : https://github.com/metabrainz/picard/compare/3e8d…

        • 2017-12-22 35637, 2017

      • travis-ci
        Build details : https://travis-ci.org/metabrainz/picard/builds/32…

        • 2017-12-22 35637, 2017

      • travis-ci has left the channel

        • 2017-12-22 35608, 2017

      • travis-ci joined the channel

        • 2017-12-22 35609, 2017

      • travis-ci
        metabrainz/picard#2863 (master - ec07112 : Sambhav Kothari): The build passed.

        • 2017-12-22 35609, 2017

      • travis-ci
        Change view : https://github.com/metabrainz/picard/compare/3e8d…

        • 2017-12-22 35609, 2017

      • travis-ci
        Build details : https://travis-ci.org/metabrainz/picard/builds/32…

        • 2017-12-22 35609, 2017

      • travis-ci has left the channel

        • 2017-12-22 35639, 2017

      • dragonzeron joined the channel

        • 2017-12-22 35607, 2017

      • padraic has quit

        • 2017-12-22 35645, 2017

      • ruaok
        iliekcomputers: https://gist.github.com/mayhem/7e5080d6a2cb049bf0…

        • 2017-12-22 35652, 2017

      • ruaok
        that is how the export died. O_O

        • 2017-12-22 35645, 2017

      • iliekcomputers
        user with / in their name

        • 2017-12-22 35656, 2017

      • iliekcomputers
        should have kept that in mind, ugh, my bad

        • 2017-12-22 35609, 2017

      • ruaok
        no worries.

        • 2017-12-22 35628, 2017

      • iliekcomputers
        I'll add some more logging and send a PR

        • 2017-12-22 35632, 2017

      • ruaok
        ok, so I need to add a new volume and a new container, for the scheduler, right?

        • 2017-12-22 35654, 2017

      • ruaok
        do you have any more details for me to look at?

        • 2017-12-22 35654, 2017

      • iliekcomputers
        ruaok: yes

        • 2017-12-22 35612, 2017

      • iliekcomputers
        wait, Dockerfile.prod hasn't been modified to accomodate cron

        • 2017-12-22 35634, 2017

      • ruaok
        I kinda assumed that was my job.

        • 2017-12-22 35637, 2017

      • ruaok
        what does it need?

        • 2017-12-22 35604, 2017

      • iliekcomputers
        install cron: https://github.com/metabrainz/listenbrainz-server…

        • 2017-12-22 35622, 2017

      • iliekcomputers
        then add the crontab file: https://github.com/metabrainz/listenbrainz-server…

        • 2017-12-22 35627, 2017

      • ruaok
        cake. I'll do it in the next batch of changes I need to make.

        • 2017-12-22 35632, 2017

      • ruaok
        you focus on the dump stuff.

        • 2017-12-22 35635, 2017

      • drsaunders has quit

        • 2017-12-22 35608, 2017

      • iliekcomputers
        the crontab is configured for dev only so far: https://github.com/metabrainz/listenbrainz-server…

        • 2017-12-22 35616, 2017

      • iliekcomputers
        ruaok: cool 👍🏽

        • 2017-12-22 35606, 2017

      • heyoni joined the channel

        • 2017-12-22 35649, 2017

      • skGCI has quit

        • 2017-12-22 35601, 2017

      • rgunkar joined the channel

        • 2017-12-22 35615, 2017

      • github joined the channel

        • 2017-12-22 35615, 2017

      • github
        [listenbrainz-server] mayhem opened pull request #324: Use separate prod and beta configuration directories. (master...use-separate-env-dirs) https://git.io/vb9Am

        • 2017-12-22 35615, 2017

      • github has left the channel

        • 2017-12-22 35625, 2017

      • ruaok
        this is for the lemmy proc count fix, zas, iliekcomputers ^^

        • 2017-12-22 35640, 2017

      • zas
        k thx ;)

        • 2017-12-22 35641, 2017

      • ruaok
        which nagios just gave us thumbs up on. :)

        • 2017-12-22 35654, 2017

      • Slurpee joined the channel

        • 2017-12-22 35654, 2017

      • Slurpee has quit

        • 2017-12-22 35654, 2017

      • Slurpee joined the channel

        • 2017-12-22 35645, 2017

      • culinko joined the channel

        • 2017-12-22 35650, 2017

      • culinko has left the channel

        • 2017-12-22 35611, 2017

      • github joined the channel

        • 2017-12-22 35611, 2017

      • github
        [listenbrainz-server] paramsingh opened pull request #325: LB-268: Escape usernames before creating files in dumps (master...escape-username-dumps) https://git.io/vb9xN

        • 2017-12-22 35611, 2017

      • github has left the channel

        • 2017-12-22 35633, 2017

      • ruaok
        your favorite task ever, no?

        • 2017-12-22 35645, 2017

      • iliekcomputers
        lol

        • 2017-12-22 35658, 2017

      • iliekcomputers
        i don't even now how i could forget about this :P

        • 2017-12-22 35614, 2017

      • ruaok
        painful memories get repressed.

        • 2017-12-22 35633, 2017

      • ruaok
        or repossesed. I forget which

        • 2017-12-22 35642, 2017

      • iliekcomputers
        heh

        • 2017-12-22 35639, 2017

      • dragonzeron has quit

        • 2017-12-22 35609, 2017

      • ruaok
        urllib quote plus for filenames. interesting concept. :)

        • 2017-12-22 35652, 2017

      • ruaok
        ... will that... work on windows?

        • 2017-12-22 35606, 2017

      • ruaok 's head asplodes

        • 2017-12-22 35622, 2017

      • alastairp
        if you need to support windows and you want to protect from malicious/bad usernames perhaps you need to keep in mind http://kizu514.com/blog/forbidden-file-names-on-w… too ?

        • 2017-12-22 35624, 2017

      • iliekcomputers
        it encodes all invalid characters

        • 2017-12-22 35645, 2017

      • ruaok
        yes, but invalid for URLs, no?

        • 2017-12-22 35659, 2017

      • ruaok
        not for filenames -- those are two different domains.

        • 2017-12-22 35620, 2017

      • ruaok
        and sadly, yes, we need to look out for winblows users who want to import 60M listens into excel.

        • 2017-12-22 35620, 2017

      • iliekcomputers
        yeah, but invalid for urls is a superset of invalid for filenames (unix / windows), although I'd be happy to do something more valid here.

        • 2017-12-22 35609, 2017

      • ruaok
        is that superset assertion correct?