#metabrainz

/

      • yokel has quit
      • 2020-09-13 25726, 2020

      • gr0uch0mars has quit
      • 2020-09-13 25751, 2020

      • yokel joined the channel
      • 2020-09-13 25704, 2020

      • yokel has quit
      • 2020-09-13 25748, 2020

      • yokel joined the channel
      • 2020-09-13 25732, 2020

      • supersandro2000 has quit
      • 2020-09-13 25753, 2020

      • supersandro2000 joined the channel
      • 2020-09-13 25715, 2020

      • yokel has quit
      • 2020-09-13 25705, 2020

      • yokel joined the channel
      • 2020-09-13 25720, 2020

      • d4rkie has quit
      • 2020-09-13 25708, 2020

      • yokel has quit
      • 2020-09-13 25758, 2020

      • Nyanko-sensei joined the channel
      • 2020-09-13 25755, 2020

      • yokel joined the channel
      • 2020-09-13 25740, 2020

      • yokel has quit
      • 2020-09-13 25714, 2020

      • yokel joined the channel
      • 2020-09-13 25738, 2020

      • nelgin
        yvanzo,I don't understand it. It only took me 397 minutes to do a complete reindex. It's taken almost 24 hours to live index a days worth of replication data and I still don't think it's going to finish.
      • 2020-09-13 25750, 2020

      • nelgin
        Sun 13 Sep 2020 01:34:36 AM UTC
      • 2020-09-13 25750, 2020

      • nelgin
        | search.index | 42695 |
      • 2020-09-13 25751, 2020

      • nelgin
        Sun 13 Sep 2020 01:41:46 AM UTC
      • 2020-09-13 25751, 2020

      • nelgin
        | search.index | 42694 |
      • 2020-09-13 25751, 2020

      • nelgin
        Not when it drops by 1 in 5 minutes.
      • 2020-09-13 25725, 2020

      • yokel has quit
      • 2020-09-13 25728, 2020

      • yokel joined the channel
      • 2020-09-13 25741, 2020

      • yokel has quit
      • 2020-09-13 25701, 2020

      • yokel joined the channel
      • 2020-09-13 25708, 2020

      • yokel has quit
      • 2020-09-13 25728, 2020

      • iconoclasthero has quit
      • 2020-09-13 25745, 2020

      • yokel joined the channel
      • 2020-09-13 25716, 2020

      • pristine___
        iliekcomputers: request consumers should be restarted I think. Train models failed with RPC client dissociated erroe
      • 2020-09-13 25719, 2020

      • pristine___
        Error*
      • 2020-09-13 25712, 2020

      • pristine___
      • 2020-09-13 25723, 2020

      • pristine___
        Can we increase this? Might help.
      • 2020-09-13 25728, 2020

      • yokel has quit
      • 2020-09-13 25718, 2020

      • pristine___
        And reduce the driver memory a bit.
      • 2020-09-13 25715, 2020

      • yokel joined the channel
      • 2020-09-13 25719, 2020

      • pristine___
        Driver memory: 2GB executor memory: 6 GB looks good to ke
      • 2020-09-13 25722, 2020

      • pristine___
        Me*
      • 2020-09-13 25728, 2020

      • pristine___
        Also, I will remove some of the .persist() from train_models
      • 2020-09-13 25704, 2020

      • pristine___
        And few log statements to understand at which stage it exactly failed.
      • 2020-09-13 25705, 2020

      • yokel has quit
      • 2020-09-13 25725, 2020

      • Nyanko-sensei has quit
      • 2020-09-13 25735, 2020

      • yokel joined the channel
      • 2020-09-13 25713, 2020

      • yokel has quit
      • 2020-09-13 25733, 2020

      • yokel joined the channel
      • 2020-09-13 25705, 2020

      • yokel has quit
      • 2020-09-13 25707, 2020

      • Gazooo794 joined the channel
      • 2020-09-13 25741, 2020

      • yokel joined the channel
      • 2020-09-13 25725, 2020

      • Gazooo794 has quit
      • 2020-09-13 25758, 2020

      • yokel has quit
      • 2020-09-13 25702, 2020

      • sumedh joined the channel
      • 2020-09-13 25731, 2020

      • Nyanko-sensei joined the channel
      • 2020-09-13 25713, 2020

      • yokel joined the channel
      • 2020-09-13 25752, 2020

      • pristine___
        iliekcomputers: also, sometimes errors don't pop up during rerun. Is there a way by which request is resend if the current request fails?
      • 2020-09-13 25725, 2020

      • Nyanko-sensei has quit
      • 2020-09-13 25730, 2020

      • Nyanko-sensei joined the channel
      • 2020-09-13 25722, 2020

      • yokel has quit
      • 2020-09-13 25708, 2020

      • BrainzGit
        [listenbrainz-server] vansika opened pull request #1088 (master…spark-config-update): Update spark config values to try and avoid RPC client error https://github.com/metabrainz/listenbrainz-server…
      • 2020-09-13 25719, 2020

      • yokel joined the channel
      • 2020-09-13 25715, 2020

      • iconoclasthero joined the channel
      • 2020-09-13 25755, 2020

      • yokel has quit
      • 2020-09-13 25719, 2020

      • pristine___
        shivam-kapila: you about!
      • 2020-09-13 25720, 2020

      • pristine___
        ?
      • 2020-09-13 25726, 2020

      • shivam-kapila
        Yup
      • 2020-09-13 25749, 2020

      • pristine___
        shivam-kapila: the recs will update every week. So how often the web sockets or whatever look for updates to render them
      • 2020-09-13 25750, 2020

      • pristine___
      • 2020-09-13 25709, 2020

      • pristine___
        is this func related to that? I mean what exactly is it doing
      • 2020-09-13 25700, 2020

      • shivam-kapila
        this adds a new listen to the list
      • 2020-09-13 25743, 2020

      • yokel joined the channel
      • 2020-09-13 25725, 2020

      • sumedh has quit
      • 2020-09-13 25758, 2020

      • Sophist_UK has quit
      • 2020-09-13 25732, 2020

      • sumedh joined the channel
      • 2020-09-13 25705, 2020

      • ShraddhaAg has quit
      • 2020-09-13 25743, 2020

      • BestSteve has quit
      • 2020-09-13 25716, 2020

      • yvanzo
      • 2020-09-13 25717, 2020

      • BrainzBot
        SEARCH-593: Update MB Solr tests to work with 3.1
      • 2020-09-13 25730, 2020

      • _lucifer
        yvanzo: i was searching in bugs so missed that ticket. any ideas as to the cause?
      • 2020-09-13 25721, 2020

      • yvanzo
        nelgin: it's caused by the indexer which we are currently debugging
      • 2020-09-13 25759, 2020

      • Sophist-UK joined the channel
      • 2020-09-13 25727, 2020

      • yvanzo
        _lucifer: it’s just that development has been done without updating tests since 1.6.
      • 2020-09-13 25755, 2020

      • sumedh has quit
      • 2020-09-13 25731, 2020

      • _lucifer
        oh ok! I'll try to see if I can update those.
      • 2020-09-13 25707, 2020

      • yokel has quit
      • 2020-09-13 25725, 2020

      • _lucifer
        Regarding the solr 8 update, it seems to be a deployment thing. doesn't seem any changes are required in the code base.
      • 2020-09-13 25745, 2020

      • yvanzo
        iirc, travis was even down for a while, but that has been fixed: tests successfully fail now.
      • 2020-09-13 25718, 2020

      • yvanzo
        I’m not sure that Solr 8 upgrade is that straightforward.
      • 2020-09-13 25704, 2020

      • sumedh joined the channel
      • 2020-09-13 25755, 2020

      • Sophist-UK has quit
      • 2020-09-13 25723, 2020

      • yvanzo
        Java 11 would be a step forward at least.
      • 2020-09-13 25754, 2020

      • _lucifer
        yeah but solr 8 supports java 8
      • 2020-09-13 25724, 2020

      • sumedh has quit
      • 2020-09-13 25740, 2020

      • _lucifer
        solr 9 onwards require java 11
      • 2020-09-13 25754, 2020

      • Gazooo794 joined the channel
      • 2020-09-13 25744, 2020

      • sumedh joined the channel
      • 2020-09-13 25749, 2020

      • Sophist-UK joined the channel
      • 2020-09-13 25733, 2020

      • yokel joined the channel
      • 2020-09-13 25700, 2020

      • ShraddhaAg joined the channel
      • 2020-09-13 25709, 2020

      • BestSteve joined the channel
      • 2020-09-13 25721, 2020

      • yokel has quit
      • 2020-09-13 25718, 2020

      • yokel joined the channel
      • 2020-09-13 25747, 2020

      • yokel has quit
      • 2020-09-13 25747, 2020

      • yokel joined the channel
      • 2020-09-13 25733, 2020

      • pristine___
        shivam-kapila: have a look when you get the time, fails on line 11
      • 2020-09-13 25735, 2020

      • pristine___
      • 2020-09-13 25754, 2020

      • shivam-kapila
        strange
      • 2020-09-13 25723, 2020

      • shivam-kapila
        please paste logs
      • 2020-09-13 25738, 2020

      • yokel has quit
      • 2020-09-13 25701, 2020

      • yokel joined the channel
      • 2020-09-13 25703, 2020

      • Gazooo794 has quit
      • 2020-09-13 25755, 2020

      • Gazooo794 joined the channel
      • 2020-09-13 25737, 2020

      • pristine___
        shivam-kapila: A sec, but fyi I have added a listen to recentListenprops. Json
      • 2020-09-13 25718, 2020

      • pristine___
        shivam-kapila: logs
      • 2020-09-13 25720, 2020

      • pristine___
      • 2020-09-13 25730, 2020

      • _lucifer
        yvanzo: i am trying to get the tests to work. It seems that Solr is not returning any results for queries. Do I need to do some setup steps before running solr tests ?
      • 2020-09-13 25711, 2020

      • Gazooo794 has quit
      • 2020-09-13 25728, 2020

      • shivam-kapila
        pristine___: strangely the listens swapped
      • 2020-09-13 25735, 2020

      • _lucifer
        I cloned the git repo of mb-solr and installed mmd-schema jar and built mb-solr jar. Then, rant the tests
      • 2020-09-13 25737, 2020

      • pristine___
        shivam-kapila: I only called the handleNewRecommendations function.
      • 2020-09-13 25756, 2020

      • shivam-kapila
        you need to mock what it returns IG
      • 2020-09-13 25713, 2020

      • pristine___
        But it is not returning anything
      • 2020-09-13 25719, 2020

      • pristine___
        Just updating states
      • 2020-09-13 25722, 2020

      • pristine___
        shivam-kapila: I hope it is not because of `slice`. But the general structure of test function, it is alright? The calls and stuff
      • 2020-09-13 25736, 2020

      • shivam-kapila
        lgtm
      • 2020-09-13 25755, 2020

      • yokel has quit
      • 2020-09-13 25747, 2020

      • yvanzo
        _lucifer: you might need the jar from the assets: https://github.com/metabrainz/mb-solr/releases
      • 2020-09-13 25737, 2020

      • _lucifer
        okay, I'll check that. Do I need to setup anything else like run the indexer or solr core ?
      • 2020-09-13 25749, 2020

      • yvanzo
        no, you don't need the indexer at all
      • 2020-09-13 25745, 2020

      • pristine___
        shivam-kapila: no clue :(
      • 2020-09-13 25742, 2020

      • shivam-kapila
        a sec. I was having lunch
      • 2020-09-13 25733, 2020

      • yokel joined the channel
      • 2020-09-13 25745, 2020

      • yokel has quit
      • 2020-09-13 25738, 2020

      • _lucifer
        I have been able to figure out and fix some issues. 1) Increasing sleep time for 2s to 15s fixes some tests. 2) Some tests probably are broken because they were not updated with schema changes.
      • 2020-09-13 25740, 2020

      • _lucifer
        3) The current Solr codebase does need some changes to work with Java 11 because it does use JAXB (see MBXMLWriter).
      • 2020-09-13 25743, 2020

      • _lucifer
        yvanzo: ^
      • 2020-09-13 25730, 2020

      • ishaanshah
        iliekcomputers: Yesterdays dump didn't get imported
      • 2020-09-13 25745, 2020

      • ishaanshah
        Any reason why, I didnt see any error on sentry
      • 2020-09-13 25715, 2020

      • iliekcomputers
        I think the recommendation job broke something
      • 2020-09-13 25723, 2020

      • ishaanshah
        Oh, can you retrigger the import
      • 2020-09-13 25733, 2020

      • ishaanshah
        Otherwise we'll skip one days data
      • 2020-09-13 25704, 2020

      • pristine___
        > request consumers should be restarted I think. Train models failed with RPC client dissociated erroe
      • 2020-09-13 25718, 2020

      • pristine___
        iliekcomputers: request consumer should be restarted I think.
      • 2020-09-13 25716, 2020

      • yokel joined the channel
      • 2020-09-13 25705, 2020

      • yvanzo
        !m _lucifer
      • 2020-09-13 25705, 2020

      • BrainzBot
        You're doing good work, _lucifer!
      • 2020-09-13 25758, 2020

      • yokel has quit
      • 2020-09-13 25720, 2020

      • shivam-kapila
        pristine___: i will dig in
      • 2020-09-13 25730, 2020

      • yokel joined the channel
      • 2020-09-13 25741, 2020

      • yokel has quit
      • 2020-09-13 25729, 2020

      • pristine___
        Thanks shivam-kapila
      • 2020-09-13 25744, 2020

      • yokel joined the channel
      • 2020-09-13 25746, 2020

      • iliekcomputers
        Request consumer restarted