16:58 PM
Sophist_UK joined the channel
17:00 PM
supersandro2000 has quit
17:00 PM
supersandro2000 joined the channel
17:00 PM
ruaok picked a bad time for a nap.
17:00 PM
Sophist-UK has quit
17:02 PM
zas
17:02 PM
Can MB provide such JWT tokens using current oauth stuff?
17:08 PM
yvanzo, bitmap: one thing prevents us to provide embedded bandcamp player, we have Bandcamp URLs for a streameable album, but not the Bandcamp Album ID (named in following BAID)
17:09 PM
BrainzGit
17:09 PM
zas
we can actually retrieve it from Bandcamp web pages
17:09 PM
because it is in head/meta (and in cookies)
17:10 PM
I think it would be actually useful to store it in db: release:mbid -> BAID
17:10 PM
what do you think?
17:13 PM
ruaok
130M listens imported, no degradation
17:14 PM
iliekcomputers
pristine__: hi
17:17 PM
Sophist-UK joined the channel
17:18 PM
ruaok: the newleader isn't a hetzner vm now, right?
17:18 PM
i'm seeing degraded performance on it while loading a new dump
17:18 PM
ruaok
hetzner bare metal.
17:18 PM
iliekcomputers
so i thought i'd restart the spark containers
17:19 PM
and now it's stuck at that for some reason.
17:19 PM
17:19 PM
ruaok
let see if the nodes are healthy
17:20 PM
Sophist_UK has quit
17:20 PM
iliekcomputers
restart worked
17:21 PM
i mean it restarted
17:21 PM
not sure of performance yet
17:22 PM
pristine__
iliekcomputers: hi
17:24 PM
iliekcomputers
pristine__: nvm, just saw your reply to my reply on the pr
17:25 PM
ruaok
iliekcomputers: remind me, what were our rules for listen uniqueness?
17:25 PM
(user, timestamp) ?
17:25 PM
(user, timestamp, recording_msid) ?
17:25 PM
iliekcomputers
B
17:25 PM
ruaok
K
17:26 PM
that will be fun to put that unique index on the old data.
17:26 PM
I guess I should make the importer handle it, since all the listens are sorted and doing a dup check would be rather simple.
17:27 PM
zas: I'm eyeing a dedicated time series database server in order to bring more capacity to LB.
17:28 PM
but we have no rackspace.
17:28 PM
could we get rid of another server or do we need to go do using VLANs?
17:28 PM
zas
I don't think we can get rid of any atm
17:30 PM
ruaok
I see a lot of underutilized machines when I look at stats.
17:31 PM
paco, serge, cage.
17:32 PM
paco, for instance.
17:32 PM
60GB of ram free. 8% CPU
17:33 PM
are you *sure* we can't forego one of these machines?
17:35 PM
the timescale import is 50% done, 35k rows/s still, 20 - 30% disk, 8% CPU
17:49 PM
yvanzo
zas: that could also be retrieved client-side using React.js
17:50 PM
zas: storing it in the DB will be possible with attributes, on my todolist.
17:52 PM
ruaok: can you please add me on trello?
17:52 PM
ruaok
what was your username?
17:53 PM
yvanzo
yvanzo
17:54 PM
ruaok
added to team and invited to that board.
17:54 PM
yvanzo
received, thanks!
18:08 PM
Etua has quit
18:20 PM
Etua joined the channel
18:20 PM
Protab joined the channel
18:20 PM
Rotab has quit
18:30 PM
zas
ruaok: I'll have a look on Monday, we can reduce a bit redundancy of mb services, and free up cage for example
18:31 PM
ruaok
as long as it doesn't lead to SPoF, then great!
18:32 PM
Protab is now known as Rotab
18:32 PM
Etua has quit
18:38 PM
Cyna
18:38 PM
Waiting for CI to complete it's testing
19:14 PM
eharris has quit
19:24 PM
Etua joined the channel
19:26 PM
ruaok
import complete 215 minutes, including building an index.
19:39 PM
rdswift
outsidecontext, ping.
19:41 PM
outsidecontext
rdswift: hi
19:41 PM
rdswift
Just wondering if you want me to have the PR for the Picard function documentation to go to metabrainz/picard-website/master or to the 2.3.1 branch on your clone.
19:42 PM
outsidecontext
please make it against the 2.3.1 branch. then we merge it and finally merge this 2.3.1 pull request
19:42 PM
rdswift
I didn't see a 2.3.1 branch on metabrainz
19:43 PM
outsidecontext
ah, I thought I had it pushed there :(
19:43 PM
rdswift
The PR from your clone points to the master branch on metabrainz
19:44 PM
Thus my confusion.
19:44 PM
outsidecontext
fixed now, branch is picard-2.3.1
19:44 PM
rdswift
Perfect! Thanks.
19:45 PM
Also, I thought I would just add the new functions to the documentation for now, and treat the restructure of the docs as a separate project.
19:45 PM
outsidecontext
yes, that makes sense
19:46 PM
rdswift
Thanks.
19:46 PM
outsidecontext
thank you :)
19:51 PM
Sophist-UK has quit
19:58 PM
rdswift
Not sure if it's an issue, but codacy is not set up to run on the new metabrainz/picard-website/2.3.1 branch so the testing on my PR is failing.
20:03 PM
tmontney joined the channel
20:05 PM
tmontney
20:05 PM
Running on Ubuntu 18.04, first issue started at cpanm --installdeps --notest .
20:06 PM
LWP::Protocol::https failed, but running sudo apt-get install libssl-dev solved it
20:06 PM
20:07 PM
however still fails due to version mismatch
20:07 PM
./admin/InitDb.pl --createdb --clean claims 'could not open extension control file "/usr/share/postgresql/11/extension/musicbrainz_collate.control"'
20:07 PM
but the file exists
20:09 PM
Instance is a fresh VMWare VM
20:09 PM
20:10 PM
ruaok
we actually suggest using this now:
20:10 PM
20:10 PM
20:10 PM
fire up a VM, install docker, then run that. wait. done.
20:10 PM
tmontney
Obviously I missed that but where was that on the site?
20:10 PM
20:11 PM
Saw "Download and/or install the database"
20:11 PM
the Setup section should include the docker option
20:11 PM
because that's awesome, that should be much easier
20:11 PM
ruaok
ok, correction. we should state that that is now the preferred version.
20:11 PM
yvanzo: don't you think?
20:12 PM
i'm super happy to get rid of VM images.
20:12 PM
tmontney
I'll give the docker option a try
20:12 PM
but is the Install.md still valid?
20:12 PM
ruaok
20:12 PM
tmontney
I assume it should be install docker or install manually
20:14 PM
ok
20:14 PM
ruaok
20:16 PM
iliekcomputers: so far I am really appreciating timescale. using it, it seems about as snappy as influx for the operations we need.
20:16 PM
but it has updates and it is postgres. the one hack we wont be able to get rid of is the per user listen count time series.
20:18 PM
tmontney
no I meant say someone didn't want to use docker
20:18 PM
is there a manual method still, or is this the only supported way?
20:18 PM
e.g. their distro doesn't support docker
20:19 PM
ruaok
only documented way. clearly it will always be feasible to install by hand, but it would require some small amount of insanity.
20:19 PM
its just grown too complex to realistically install by hand
20:37 PM
tmontney
fair enough
20:47 PM
Etua has quit
21:20 PM
eharris joined the channel
21:34 PM
one other question
21:34 PM
the whole reason I'm interested in musicbrainz is search
21:34 PM
ruaok
if you use the VM, search is built in.
21:34 PM
tmontney
there's an API right?
21:35 PM
ruaok
yep.
21:35 PM
tmontney
I have strings of text, some a bit garbled/partial
21:35 PM
that I'd like to find the closest match
21:35 PM
ruaok
it is exactly the one we use in production, but no rate limits.
21:35 PM
OH!
21:35 PM
really? I have a data set for you.
21:35 PM
tmontney
a data set?
21:35 PM
what for
21:36 PM
ruaok
given an artist and a recording string, it give give you a MB recording ID (and consistent release) and artist ID.
21:36 PM
21:36 PM
tmontney
A string could look like Man In The Box by ALICE IN CHAINS
21:37 PM
however, I believe they all come in as {Title} by {Artist}
21:37 PM
so i could do a simple string split
21:37 PM
ruaok
if you can reliably split then, this could work.
21:37 PM
it would be a simple index lookup.
21:37 PM
tmontney
I'd say like 95% the text is fully accurate
21:37 PM
but I'd rather not miss any
21:37 PM
ruaok
but it doesn't handle inexact searches. this data is based on all the data that has ever been submitted to listenbrainz.
21:38 PM
if you care about accuracy, the I would go with the VM with search.
21:38 PM
tmontney
Yeah, because otherwise it would've been like me hitting YouTube or Google and taking the first result as 100% accurate
21:39 PM
i shouldn't even say I care about accuracy
21:39 PM
ruaok
21:39 PM
tmontney
just "here's another opinion"
21:39 PM
interesting
21:40 PM
ruaok
iliekcomputers: 120k rows/s now. 4 threads. SSD is saturated.
21:40 PM
with dedup