[bookbrainz-site] dependabot[bot] opened pull request #640 (master…dependabot/npm_and_yarn/less-loader-9.0.0): chore(deps-dev): bump less-loader from 7.2.1 to 9.0.0 https://github.com/bookbrainz/bookbrainz-site/p...
[bookbrainz-site] dependabot[bot] opened pull request #641 (master…dependabot/npm_and_yarn/superagent-6.1.0): chore(deps): bump superagent from 5.3.1 to 6.1.0 https://github.com/bookbrainz/bookbrainz-site/p...
[musicbrainz-server] y-young opened pull request #2114 (master…MBS-11680): MBS-11680: New UI for external links editor to display relationships as a list https://github.com/metabrainz/musicbrainz-serve...
[bookbrainz-site] dependabot[bot] opened pull request #642 (master…dependabot/npm_and_yarn/eslint-7.27.0): chore(deps-dev): bump eslint from 7.13.0 to 7.27.0 https://github.com/bookbrainz/bookbrainz-site/p...
[bookbrainz-site] dependabot[bot] opened pull request #643 (master…dependabot/npm_and_yarn/babel/register-7.13.16): chore(deps-dev): bump @babel/register from 7.10.1 to 7.13.16 https://github.com/bookbrainz/bookbrainz-site/p...
atj: my understanding of rsync is that source/ with a trailing / will copy only the contents of the directory. However, see here. rsync -a is also copying the permissions of the containing directory. from your experience is that expected?
ruaok
alastairp: -a means archive, which means yes, copy perms
to not copy perms, try just -r ?
alastairp
ruaok: I could understand that if it was `rsync src dest`, but this is `rsync src/ dest` which explicitly is "copy files in the directory, not the directory"
DjSlash
-a implies other options too
alastairp
I think it's easier (and more explicit) to just fix it with a chmod afterwards
ruaok
from what I understand -a/-r doesn't affect the behaviour of the trailing or not slashes.
DjSlash
indeed
alastairp
> You can think of a trailing / on a source as meaning "copy the contents of this directory" as opposed to "copy the directory by name", but in both cases th attributes of the containing directory are transferred to the containing directory on the destination.
that being said, it's public holiday so I'm planning on taking the rest of the day off. see you all tomorrow!
sumedh has quit
_lucifer
hi!
ruaok: do you want to trigger a full dump spark is again a week behind?
also i was wondering if it would be possible to make incremental dumps start dumping since the last dump instead of the fixed 24 hr period?
ruaok: thinking again, we dont need full dump as we have incremental dumps for most of last week, since permissions are now fixed we can import using the id.
!m alastairp
BrainzBot
You're doing good work, alastairp!
ruaok
_lucifer: if all the dumps are present and we can catch up, by all means lets do that.
_lucifer
yes, right. i'll trigger incremental dump imports in some time.
sumedh joined the channel
imports requested for incremental dumps from 440 to 445.
will request stats once the imports succeed.
ruaok
🤞
_lucifer
this will take at least 3 hrs i think. 😞
we move round all listens for each incremental dumps, not sure why but not the most efficient way to do it.
> we move round all listens for each incremental dumps, not sure why but not the most efficient way to do it.
we initially didn't see a way to "stream" listens to spark. but as you and I discussed last week, we could write more data to HDFS on the fly -- I think it is possible, really.
_lucifer
indeed that should be possible. i'll be taking a look into it soon.
ruaok
as for LB-899, not really. our not ideal fallback is relying on the dumps and then losing some data.
we should probably come up with one soon and work on putting it in place the short/mid term methinks :)
fnurl joined the channel
ruaok
I'm open for suggestions.
_lucifer
is there any this sort of plan for MB?
ruaok
the main DB for LB is being backed up via Barman.
the listens DB is not.
_lucifer
can we back listens db with barman as well?
ruaok
I'm sure we can. we'd need to buy more diskspace for the backup server in the office, I think.
something to chat to zas about, but he's already got loads on his plate.
_lucifer
yup, right.
i cannot think what else is needed for recovery but there are probably some other things i am forgetting.
zas
actually Barman is more a bitmap-thing ;) we'll prolly need more diskspace for that though, perhaps time to think about a dedicated barman machine if we go this way (which, imho, makes sense)
ruaok
yes, indeed.
we need a tiny VM instance with 1 or 2 vCPU and a BOAT LOAD of diskspace.
still a very odd/expensive thing to have, I see.
but this pattern fits our LB listen backup AND for our own docker registry.
zas
we could use a hetnzer physical machine + huge disks too
Hi There! I'm running MusicBrainz server on my Ubuntu 14.4 server as a slave and received message about schema change (to version 26). My problem is perl version ... in 14.4 i'm running on 5.18 and 5.30 is required. As i know upgrading perl can be a very tricky ... do anybody know any way or do i need to upgrade ubuntu to newer version (18
probably) ?
_lucifer
afaik, its recommended to run the server using docker. i think it would be better if you could switch to that.
yvanzo or bitmap might be able to help you out either way.
yvanzo
PetrCBR: I would recommend to upgrade ubuntu focal (20.04LTS).
+to
atj
alastairp: sorry, was getting vaccinated! Looks like your question has been answered though.
Didn't anticipate the permissions issue with rsync
_lucifer
🎉🥳
reosarevok
yvanzo: thanks for the reviews, I'll check and merge stuff :)
_lucifer
ruaok: thoughts on adding Zeppelin or Jupyter Notebook on leader? if we set that up we should be able to ssh into the cluster and explore it interactively.
ruaok
go for it.
_lucifer
👍 thanks!
one question, if during setup i mistakenly set it up on 0.0.0.0 firewall will block it right?
yyoung
yvanzo: Are there any instructions on how to run tests(js and selenium) using musicbrainz-docker? Do I have to install Chrome driver in musicbrainz container?