[bookbrainz-site] dependabot[bot] opened pull request #640 (master…dependabot/npm_and_yarn/less-loader-9.0.0): chore(deps-dev): bump less-loader from 7.2.1 to 9.0.0 https://github.com/bookbrainz/bookbrainz-site/pul…
2021-05-24 14447, 2021
BrainzGit
[bookbrainz-site] dependabot[bot] opened pull request #641 (master…dependabot/npm_and_yarn/superagent-6.1.0): chore(deps): bump superagent from 5.3.1 to 6.1.0 https://github.com/bookbrainz/bookbrainz-site/pul…
2021-05-24 14410, 2021
BrainzGit
[musicbrainz-server] y-young opened pull request #2114 (master…MBS-11680): MBS-11680: New UI for external links editor to display relationships as a list https://github.com/metabrainz/musicbrainz-server/…
2021-05-24 14458, 2021
BrainzGit
[bookbrainz-site] dependabot[bot] opened pull request #642 (master…dependabot/npm_and_yarn/eslint-7.27.0): chore(deps-dev): bump eslint from 7.13.0 to 7.27.0 https://github.com/bookbrainz/bookbrainz-site/pul…
[bookbrainz-site] dependabot[bot] opened pull request #643 (master…dependabot/npm_and_yarn/babel/register-7.13.16): chore(deps-dev): bump @babel/register from 7.10.1 to 7.13.16 https://github.com/bookbrainz/bookbrainz-site/pul…
atj: my understanding of rsync is that source/ with a trailing / will copy only the contents of the directory. However, see here. rsync -a is also copying the permissions of the containing directory. from your experience is that expected?
2021-05-24 14438, 2021
ruaok
alastairp: -a means archive, which means yes, copy perms
2021-05-24 14406, 2021
ruaok
to not copy perms, try just -r ?
2021-05-24 14418, 2021
alastairp
ruaok: I could understand that if it was `rsync src dest`, but this is `rsync src/ dest` which explicitly is "copy files in the directory, not the directory"
2021-05-24 14444, 2021
DjSlash
-a implies other options too
2021-05-24 14449, 2021
alastairp
I think it's easier (and more explicit) to just fix it with a chmod afterwards
2021-05-24 14424, 2021
ruaok
from what I understand -a/-r doesn't affect the behaviour of the trailing or not slashes.
2021-05-24 14436, 2021
DjSlash
indeed
2021-05-24 14440, 2021
alastairp
> You can think of a trailing / on a source as meaning "copy the contents of this directory" as opposed to "copy the directory by name", but in both cases th attributes of the containing directory are transferred to the containing directory on the destination.
that being said, it's public holiday so I'm planning on taking the rest of the day off. see you all tomorrow!
2021-05-24 14434, 2021
sumedh has quit
2021-05-24 14420, 2021
_lucifer
hi!
2021-05-24 14438, 2021
_lucifer
ruaok: do you want to trigger a full dump spark is again a week behind?
2021-05-24 14448, 2021
_lucifer
also i was wondering if it would be possible to make incremental dumps start dumping since the last dump instead of the fixed 24 hr period?
2021-05-24 14445, 2021
_lucifer
ruaok: thinking again, we dont need full dump as we have incremental dumps for most of last week, since permissions are now fixed we can import using the id.
2021-05-24 14451, 2021
_lucifer
!m alastairp
2021-05-24 14451, 2021
BrainzBot
You're doing good work, alastairp!
2021-05-24 14424, 2021
ruaok
_lucifer: if all the dumps are present and we can catch up, by all means lets do that.
2021-05-24 14424, 2021
_lucifer
yes, right. i'll trigger incremental dump imports in some time.
2021-05-24 14417, 2021
sumedh joined the channel
2021-05-24 14400, 2021
_lucifer
imports requested for incremental dumps from 440 to 445.
2021-05-24 14411, 2021
_lucifer
will request stats once the imports succeed.
2021-05-24 14420, 2021
ruaok
🤞
2021-05-24 14448, 2021
_lucifer
this will take at least 3 hrs i think. 😞
2021-05-24 14422, 2021
_lucifer
we move round all listens for each incremental dumps, not sure why but not the most efficient way to do it.
> we move round all listens for each incremental dumps, not sure why but not the most efficient way to do it.
2021-05-24 14425, 2021
ruaok
we initially didn't see a way to "stream" listens to spark. but as you and I discussed last week, we could write more data to HDFS on the fly -- I think it is possible, really.
2021-05-24 14408, 2021
_lucifer
indeed that should be possible. i'll be taking a look into it soon.
2021-05-24 14413, 2021
ruaok
as for LB-899, not really. our not ideal fallback is relying on the dumps and then losing some data.
we should probably come up with one soon and work on putting it in place the short/mid term methinks :)
2021-05-24 14450, 2021
fnurl joined the channel
2021-05-24 14421, 2021
ruaok
I'm open for suggestions.
2021-05-24 14452, 2021
_lucifer
is there any this sort of plan for MB?
2021-05-24 14409, 2021
ruaok
the main DB for LB is being backed up via Barman.
2021-05-24 14415, 2021
ruaok
the listens DB is not.
2021-05-24 14444, 2021
_lucifer
can we back listens db with barman as well?
2021-05-24 14416, 2021
ruaok
I'm sure we can. we'd need to buy more diskspace for the backup server in the office, I think.
2021-05-24 14431, 2021
ruaok
something to chat to zas about, but he's already got loads on his plate.
2021-05-24 14453, 2021
_lucifer
yup, right.
2021-05-24 14444, 2021
_lucifer
i cannot think what else is needed for recovery but there are probably some other things i am forgetting.
2021-05-24 14459, 2021
zas
actually Barman is more a bitmap-thing ;) we'll prolly need more diskspace for that though, perhaps time to think about a dedicated barman machine if we go this way (which, imho, makes sense)
2021-05-24 14445, 2021
ruaok
yes, indeed.
2021-05-24 14400, 2021
ruaok
we need a tiny VM instance with 1 or 2 vCPU and a BOAT LOAD of diskspace.
2021-05-24 14408, 2021
ruaok
still a very odd/expensive thing to have, I see.
2021-05-24 14429, 2021
ruaok
but this pattern fits our LB listen backup AND for our own docker registry.
2021-05-24 14402, 2021
zas
we could use a hetnzer physical machine + huge disks too
Hi There! I'm running MusicBrainz server on my Ubuntu 14.4 server as a slave and received message about schema change (to version 26). My problem is perl version ... in 14.4 i'm running on 5.18 and 5.30 is required. As i know upgrading perl can be a very tricky ... do anybody know any way or do i need to upgrade ubuntu to newer version (18
2021-05-24 14412, 2021
PetrCBR
probably) ?
2021-05-24 14429, 2021
_lucifer
afaik, its recommended to run the server using docker. i think it would be better if you could switch to that.
2021-05-24 14423, 2021
_lucifer
yvanzo or bitmap might be able to help you out either way.
2021-05-24 14457, 2021
yvanzo
PetrCBR: I would recommend to upgrade ubuntu focal (20.04LTS).
2021-05-24 14407, 2021
yvanzo
+to
2021-05-24 14430, 2021
atj
alastairp: sorry, was getting vaccinated! Looks like your question has been answered though.
2021-05-24 14436, 2021
atj
Didn't anticipate the permissions issue with rsync
2021-05-24 14447, 2021
_lucifer
🎉🥳
2021-05-24 14442, 2021
reosarevok
yvanzo: thanks for the reviews, I'll check and merge stuff :)
2021-05-24 14429, 2021
_lucifer
ruaok: thoughts on adding Zeppelin or Jupyter Notebook on leader? if we set that up we should be able to ssh into the cluster and explore it interactively.
2021-05-24 14431, 2021
ruaok
go for it.
2021-05-24 14451, 2021
_lucifer
👍 thanks!
2021-05-24 14427, 2021
_lucifer
one question, if during setup i mistakenly set it up on 0.0.0.0 firewall will block it right?
2021-05-24 14445, 2021
yyoung
yvanzo: Are there any instructions on how to run tests(js and selenium) using musicbrainz-docker? Do I have to install Chrome driver in musicbrainz container?
