ruaok: Yes. I just pushed an updated and rebuilt image of PR 1390 for test.lb if you want to deploy it.
2021-05-06 12604, 2021
Mr_Monkey
I'm now looking at front-end tests that I borked during a merge.
2021-05-06 12608, 2021
alastairp
_lucifer: yeah, look at every other dockerfile in metabrainz that uses a pre-defined ARG in a LABEL, and you'll see that yvanzo follows this exact patter
2021-05-06 12622, 2021
alastairp
its... weird. but apparently they way that it has to work
2021-05-06 12652, 2021
alastairp
oh, did _lucifer fix the babel thing??!?
2021-05-06 12600, 2021
alastairp
!m _lucifer (and Mr_Monkey, I guess)
2021-05-06 12600, 2021
BrainzBot
You're doing good work, _lucifer (and Mr_Monkey, I guess)!
2021-05-06 12659, 2021
Mr_Monkey
I fixed it in another way that a drawback, so not as good :p
2021-05-06 12609, 2021
Mr_Monkey
that added a drawback*
2021-05-06 12622, 2021
alastairp
weiiird
2021-05-06 12607, 2021
alastairp
rebasing and testing 1424, then
2021-05-06 12649, 2021
alastairp
Mr_Monkey: oh, I ran into something testing on my other computer. new docker uses this thing called "buildkit", and it doesn't leave behind intermediate images like old "docker build" does, so this issue that we had about having old LABELs that don't make sense isn't an issue anymore
2021-05-06 12657, 2021
alastairp
because those intermediate layers appear to no longer exist
2021-05-06 12616, 2021
Mr_Monkey
Huh
2021-05-06 12643, 2021
Mr_Monkey
But there's no ill effects to setting it to blank?
2021-05-06 12625, 2021
alastairp
no. I've left the "setting it to blank" in place until I can work out how buildkit fits into the grand scheme of things
2021-05-06 12644, 2021
alastairp
It'll probably become a defult in future versions of docker, but until then let's keep the workaround in place
2021-05-06 12634, 2021
BrainzGit
[listenbrainz-server] alastair merged pull request #1424 (master…add-git-hash-label-last): Don't add GIT_COMMIT_SHA build to the label at the beginning of the build https://github.com/metabrainz/listenbrainz-server…
2021-05-06 12608, 2021
alastairp
_lucifer: ^ fixed - can you please add the tricks for the duplicate ARG to the spark PR?
nable to reserve cache with key layer-docker-layer-caching-ListenBrainz Unit Tests-d261fe9be9433846e3ce7055d0a028c5a11ad8e664d1ce7e81b5e00730bb7eea, another job may be creating this cache.
its added only to the prod image only to avoid invalidating the entire test setup.
2021-05-06 12632, 2021
alastairp
right, nice
2021-05-06 12657, 2021
alastairp
one thing - I've encountered an issue before in docker multistage builds where in a file like this, if you ask it to build test then it'll also do the items in prod
2021-05-06 12612, 2021
alastairp
I don't know if this is something that still happens. did you see anything like it?
2021-05-06 12622, 2021
alastairp
(sorry, I missed the fact that this was a multi-stage file)
2021-05-06 12632, 2021
_lucifer
yeah, i noticed that. but its only one step so i let it slide.
2021-05-06 12613, 2021
alastairp
run me through - why does the prod version not need java, but the test one does?
2021-05-06 12636, 2021
_lucifer
the prod version has java, spark and hadoop installed directly on the node
2021-05-06 12601, 2021
_lucifer
we only use docker to manage python dependencies
2021-05-06 12651, 2021
_lucifer
but i think i might be able to get that done without the docket image soon, at that point entire prod setup will be dockerless
2021-05-06 12619, 2021
alastairp
and you build a docker image with the dependencies and then copy them out of the docker image into the spark environment?
2021-05-06 12612, 2021
alastairp
yeah, I think it makes sense to either run everything in docker, or nothing. but we should also think about consistency - if local dev uses docker and production doesn't, is there a risk of things working in one and not the other?
2021-05-06 12617, 2021
_lucifer
no the spark installation is mounted to the docker image so it runs inside the container
