我正在努力通过这个基本工作流程将 SBT 用于 CI 过程:
~/.sbt 和 ~/.ivy2/cache target 目录 在后续步骤中:
~/.sbt 和 ~/.ivy2/cache target 目录，其中包含 .class 文件和相同的源代码(应该是相同的 check out ) sbt test 100% 的情况下，
sbt test 会重新编译整个项目。我想了解或调试为什么会这样，因为自上次编译以来没有任何变化(好吧，什么都不应该改变，那么是什么让它相信某些东西？)我目前正在使用带有 docker executor 的 circleci。这意味着有一个新的 docker 实例，来自同一个镜像，运行每个步骤，尽管我希望缓存可以解决这个问题。
.circleci/config.yml的相关部分(如果你不使用圆，这应该仍然可以理解；我已经注释了我能做的):---
version: 2
jobs:
# compile and cache compilation
test-compile:
working_directory: /home/circleci/myteam/myproj
docker:
- image: myorg/myimage:sbt-1.2.8
steps:
# the directory to be persisted (cached/restored) to the next step
- attach_workspace:
at: /home/circleci/myteam
# git pull to /home/circleci/myteam/myproj
- checkout
- restore_cache:
# look for a pre-existing set of ~/.ivy2/cache, ~/.sbt dirs
# from a prior build
keys:
- sbt-artifacts-{{ checksum "project/build.properties"}}-{{ checksum "build.sbt" }}-{{ checksum "project/Dependencies.scala" }}-{{ checksum "project/plugins.sbt" }}-{{ .Branch }}
- restore_cache:
# look for pre-existing set of 'target' dirs from a prior build
keys:
- build-{{ checksum "project/build.properties"}}-{{ checksum "build.sbt" }}-{{ checksum "project/Dependencies.scala" }}-{{ checksum "project/plugins.sbt" }}-{{ .Branch }}
- run:
# the compile step
working_directory: /home/circleci/myteam/myproj
command: sbt test:compile
# per: https://www.scala-sbt.org/1.0/docs/Travis-CI-with-sbt.html
# Cleanup the cached directories to avoid unnecessary cache updates
- run:
working_directory: /home/circleci
command: |
rm -rf /home/circleci/.ivy2/.sbt.ivy.lock
find /home/circleci/.ivy2/cache -name "ivydata-*.properties" -print -delete
find /home/circleci/.sbt -name "*.lock" -print -delete
- save_cache:
# cache ~/.ivy2/cache and ~/.sbt for subsequent builds
key: sbt-artifacts-{{ checksum "project/build.properties"}}-{{ checksum "build.sbt" }}-{{ checksum "project/Dependencies.scala" }}-{{ checksum "project/plugins.sbt" }}-{{ .Branch }}-{{ .Revision }}
paths:
- /home/circleci/.ivy2/cache
- /home/circleci/.sbt
- save_cache:
# cache the `target` dirs for subsequenet builds
key: build-{{ checksum "project/build.properties"}}-{{ checksum "build.sbt" }}-{{ checksum "project/Dependencies.scala" }}-{{ checksum "project/plugins.sbt" }}-{{ .Branch }}-{{ .Revision }}
paths:
- /home/circleci/myteam/myproj/target
- /home/circleci/myteam/myproj/project/target
- /home/circleci/myteam/myproj/project/project/target
# in circle, a 'workflow' undergoes several jobs, this first one
# is 'compile', the next will run the tests (see next 'job' section
# 'test-run' below).
# 'persist to workspace' takes any files from this job and ensures
# they 'come with' the workspace to the next job in the workflow
- persist_to_workspace:
root: /home/circleci/myteam
# bring the git checkout, including all target dirs
paths:
- myproj
- persist_to_workspace:
root: /home/circleci
# bring the big stuff
paths:
- .ivy2/cache
- .sbt
# actually runs the tests compiled in the previous job
test-run:
environment:
SBT_OPTS: -XX:+UseConcMarkSweepGC -XX:+UnlockDiagnosticVMOptions -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -Duser.timezone=Etc/UTC -Duser.language=en -Duser.country=US
docker:
# run tests in the same image as before, but technically
# a different instance
- image: myorg/myimage:sbt-1.2.8
steps:
# bring over all files 'persist_to_workspace' in the last job
- attach_workspace:
at: /home/circleci/myteam
# restore ~/.sbt and ~/.ivy2/cache via `mv` from the workspace
# back to the home dir
- run:
working_directory: /home/circleci/myteam
command: |
[[ ! -d /home/circleci/.ivy2 ]] && mkdir /home/circleci/.ivy2
for d in .ivy2/cache .sbt; do
[[ -d "/home/circleci/$d" ]] && rm -rf "/home/circleci/$d"
if [ -d "$d" ]; then
mv -v "$d" "/home/circleci/$d"
else
echo "$d does not exist" >&2
ls -la . >&2
exit 1
fi
done
- run:
# run the tests, already compiled
# note: recompiles everything every time!
working_directory: /home/circleci/myteam/myproj
command: sbt test
no_output_timeout: 3900s
workflows:
version: 2
build-and-test:
jobs:
- test-compile
- test-run:
requires:
- test-compile
第二阶段的输出通常如下所示:
#!/bin/bash -eo pipefail
sbt test
[info] Loading settings for project myproj-build from native-packager.sbt,plugins.sbt ...
[info] Loading project definition from /home/circleci/myorg/myproj/project
[info] Updating ProjectRef(uri("file:/home/circleci/myorg/myproj/project/"), "myproj-build")...
[info] Done updating.
[warn] There may be incompatibilities among your library dependencies; run 'evicted' to see detailed eviction warnings.
[info] Compiling 1 Scala source to /home/circleci/myorg/myproj/project/target/scala-2.12/sbt-1.0/classes ...
[info] Done compiling.
[info] Loading settings for project root from build.sbt ...
[info] Set current project to Piranha (in build file:/home/circleci/myorg/myproj/)
[info] Compiling 1026 Scala sources to /home/circleci/myorg/myproj/target/scala-2.12/classes ...
我能做些什么来确定为什么这是第二次重新编译所有源并缓解它？
我在 linux 容器中运行 sbt 1.2.8 和 scala 2.12.8。
更新
我还没有解决这个问题，但我想我会分享一个解决我最严重问题的方法。
主要问题:将“测试编译”与“测试运行”分开
次要问题:更快的构建，而不必在每次推送时重新编译所有内容
我对次要没有解决办法。对于初级:
我可以通过
scala -cp ... org.scalatest.tools.Runner 而不是通过 sbt test 从 CLI 运行 scalatest runner 以避免任何重新编译的尝试。运行程序可以对 .class 文件的目录进行操作。变更概要:
sbt test:compile 'inspect run' 'export test:fullClasspath' | tee >(grep -F '.jar' > ~test-classpath.txt)scala -cp VALUE_HERE 运行测试 scala -cp "$(cat test-classpath.txt)" org.scalatest.tools.Runner -R target/scala-2.12/test-classes/ -u target/test-reports -oD.class 中编译的 target/scala-2.12/test-classes 文件，使用在编译阶段报告的类路径，并打印到标准输出以及报告目录 1121423我不喜欢这个，它有一些问题，但我想我会分享这个解决方法。
最佳答案
如果您使用比 1.0.4 更新的 sbt 版本，则缓存对您不起作用，因为编译器将始终使所有内容无效。
此锌编译器问题已在此处报告:https://github.com/sbt/sbt/issues/4168
我的建议是降级 CI 的 sbt 版本。还要检查和验证 CI 是否正在更改 .sbt 或 .ivy2 文件时间戳。如果它们被更改，请通过压缩和解压缩它们来单独缓存它们。
我对 Bitbucket Pipelines CI 遇到了同样的问题，并成功使其工作 here
关于scala - sbt 总是在 CI 中重新编译整个项目，即使有缓存？，我们在Stack Overflow上找到一个类似的问题：https://stackoverflow.com/questions/55282629/