如何使用workspace_status_command的输出来构建自定义规则?

问题描述

bazel build标志--workspace_status_command支持调用脚本以检索例如存储库元数据，这也称为构建标记，并在java_binary之类的规则中可用.

The bazel build flag --workspace_status_command supports calling a script to retrieve e.g. repository metadata, this is also known as build stamping and available in rules like java_binary.

我想使用此元数据创建自定义规则.我想将其用于常见的支持功能.它应该接收git版本和一些其他属性，并创建一个可用作依赖项的version.go输出文件.

I'd like to create a custom rule using this metadata.I want to use this for a common support function. It should receive the git version and some other attributes and create a version.go output file usable as a dependency.

因此，我开始了探索各种bazel存储库中的规则的旅程.

So I started a journey looking at rules in various bazel repositories.

类似rules_docker的规则支持在container_image中使用stamp标记，并允许您在属性中引用状态输出.

Rules like rules_docker support stamping with stamp in container_image and let you reference the status output in attributes.

rules_go在go_binary的x_defs属性中支持它.

这对我来说是理想的选择，我挖了...

This would be ideal for my purpose and I dug in...

看起来我可以通过 ctx.actions.expand_template 使用 或ctx.version_file 作为substitutions的字典.但是我没有弄清楚如何获得这些文件的字典.这两个文件似乎是非官方的"文件，它们不是ctx文档的一部分.

It looks like I can get what I want with ctx.actions.expand_template using the entries in ctx.info_file or ctx.version_file as a dictionary for substitutions. But I didn't figure out how to get a dictionary of those files. And those two files seem to be "unofficial", they are not part of the ctx documentation.

基于已经发现的内容:如何根据状态命令输出获取dict?

Building on what I found out already: How do I get a dict based on the status command output?

如果不可能，从自定义规则访问workspace_status_command输出的最短/最简单的方法是什么?

If that's not possible, what is the shortest/simplest way to access workspace_status_command output from custom rules?

推荐答案

我一直在您的位置，而我最终遵循了您开始探索的道路.我生成了一个JSON描述，其中也包含了从git收集到的信息并打包到结果中，最终我做了这样的事情:

I've been exactly where you are and I ended up following the path you've started exploring. I generate a JSON description that also includes information collected from git to package with the result and I ended up doing something like this:

def _build_mft_impl(ctx):
    args = ctx.actions.args()
    args.add('-f')
    args.add(ctx.info_file)
    args.add('-i')
    args.add(ctx.files.src)
    args.add('-o')
    args.add(ctx.outputs.out)
    ctx.actions.run(
        outputs = [ctx.outputs.out],
        inputs = ctx.files.src + [ctx.info_file],
        arguments = [args],
        progress_message = "Generating manifest: " + ctx.label.name,
        executable = ctx.executable._expand_template,
    )

def _get_mft_outputs(src):
    return {"out": src.name[:-len(".tmpl")]}

build_manifest = rule(
        implementation = _build_mft_impl,
        attrs = {
            "src": attr.label(mandatory=True,
                              allow_single_file=[".json.tmpl", ".json_tmpl"]),
            "_expand_template": attr.label(default=Label("//:expand_template"),
                                           executable=True,
                                           cfg="host"),
        },
        outputs = _get_mft_outputs,
    )

在我的情况下，

//:expand_template是一个标签，它指向执行转换本身的py_binary.我很乐意了解一种更好的方法(更多本地方法，更少的跃点)，但是(暂时)我同意:它有效.关于该方法和您的担忧的评论很少:

//:expand_template is a label in my case pointing to a py_binary performing the transformation itself. I'd be happy to learn about a better (more native, fewer hops) way of doing this, but (for now) I went with: it works. Few comments on the approach and your concerns:

• 您自己无法读取(文件并在Skylark中执行操作)的AFAIK ...
• ...说到这，将转换(工具)和构建说明(框架)始终保持分开可能不是一件坏事.
• 可能会争论什么构成正式文档，但ctx.info_file可能未出现在参考手册中，而是在源代码树中进行了记录. :)其他领域也是如此(我希望不是因为这些接口还没有提交).

• AFAIK you cannot read in (the file and perform operations in Skylark) itself...
• ...speaking of which, it's probably not a bad thing to keep the transformation (tool) and build description (bazel) separate anyways.
• It could be debated what constitutes the official documentation, but ctx.info_file may not appear in the reference manual, it is documented in the source tree. :) Which is case for other areas as well (and I hope that is not because those interfaces are considered not committed too yet).

为了src/main/java/com/google/devtools/build/lib/skylarkbuildapi/SkylarkRuleContextApi.java中的完整性，

@SkylarkCallable(
  name = "info_file",
  structField = true,
  documented = false,
  doc =
  "Returns the file that is used to hold the non-volatile workspace status for the "
      + "current build request."
)
public FileApi getStableWorkspaceStatus() throws InterruptedException, EvalException;

---

评论中要求的其他一些细节.

---

few extra details as asked in the comment.

在我的workspace_status.sh中，例如，以下行:

In my workspace_status.sh I would have for instance the following line:

echo STABLE_GIT_REF $(git log -1 --pretty=format:%H)

在我的.json.tmpl文件中，我将拥有:

In my .json.tmpl file I would then have:

"ref": "${STABLE_GIT_REF}",

我选择要替换类似于shell的文本表示法，因为它对许多用户来说很直观，而且易于匹配.

I've opted for shell like notation of text to be replaced, since it's intuitive for many users as well as easy to match.

关于替换，实际代码的相关部分(CLI不在此范围内)将是:

As for the replacement, relevant (CLI kept out of this) portion of the actual code would be:

def get_map(val_file):
    """
    Return dictionary of key/value pairs from ``val_file`.
    """
    value_map = {}

    for line in val_file:
        (key, value) = line.split(' ', 1)
        value_map.update(((key, value.rstrip('\n')),))
    return value_map


def expand_template(val_file, in_file, out_file):
    """
    Read each line from ``in_file`` and write it to ``out_file`` replacing all
    ${KEY} references with values from ``val_file``.
    """
    def _substitue_variable(mobj):
        return value_map[mobj.group('var')]
    re_pat = re.compile(r'\${(?P<var>[^} ]+)}')
    value_map = get_map(val_file)
    for line in in_file:
        out_file.write(re_pat.subn(_substitue_variable, line)[0])

---

这是Python脚本的方式，也是我将python脚本暴露给其他对象的方法.

---

This is how the Python script is how I expose the python script to rest of bazel.

py_binary(
    name = "expand_template",
    main = "expand_template.py",
    srcs = ["expand_template.py"],
    visibility = ["//visibility:public"],
)

这篇关于如何使用workspace_status_command的输出来构建自定义规则?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！