How do you review a WebAssembly plugin for AI agents?

A plugin for an AI agent that comes from somebody else's repository needs reviewing before anyone installs it. For a JSON manifest that review is manageable. The pull request shows the whole file, every action with its method and path, and a later change arrives as a diff of a few lines.

A WebAssembly module has no such shortcut. The catalogue entry holds a URL and a digest over compiled code, and the linked source sits next to it with nothing that connects the two.

A manifest is read in the diff, a module has to be rebuilt

Besides manifests and MCP configurations, the covey catalogue distributes modules with kind: "wasm": ordinary Go programs compiled with GOOS=wasip1 GOARCH=wasm. One of them is the vulndb plugin, which checks lock files against OSV.dev and was compiled into covey up to 0.5.0. Its version 1.0.0 is published as vulndb.wasm in release v0.7.0 of the covey-plugin-pack repository.

The spec puts the two cases side by side. A manifest can be reviewed in the pull request, because the diff shows the behaviour. A binary gets rebuilt instead.

The catalogue entry names the source, the Go version and the digest

For entries with kind: "wasm", the schema of the index repository covey-plugins requires a build block in every version. Its required fields are repo, ref and go; dir names a subdirectory and is optional. For vulndb the block holds the repository covey-plugin-pack, commit c6da03f0, Go 1.26.6 and the directory wasm/vulndb. Next to it sits the module's sha256, which starts with 85a9a640.

The guide for authors (CONTRIBUTING.md) asks for a tag or a commit in the artefact URL, and the schema describes the source ref the same way, as a tag or a commit. A URL that points at a branch passes CI once and then drifts away from its digest. That is why, for some URL forms on GitHub, GitLab and Gitea, build_catalog.py recognises artefact URLs that point at a branch such as main, master or trunk, and rejects them with an error.

The catalogue's CI rebuilds the module and compares the bytes

The workflow rebuild.yml runs on every pull request and every push to main in the index repository. For each version with a build block it clones the named repository and checks out the named ref. Then it downloads exactly the named Go version from go.dev, because a different compiler produces different bytes.

The build runs in the directory from dir, with -trimpath. The job compares the sha256 of the result with the digest in the entry and fails on a mismatch, with both values in the error. For vulndb that means CI compiles commit c6da03f0 with Go 1.26.6. It passes only if the result is again the module with digest 85a9a640.

This works because Go's wasm output is byte-identical under those conditions. The spec and the workflow both say this was verified across different machines and paths. Without -trimpath the build path ends up in the binary, and two builds of the same source differ.

So the template repository, covey-plugin-template, sets the flag in its Makefile. Its make build prints the digest and the Go version for the catalogue entry. Its own CI builds a second time in another directory on every push and compares the two results. A reviewer at the index then reads the source and checks that the code does what the description claims.

The instance rebuilds nothing, it checks against the entry and adds new modules disabled

The covey instance leaves the rebuild to the catalogue. When somebody installs from the catalogue, internal/marketplace fetches the artefact from the URL in the entry and computes its sha256. On a mismatch the install stops, and the error names the expected and the actual value.

The minimum version is checked as well. If an entry's covey_min_version is newer than the instance's known version, covey refuses the install; for vulndb that floor is 0.6.0. A newly installed module is stored in the database with its source, version and digest, and stays disabled until somebody in the organisation switches it on.

Rebuilding moves the trust to the merge into the index

An instance knows that the digest matches the source only from the index's CI. The catalogue uses no signatures, and the spec names the reviewed pull request as the anchor of trust. Whoever serves their own catalogue from somewhere else gets the rebuild only with a pipeline of their own.

There is an operating cost as well. The workflow rebuilds every wasm version in the catalogue on every run. If a source repository becomes unreachable, every later pull request to the index fails until the entry is changed or removed. And according to the guide, withdrawing a version prevents new installs only; instances that already have it keep it.

For a plugin like vulndb, the question of trust therefore moves from a compiled binary to two readable things with a history: a commit in the source repository and a merge in the index.