Plugins

How OpenFief describes, approves and runs a plugin, and how much of that the code implements today.

A plugin is how OpenFief reaches the outside world: a connector brings data in, a view shows the record somewhere else. Each plugin runs in its own container, out of the core’s process, and gets only what its vendor asked for and its owner approved. This page explains that model, what the pure plugins package builds from it, and the connector contract and Go SDK a plugin is written with. The Operator page covers how the operator applies it to a cluster, and Egress covers the proxy that enforces plugins’ outbound hosts.

Concepts#

Plugin and instance. A plugin is one installed, named instance of a plugin image. The same image can be installed twice under two names, such as one instance per IBKR account, and each instance has its own config, Secrets, grant and egress allow-list (D-48). Its ServiceAccount, objects, grant and allow-list entry are all keyed by its name. Secrets are the exception: their names come from the manifest, which must equal the image’s signed label, so every instance of one image mounts the same Secret names (see Decisions).

Capability. What a plugin may act as: connector or view in M0. A plugin that ends up with no approved capability has nothing it may do, so it does not run.

Manifest. The manifest is what a plugin declares: capabilities, egress hosts (hostname and port), Secrets with the path to mount each at, data tiers, and a JSON Schema for its config. D-42 puts it in a label on the image config, so one signature covers both the code and the permissions it asks for, and it can be read from the container registry without running the image. The Plugin resource carries a copy, which must equal the signed label.

Requested, granted and effective permissions. The principle of least privilege (Saltzer and Schroeder, 1975) says every program should hold only the privileges its job needs. Capability-based security (Dennis and Van Horn, 1966) goes further: a permission is something you hold and can be shown, not something looked up from who you are. OpenFief splits a plugin’s permissions in two (D-16). The Plugin resource carries what the vendor requests; the grant, a PluginGrant resource, carries what the owner approved. Only what both hold is enforced: the plugin’s effective permissions. A grant that approves more than requested adds nothing, and a request beyond the grant gets nothing.

Pending approval and rollout. When a new version of a plugin requests something its grant lacks, that difference is pending approval. The owner approves it by adding it to the grant. Until then the version that already ran keeps running, limited to the current grant. Which version runs, if any, is the plugin’s rollout. Upgrades are always manual, even when the permissions don’t change (D-47): the owner changes the image, nothing else does.

Plugin identity. Each instance runs as the ServiceAccount plugin-<name> in the openfief namespace and proves it with projected ServiceAccount tokens. That is its plugin identity. It presents a token with audience openfief-core to the core (D-27). A plugin with egress also gets a second token, audience openfief-egress, for the egress proxy, so the proxy never sees a token the core would accept (D-63). The Egress page explains projected tokens.

Image references and digest pinning. A tag such as :1.4 is a pointer that anyone who can push to the repository can move. A digest (sha256:...) is the hash of the image’s OCI manifest, which in turn hashes every layer, so it names exactly one image. Pulling by digest has been possible since the Docker Registry v2 API (2015). Pinning by digest means the image the owner approved is the image that runs, and it is what a signature covers. An image reference in OpenFief is always repository@sha256:<hex>, never a tag (D-17).

Signatures and trusted signers. Sigstore is an open project, started in 2021, for signing software; cosign is its tool for container images. With keyless signing, the signer proves an identity such as a CI workflow through OpenID Connect, gets a short-lived certificate for it, and the signature is recorded in a public transparency log. A verifier checks who signed instead of guarding long-lived keys. The operator accepts three kinds of trusted signer: OpenFief’s CI, publishers pinned in the catalog, and signers the owner adds (D-45). It checks every image itself, offline, with a bundled trust root (D-56). An image without a valid signature from a trusted signer is never deployed, with no override (D-57).

Catalog and verified plugins. The catalog is one file in this repository, released with the product, that lists first-party and verified plugins (D-43). A verified plugin is one whose catalog entry pins a version the author reviewed: the publisher’s signed image digest and signer identity (D-44). Anything installed by image reference alone is shown as unverified. There are no subscribable third-party catalogs; R-12 records what happened to OpenClaw’s open skill registry when anyone could publish to it.

Workload. The workload is everything the operator creates for one plugin, compiled from its effective permissions: identity, token, Secret mounts, environment, NetworkPolicy and egress allow-list (D-11). The Kubernetes Deployment is only one part of it.

Contracts. Plugins speak protobuf contracts over Connect RPC, which serves one service as gRPC or as plain HTTP with JSON (D-11). A connector serves the connector contract and the core calls it, to sync and to import files; the core owns the schedules, retries and import batches (D-49). Only the core may call a plugin, and it proves that with a short-lived token whose audience is that plugin instance (D-59). The SDK hides both the serving side and the token handling from a plugin’s author.

How it works#

Everything in internal/plugins is pure: no I/O, no Kubernetes types, no clock. The operator adapter in internal/plugins/operator (Operator) translates Plugin and PluginGrant resources into these types, asks them what to do, and turns the answer into Kubernetes objects.

From a plugin and its grant to a workload#

Validation. Plugin.Validate, Grant.Validate and Manifest.Validate return every problem at once, joined, each a *FieldError that names its field (such as manifest.egress[1]) and wraps a sentinel from errors.go for errors.Is. The rules:

• A plugin name is a DNS label that starts with a letter. The CRD caps it at 56 characters, so plugin-<name> still fits Kubernetes’ 63.
• A manifest has at least one capability, and only known ones. Lists may not repeat an entry.
• An egress host is a canonical hostname: lower-case ASCII, at least two labels, no trailing dot, other scripts in their xn-- form, port 443 when none is written. IP literals, wildcards and single-label names such as core are refused, because the proxy, not the plugin, resolves names.
• A Secret name follows Kubernetes’ rule. Its mount path is absolute and clean, isn’t /, doesn’t overlap another mount, and stays clear of /var/run/secrets, where tokens live.
• A data tier is 1 to 32 letters, digits, - or _. It is only a name: what a tier holds is undecided (P-33), and nothing enforces tiers yet.
• Config and config schema must be JSON. Checking the config against its schema needs a JSON Schema library, so the operator does it (operator.ValidateConfig, which lets $ref reach only the schema itself and the standard meta-schemas, never a file or URL).

Images. ParseImageRef accepts only repository@sha256:<64 lower-case hex digits>, with an optional registry host and port, following the OCI name grammar. A tag is refused even next to a digest, since an image named both ways is ambiguous. An ImageRef has unexported fields, so a parsed one is always valid.

Effective permissions. Effective(p, g) keeps the capabilities, egress hosts and data tiers that appear in both. Secrets match by name and keep the mount path the manifest asks for, so a vendor can move a mount without new approval. If no requested capability is granted it returns ErrNoCapabilityGranted. EffectivePermissions also has unexported fields and copying accessors: the only way to get one is through Effective.

Compiling. Compile(e, topology) is deterministic: every list is sorted, so the same input always gives the same Workload. It produces:

• a ServiceAccount plugin-<name> in openfief with AutomountAPIToken false and no Role: plugins never talk to the Kubernetes API;
• a projected token at /var/run/secrets/openfief/token, audience openfief-core, valid for an hour and refreshed by the kubelet;
• only when some egress host is granted, EgressToken: a second projected token at /var/run/secrets/openfief-egress/token, audience openfief-egress (D-63). The operator names its path in OPENFIEF_EGRESS_TOKEN_FILE;
• the granted Secrets, read-only;
• NetworkPolicy rules: ingress from the core on the plugin port; egress to the core’s pod port, plus the egress proxy’s pod port only when some egress host is granted. No rule ever allows DNS (D-32);
• HTTPS_PROXY=http://<proxy ClusterIP>:<port> and NO_PROXY=<core ClusterIP>, again only with egress hosts. Topology takes addresses, not names, because plugins have no DNS;
• the plugin’s EgressAllowList: exactly its effective egress hosts.

The operator’s BuildObjects maps this onto a ServiceAccount, a config ConfigMap, a ClusterIP Service, a NetworkPolicy that denies everything except those rules, and a single-replica Deployment with the Recreate strategy, so an old pod never runs beside a new one with other permissions.

Planning which version runs#

PlanRollout takes the version the Plugin asks for (Current), whether checks outside the package passed (CurrentDeployable), the version that runs now (Previous), and the grant. It returns the outcome, the effective permissions of whatever runs (Run), and Pending: exactly what Current requests beyond the grant, computed by NeedsApproval.

Because the previous version runs under the current grant, narrowing a grant takes effect at once, even while an upgrade waits. A first install that asks for more than its grant runs nothing.

The operator fills the input from the cluster: Current from spec, nil when the spec is invalid; CurrentDeployable only when the spec is valid, the config matches its schema and the image signature is verified; Previous from status.deployed. Before it runs the previous version it checks that image’s signature again. It writes Pending to status.pendingApproval with an ApprovalRequired condition.

A typical upgrade, from the owner’s side:

In a cluster, CurrentDeployable is false until the image passes the signature check in the next section, and a previous version runs only while its own image still passes it.

Signatures, trusted signers and the catalog#

Whether an image may run is decided by TrustPolicy.Decide in trust.go, a pure function over what an adapter found in the registry: a SignedImage with the identities whose signatures over the digest verified, and the manifest from the image’s signed label. The adapter, internal/plugins/cosign, finds cosign bundles attached to the image as OCI referrers, verifies them offline with sigstore-go against an embedded Sigstore trust root, and reads the label only from a signed image. It and the operator’s PolicyVerifier, which caches signed images by digest and retries an unreachable registry every minute, are on the Operator page.

• Signers. A Signer has a name, one of D-45’s three kinds (SignerOpenFiefCI, SignerPublisher, SignerOwner) and a SignerIdentity. A keyless identity is an https OIDC issuer plus either an exact subject or an RE2 SubjectPattern that must match the whole subject; a key identity is one PEM public key. SignerIdentity.Matches compares a verified signature’s identity with it, and a malformed identity matches nothing.
• OpenFief’s CI is always trusted: OpenFiefCI() is the GitHub Actions issuer with the exact subject of the images.yml workflow on main.
• The catalog is plugins.json, embedded in the operator binary with go:embed and read by catalog.Load (D-43). ParseCatalog reads it strictly: unknown fields, trailing data or any invalid entry refuse the whole file. An entry has a name, an image whose digest it pins, and a publisher with an exact identity, never a pattern (D-44). Only the digest counts, so a mirror of the same image is verified too. The file has no entries yet.
• Owner signers come from ParseOwnerSigners, with the same strictness; they may use a subject pattern or a public key. The operator reads them at start from the optional ConfigMap openfief-trusted-signers and refuses to start when the file is invalid.
• The policy. NewTrustPolicy(catalog, owner) trusts OpenFief’s CI, every catalog publisher and every owner signer. Decide returns ErrUnsigned without signatures and ErrUntrustedSigner when none is by a trusted signer. Otherwise the image is Verified when a catalog entry pins its digest and that entry’s publisher signed it, and Unverified when any other trusted signer did. With a declared manifest, it then returns ErrManifestMismatch unless Manifest.Equal holds. Nothing in the policy can admit an unsigned image (D-57).
• The manifest label is io.openfief.plugin.manifest on the image config. ParseManifestLabel reads the same fields as spec.manifest, refuses unknown fields and validates the result. Manifest.Equal compares capabilities, egress hosts, Secret mounts and data tiers as sets, and config schemas as JSON values with exact numbers.

Handing allow-lists to the egress proxy#

The operator publishes every running plugin’s egress allow-list in the ConfigMap openfief-egress-allowlist. EncodeAllowLists writes one key per plugin, valued as sorted host:port lines, each ending in a newline; a plugin without egress hosts gets an empty value. DecodeAllowLists is strict, because the proxy refuses everything it cannot read: lower case, port always written, no duplicates. Each plugin’s key is applied with its own server-side apply field manager, so one plugin’s reconcile never touches another’s entry. How the proxy uses the lists is on the Egress page.

ServiceAccountName and PluginForServiceAccount map between a plugin name and its Kubernetes username. The reverse map accepts only system:serviceaccount:openfief:plugin-<valid name>, so another namespace, a prefix trick such as openfief-x, or the core’s own ServiceAccount is never taken for a plugin.

The connector contract#

connector.proto defines openfief.connector.v1.ConnectorService, which a connector serves and the core calls (D-49):

• Sync(SyncRequest) pulls what the institution reported since the previous sync’s watermark, an opaque string the connector returns and the core stores once the delivery is imported.
• ImportFile(stream ImportFileRequest) maps one statement file. The first message is a FileHeader (batch ID, ingest mode, file name as a hint, media type); every later one carries the next bytes.

Both answer with a Delivery: the batch ID echoed back, the institution, and the connector’s mapping (transactions with their postings, balance assertions and ingest problems), plus any prices, FX rates and index readings. The file’s conventions keep money exact and the record clean: amounts, prices, rates and quantities are decimal strings, never floating point; dates are YYYY-MM-DD; currencies are ISO 4217 codes; account_id is OpenFief’s own ID, never the institution’s account number. Every call carries a batch ID the core generates and reuses on a retry, and the core prefixes each transaction’s source with the instance’s name, so one instance can never write over another’s transactions.

task proto:gen generates the Go code into sdk/go/gen, with protoc-gen-go and protoc-gen-connect-go pinned as local binaries (buf.gen.yaml). buf.yaml lints the contract with Buf’s STANDARD rules and checks breaking changes at FILE level; task proto runs lint, the breaking check against PROTO_BASE or DOCS_BASE (default HEAD), and a check that the generated code is up to date. The core’s side of the contract, which translates deliveries into domain values, is on the Ingest page.

The Go SDK#

sdk/go is its own Apache-2.0 module, github.com/arkadiuss/openfief/sdk/go, beside the AGPL core in go.work (D-5). Package openfief gives a plugin what the operator set up for it:

• Environment (env.go): CoreURL from OPENFIEF_CORE_ADDRESS (a ClusterIP and port; plugins have no DNS), ListenAddress from OPENFIEF_LISTEN_PORT, LoadConfig from OPENFIEF_CONFIG.
• Tokens (token.go): TokenFile reads a token file on every call, through an os.Root of its directory, because the kubelet rotates projected tokens. Its default is OPENFIEF_TOKEN_FILE, then /var/run/secrets/openfief/token.
• Serving the core (serve.go): NewConnectorServer(addr, handler, verifier) mounts the generated ConnectorService handler behind RequireCore and speaks HTTP/1.1 and HTTP/2 without TLS (h2c), since ImportFile streams. RequireCore answers 401 to any request whose Authorization: Bearer token the CoreVerifier refuses, before the handler sees it. The token’s audience is InstanceAudience(name), openfief-plugin:<instance>. ReceiveFile reads an ImportFile stream: exactly one header first, then chunks, up to a size limit the connector sets.
• Verifying the core (verifier.go): NewCoreVerifier builds the SDK’s CoreVerifier, a JWKSVerifier, from OPENFIEF_INSTANCE, OPENFIEF_CORE_SUBJECT and OPENFIEF_SA_JWKS_DIR (default /var/run/secrets/openfief-sa-jwks), which the operator sets. It checks the token with go-jose v4 against the cluster’s public keys in jwks.json and the issuer in issuer, both from the mounted openfief-sa-jwks ConfigMap (D-59). It allows only RS256, ES256, ES384 and ES512, the algorithms Kubernetes signs with, so none and HMAC are refused before any key is tried; it picks the key by the token’s key ID and keeps only public RSA and EC signing keys from the set. The claims must name the mounted issuer, the core’s ServiceAccount as subject and this instance’s audience, and carry an expiry, checked with a one-minute leeway. The files are read on every call and parsed again only when they change, so a rotated key works as soon as the kubelet updates the mount.
• Running (run.go): ServeConnector(ctx, handler) is a connector’s whole serving side: it reads the listen port, builds the CoreVerifier with NewCoreVerifier, serves until the context is done, then stops taking calls and lets the ones in flight finish for up to 20 seconds.
• Calling the core (core.go): CoreHTTPClient adds the current openfief-core token as a bearer token on every request and never uses a proxy.
• Reaching an institution (egress.go): NewEgressClient reads HTTPS_PROXY and OPENFIEF_EGRESS_TOKEN_FILE. Its EgressTransport rebuilds the proxy URL for every request with user plugin and the current egress token as the password; Go’s HTTP client sends that as a Basic Proxy-Authorization header on the CONNECT (D-63). It refuses plain http and refuses to send anything without a proxy.

A connector built on the SDK, answering one Sync:

First-party plugins#

First-party plugins live in plugins/, one directory per image: a thin main.go that wires an adapter from internal/ to the SDK, a manifest.json, and a Dockerfile. CI builds and signs every plugins/<name>/Dockerfile as ghcr.io/<owner>/<repo>/<name>, like the core’s images. The image’s final stage carries the manifest as its io.openfief.plugin.manifest label (D-42), so the signature covers it. The image runs as the numeric user 65532:65532 (distroless’s nonroot): the operator’s plugin pods set runAsNonRoot, and the kubelet refuses to start a container whose image names its user, since it can’t tell that a name isn’t root. A third-party plugin image needs a numeric USER for the same reason. manifest.json is the source: task plugins:labels writes it, compacted, into the Dockerfile, and task plugins:labels:check, part of task check, fails when a label or a demo Plugin’s spec.manifest differs from it (plugin-labels.sh). Each plugin’s TestManifest parses manifest.json with ParseManifestLabel and checks it asks for exactly what the plugin needs.

nbp-fx’s watermark is the date of the latest rates it delivered: a sync asks NBP for the days after it up to today (UTC), and keeps it when nothing new is published. ibkr-flex’s Flex Query sets its own period, and importing it again changes nothing (D-18), so it returns the watermark it was given. It reads the Flex token and query ID from their files on every sync and never logs them. Both answer Unavailable when the institution fails, so the core retries the batch, and InvalidArgument for input they can’t read. They write their deliveries with ingest/delivery (Ingest).

Decisions#

Implemented, fully or in the part named:

• D-11 (part) Compiled manifests, a protobuf contract, the Go SDK. (RBAC compiles to none: a ServiceAccount without a Role.)
• D-13, D-15 Plugin resource; operator confined to openfief.
• D-16 Enforce only what request and grant share.
• D-17 Images pinned by digest, signatures verified with cosign bundles.
• D-27 (part) Plugins get, and the SDK sends, openfief-core tokens.
• D-32 (part) Plugins reach only core and proxy, no DNS.
• D-42 Manifest in a signed image label; spec.manifest must match it.
• D-43 (part) One catalog file, embedded in the operator; still empty.
• D-44 Catalog entries pin a digest and the publisher’s identity.
• D-45 Three signer kinds: OpenFief CI, catalog publishers, owner signers.
• D-47 Upgrades happen only when the owner edits.
• D-48 (part) Every instance keyed by its own name, except Secrets.
• D-49 (part) Connector contract: the core calls Sync and ImportFile.
• D-56 (part) Operator verifies offline; status tells verified from only signed.
• D-57 No setting deploys an image the policy refuses.
• D-59 Only the core reaches a plugin; the SDK verifies its TokenRequest token with the keys the operator publishes.
• D-63 (part) Separate egress token, rebuilt into the proxy URL per request.
• D-5 Contracts and Go SDK licensed Apache-2.0.

Worth knowing when you check the code against the decisions:

• D-42: the Plugin still carries the manifest in spec.manifest, and the operator compares it with the signed label instead of taking the label as the request. A Plugin that declares anything else is refused with ManifestMismatch, so the signed label is what counts.
• D-44: a catalog entry pins the digest, not the repository, so the same image pulled from a mirror is verified too.
• D-45: a catalog publisher is trusted for any image it signs; images the catalog doesn’t pin show as unverified.
• D-48 gives each instance its own Secrets, but Secret names come from the manifest, and D-42 makes the manifest the image’s. Two instances of one image, such as one per IBKR account, therefore ask for the same Secret names, and nothing in the code gives each its own.

Verification#

Run task test from the repository root; it tests both Go modules, the core and sdk/go, and installs the API server and etcd for the operator’s envtest suite. The pure tests alone run with go test ./internal/plugins/..., though the operator package then fails for lack of KUBEBUILDER_ASSETS; the SDK’s run with go test ./... inside sdk/go. Requirement IDs come from kibi (node_modules/.bin/kibi search "<question>"); each test file is also a kibi test entity, such as TEST-plugins-rollout.

The cosign adapter’s bundle checks (REQ-plugins-cosign-bundle-verification) and the operator’s use of the decision (REQ-operator-policy-verifier) are tested with the operator. The contract’s message rules (REQ-connector-contract-messages) are verified by TestConnectorServerSync here and by the core’s Connect adapter tests (TEST-ingest-connectrpc-client, TEST-ingest-connectrpc-mapping). To check by hand that an upgrade waits, read the “upgrade requesting a new host keeps the previous version” case in TestPlanRollout: the expected Run is the previous image with the old hosts, and Pending holds only the new host.

Not built yet#

• Verified plugins (D-43, D-44). The catalog file exists but lists no plugin, so every image that runs today is unverified.
• Showing trust (D-56). The SignatureVerified condition tells verified from only signed, but there is no UI yet to show it.
• The openfief CLI (D-46). There is no cmd/openfief; Plugin and PluginGrant are applied by hand.
• More contracts and SDKs (D-11). Only the connector contract exists: no view contract (P-9 outlines one, but it is a proposal) and no TypeScript SDK.
• First-party plugins in the catalog (D-43). nbp-fx and ibkr-flex are built and signed by CI but not in the catalog, so they run as signed by a trusted signer, not verified.
• One Secret per IBKR instance (D-48). ibkr-flex’s manifest names one Secret, ibkr-flex-credentials, so two instances of it, one per IBKR account, read the same credentials (see Decisions).
• Core side of plugin identity (D-27). Plugins get openfief-core tokens and the SDK sends them, but nothing in the core verifies them yet.
• Uninstall (D-58). Deleting a Plugin removes its objects by garbage collection and drops its allow-list entry. Deleting its grant and Secrets, and marking its data as coming from a removed instance, are not built.
• Data tiers. Carried from manifest to workload but enforced nowhere (P-33 is undecided).

Source: docs/concepts/plugins.md · last changed 2026-10-28 in 4152730