Read API

How OpenFief serves its record to the built-in UI, view plugins and scripts as one read-only protobuf service, called over Connect or as REST under /v1, and how it totals valuations and builds the allocation.

The read API is how anything outside the core reads the record: sleeves, accounts, transactions, holdings, tax lots, cash, balance assertion checks, stored valuations and the allocation. The built-in UI and every view plugin are meant to read through it, and so can the owner’s own scripts. It only reads: data enters OpenFief through connectors, never through this API. This page explains the service, the pure rules behind its totals and the allocation, and the HTTP adapter that serves it. The core service, cmd/core, serves it on Service openfief-core, port http (8080).

Concepts#

A separate read side. Bertrand Meyer’s command-query separation (1988) says a method should either change state or answer a question, never both. Greg Young’s CQRS (around 2010) applies the same split to whole systems: the write path and the read path get separate models. OpenFief uses a light version of it. Data enters only through connectors’ ingest paths into the append-only transaction log (D-7, D-18). The read API answers from the ledger’s queries and from stored valuations, which are a read model: rows derived from the log, prices and rates, rebuildable at any time (D-36).

One service, two ways to call it. The API is one protobuf service, openfief.views.v1.ReadService, in read_service.proto. Protobuf (Google, open-sourced 2008) describes messages and services once and generates code for many languages. Each method also carries a google.api.http annotation, Google’s convention from its own APIs for mapping an RPC onto a REST path. Clients generated from the protobuf call the service over Connect RPC, gRPC or gRPC-Web. Everyone else calls plain REST with JSON under /v1, which Buf’s Vanguard library transcodes from the annotations into the same handlers. The OpenAPI 3.1 document, read_service.openapi.yaml, is generated from the same definitions by protoc-gen-connect-openapi, with its title and bearer scheme from read_service.openapi-base.yaml. So the Go types, the Connect service, the REST paths and the OpenAPI document can’t disagree.

Read-only by construction. Every method is unary, declares idempotency_level = NO_SIDE_EFFECTS, and maps to a GET under /v1 with no body. Over REST, any other HTTP method gets 405 Method Not Allowed with Allow: GET, before the token is even looked at. The Connect protocol still accepts its usual POST for unary calls, and also its GET form, which the no-side-effects option enables.

Money, quantities and dates on the wire. Money is a Money message: amount as a decimal string such as "-1234.56", never a JSON number, and currency as an ISO 4217 code. A JSON number is a binary float in most clients, which can’t hold every decimal amount exactly. Quantities are decimal strings too. Dates are YYYY-MM-DD strings with no time of day.

Original and reporting currency. Every amount keeps its original currency. Valuations, date totals and the allocation also carry a value in the reporting currency: fields named value_reporting and total_reporting, with a reporting_currency field on the valuation responses. Only PLN is accepted as the reporting currency today (see Decisions).

Valuations, totals and their gaps. The API reads the holding valuations the valuation context stores (Valuation); it computes no prices itself. A holding valuation that lacks an input (a price, an FX rate or an index reading) is left out of the account’s valuation list and counts in no total. How far each account is complete is its valued-through date, which GetValuationStatus and ListValuationStatuses return with whether a revaluation is pending or running. An account never valued has an empty valued_through. Stored valuations cover holdings only, never cash, so value-over-time totals don’t include cash.

Allocation. The allocation is the portfolio’s value on one date split into groups by one dimension: asset class, currency, institution or sleeve. Each allocation item is one valued holding or one cash balance of an account. A holding counts at its stored valuation on that date; cash, which has no stored valuation, is converted here at the FX rate for the date. A holding with no valued holding valuation on the date is not counted but listed in unvalued, so a chart can say what it is missing instead of silently showing less. The asset class is cash, treasury_bond for a Polish ISIN whose venue symbol is a bond series code such as EDO0536, and listed for anything else. Groups come largest first, then by key.

Paging transactions. ListTransactions returns the current version of each transaction in date, then source, then external ID order. A page that has more after it returns a nextPageToken: an opaque cursor naming the last transaction’s date and key, base64url-encoded. Passing it back starts the next page right after that transaction, so a client reads every current transaction once even if a page boundary falls between transactions on the same date. This is keyset paging: an offset (“skip 200”) skips or repeats a row when rows arrive between two requests, while a cursor that names the last row read doesn’t. The parameter names follow Google’s API improvement proposal AIP-158.

Bearer tokens. Every call, over any protocol, needs an Authorization: Bearer <token> header. The token is meant to be either a script token or a plugin’s projected ServiceAccount token (plugin identity). The API asks a TokenVerifier whether the token may read. The core’s verifier accepts script tokens only; plugins’ tokens are not accepted yet.

How it works#

The views context follows the layering in code-organization.md:

The generated Go code for the service lives with the other contracts in sdk/go/gen/openfief/views/v1 (Plugins). From the contract to the ports:

A request, from HTTP to the ports#

connectapi.New(views, verifier, opts...) builds one http.Handler. The ports are declared in views/app, by the consumer. Each is shaped like a method the other contexts already have, and cmd/core passes *ledgerapp.Ledger as Ledger and *valuation/postgres.Store as Valuations, ValuationStatuses and FXRates.

The handler parses each request field: dates with kernel.ParseDate, the page token with views.ParseTransactionCursor, and enums against the domain’s constants. An unspecified allocation dimension or an unknown assertion status is an invalid argument. TestEnumsMatch checks that every domain enum and its protobuf twin share their numbers.

Errors, logs and headers#

Errors never carry their own text to the client, because it can quote an account ID, an amount or what was asked. toConnectError maps them to Connect codes with fixed texts, and Vanguard turns the codes into HTTP statuses for REST:

withRequestLog gives each request a random ID, returns it as X-Request-ID, and logs one line with only request_id, the HTTP method and the status. The path is never logged: it can name an account. A failure adds a line with request_id and while (answering or verifying a token), and not the error, for the same reason. withSecurityHeaders sets Cache-Control: no-store, Content-Security-Policy: default-src 'none'; frame-ancestors 'none', Cross-Origin-Resource-Policy: same-origin, Referrer-Policy: no-referrer, X-Content-Type-Options: nosniff and X-Frame-Options: DENY on every response, errors and 405s included.

Account queries#

Sleeves, accounts, holdings, lots, cash and balance assertion checks come straight from the Ledger port; Ledger explains how they are derived. A query about an account the ledger doesn’t store fails with ErrUnknownAccount. Any other store failure stays a failure, so a broken database never looks like a missing account. AssertionChecks filters to one status when asked. Transactions loads the current transactions and hands them to PageTransactions, which sorts a copy, starts after the cursor, and returns at most size of them with a cursor when more follow.

Valuations and totals#

AccountValuations(account, from, to) and ValuationTotals(from, to) first check the range with ValidateRange: to may not be before from, nor more than MaxRangeDays (3700, about ten years) after it. Then they load stored holding valuations through the Valuations port, for one account or for every account, and pass them to TotalsByDate. It sums, per date, each valued holding valuation’s converted value exactly in the reporting currency, and its original-currency value per currency, in currency code order. A converted value in any other currency is ErrNotReportingCurrency. AccountValuations also returns the valued holding valuations themselves, each with its basis, source, original value and reporting value.

The allocation#

ValueHoldings pairs each holding with the valued holding valuation of the same account, instrument and date, and takes its asset class from AssetClassOf. Cash is converted with the latest stored rate on or before the date, rounded to grosze like holding values (valuation.ValueRounding, half up), so holdings and cash add up alike. A cash currency with no rate fails the whole call rather than leaving that cash out. Allocate groups the items by the dimension’s key: the asset class name, the original currency code, the account’s institution or its sleeve ID. It sums each group and the total exactly, and sorts the groups.

So on the same date the allocation’s total includes cash and ListValuationTotals doesn’t, and the allocation leaves out unvalued holdings that a later revaluation would add.

Decisions#

• D-6: the read-only public API under /v1 with an OpenAPI spec. Built in this context; nothing calls it yet. MCP access is not part of it.
• D-50: protobuf with HTTP annotations, Connect plus REST through Vanguard, OpenAPI generated from the same definitions. Built as described above.
• D-7: data never enters through the public API. Enforced by the service definition and by REST answering 405 to anything but GET.
• D-30: the allocation by asset class, currency, broker or sleeve, and value over time. The code groups by institution where D-30 says broker, since the treasury bond registry is an institution and not a broker (see the glossary). Value over time is ListValuationTotals; the charts themselves are the UI’s.
• D-36: the read API reports, per account, the valued-through date and whether a revaluation is pending or running. Built as GetValuationStatus and ListValuationStatuses; the UI part is not built.
• D-62 and T-12: totals and the allocation are in the reporting currency, next to each original currency. app.New refuses any reporting currency but PLN with an *UnsupportedReportingCurrencyError, because stored valuations and FX rates are against PLN only (D-20) and T-12 (proposal P-49) is open.
• D-27: the bearer tokens the API accepts. The core accepts script tokens (Identity); plugins’ projected tokens are not accepted yet.
• Half-up rounding of cash in the allocation follows valuation.ValueRounding, which no decision fixes (see Valuation).

Verification#

These requirements refine the firm ones imported from D-6, D-7, D-30, D-36 #6 and D-50 (REQ-views-read-api-protobuf). Every test runs over in-memory fakes of the ports with synthetic accounts and ISINs, offline, under task test. The REST tests go through the real Vanguard transcoder and check each response body against gen/openapi with kin-openapi. task proto:check fails when the generated Go or OpenAPI differs from what the protobuf produces. Over the real stores, TestCoreServesTheReadAPI in cmd/core, under task test:pg, starts the core on a fresh PostgreSQL with part of the demo dataset, issues a script token with core token issue, decodes REST answers strictly into the protobuf messages, and gets 401 without a token or with an unknown or revoked one.

One text differs from the code: REQ-views-allocation-inputs says cash rounds half-even; the code rounds half up with valuation.ValueRounding, and the tests use amounts where the two agree.

Not built yet#

• D-27: accepting plugins’ projected ServiceAccount tokens (audience openfief-core); the core’s verifier accepts script tokens only.
• D-30, D-36: the built-in UI’s allocation and value-over-time charts and its valuation status display, and the view plugins that read through this API (D-3, D-4).
• Cash in value-over-time totals. Stored valuations have no cash, so ListValuationTotals counts holdings only.
• D-62: a reporting currency other than PLN, waiting on T-12.

Source: docs/concepts/read-api.md · last changed 2026-10-28 in 3606e5d