Usage Telemetry and API Lifecycle Governance

Apicurio Registry provides usage telemetry capabilities that track which client applications fetch which schema versions. This enables API lifecycle governance features including dead schema detection, consumer version drift alerts, and deprecation readiness analysis.

Overview

Usage telemetry tracks which client applications fetch which schema versions, enabling governance decisions such as dead schema detection, version drift alerts, and deprecation readiness.

Usage telemetry answers key governance questions:

Is anyone actually using this schema? Active/Stale/Dead classification based on actual fetch patterns.
Who uses which version? Consumer version heatmap showing adoption across client applications.
Is it safe to deprecate this version? Deprecation readiness reports listing affected consumers.
Are consumers falling behind? Version drift alerts when clients use outdated versions.

Usage tracking is server-side and zero-configuration for the client. The registry records usage events automatically when it serves schema content. Clients opt in by setting an HTTP header that identifies the application:

X-Registry-Client-Id: Identifies the client application (for example, payments-svc). Must contain only letters, digits, dots, hyphens, and underscores. Maximum 256 characters.
X-Registry-Operation: Optional. The operation type: SERIALIZE or DESERIALIZE.

If no X-Registry-Client-Id header is present, the request is not tracked.

Both globalId and contentId based schema fetches are tracked. Clients configured with IdOption.contentId are fully supported.

Referenced schemas (shared types, base schemas) inherit liveness from the schemas that directly reference them in the global usage summary. If UserEvent v3 is Active and references SharedTypes v1, then SharedTypes v1 is counted as Active in GET /admin/usage/summary, even if nobody ever fetched SharedTypes v1 itself. The per-artifact endpoints do not apply this rule, so check the schemas that reference a version before you deprecate it.

Usage telemetry is an experimental feature and is disabled by default. It requires apicurio.features.experimental.enabled=true in addition to apicurio.usage.telemetry.enabled=true. It is tested with the H2, PostgreSQL, MySQL, SQL Server and KafkaSQL storage types; the tests do not check how client identifiers compare under each database collation (see below). It does nothing on the read-only GitOps and KubernetesOps storage types: events are dropped and the usage endpoints return empty results.

How versions are classified

Each schema version is classified by the age of its most recent recorded fetch.

Active: The last fetch is no older than apicurio.usage.active-threshold-days (default 7 days).
Stale: The last fetch is older than the active threshold but no older than apicurio.usage.stale-threshold-days (default 30 days).
Dead: The last fetch is older than the stale threshold, so with the defaults a version whose last recorded fetch is more than 30 days old is Dead.

A version inherits the most recent fetch of any schema that directly references it, but only in the global usage summary counts. The per-artifact metrics, the heatmap and the deprecation readiness list the fetches recorded for that version itself.

The classification uses only the active and stale thresholds. apicurio.usage.dead-threshold-days is deprecated, has no effect, and will be removed in 4.0.0. It is still accepted, and the registry logs a warning at startup if it is set. The usage summary and the Active, Stale and Dead labels cover only versions that have at least one recorded fetch within the retention period, or that are referenced by a version that has one. A version with no recorded fetch and no such referrer, for example one that was never fetched or whose events were all removed by retention, is not counted in the summary and has no label. A version that was never fetched but is referenced by a fetched version is counted in the summary and takes the classification of its most recent referrer. The per-artifact metrics, heatmap and deprecation readiness never list a version without a recorded fetch of its own.

On KafkaSQL, every usage event is one record in the journal topic, and retention removes database rows but never shrinks the journal. A journal written by an earlier release with telemetry enabled can hold usage records that were skipped until now. The first start of this release applies them all, including events older than the retention period, and the next retention run deletes the expired ones. During a rolling upgrade, nodes that still run the earlier release log an error for every usage record they cannot read.

Events older than apicurio.usage.retention-days (default 180 days) are deleted on every aggregation run, which happens hourly by default (apicurio.usage.aggregation.every). Setting apicurio.usage.retention-days to 0 or less turns the cleanup off, and the events then accumulate without bound. Because events are deleted, a version whose last fetch is older than the retention period drops out of the counts instead of becoming Dead. Keep the retention period longer than the stale threshold, or Dead versions disappear.

A fetch by contentId counts for every version that has the same content, in any artifact or group, because the identifier does not say which version was meant. Identical content registered in several artifacts therefore shares its fetches.

Client identifiers are compared with the collation of the database column. On MySQL the default comparison is case-insensitive, so Payments and payments count as one client; on PostgreSQL they are two clients. SQL Server follows its database collation, which is usually case-insensitive; this was not checked.

Reading the consumer heatmap

The consumer version heatmap answers which client applications fetch which versions of one artifact.

Each row is a client application, identified by its X-Registry-Client-Id.
Each column is a version that at least one client fetched. The number in a cell is how many fetches that client made of that version within the retention period.
versionsBehind compares the newest version the client fetched with the newest version that any client fetched. A client on the newest version shows 0.
driftAlert is true when versionsBehind is at least apicurio.usage.drift-alert-threshold (default 2).

The heatmap lists only versions that were fetched. A newer version that nobody fetched yet does not count towards versionsBehind.

Fetch counts are not exact request counts. The server drops a fetch when the same client fetched the same schema less than 60 seconds earlier. The schema is identified by the identifier of the request, so a schema that a client fetches once by globalId and once by contentId produces two events in that period. The record of recent fetches that implements this is held in the memory of each registry instance, so with several instances behind a load balancer each instance counts separately. While that record holds 10,000 entries, the instance records no new events at all. The record is pruned only during a flush that has buffered events, and a full record buffers none, so by reading the code an instance in that state may not recover until it restarts. A busy deployment can therefore show lower counts than the real traffic, or stop counting.

Server configuration

The registry server exposes the following properties to configure usage telemetry.

Table 1. Apicurio Registry server properties for usage telemetry
Property	Type	Default	Description
`apicurio.usage.telemetry.enabled`	boolean	`false`	Primary switch for usage telemetry. Must be `true` for the server to record usage events.
`apicurio.usage.active-threshold-days`	int	`7`	A schema version fetched within this many days is classified as Active.
`apicurio.usage.stale-threshold-days`	int	`30`	A schema version whose last fetch is older than the active threshold but no older than this many days is classified as Stale. A version whose last fetch is older than this is classified as Dead.
`apicurio.usage.dead-threshold-days`	int	`90`	Deprecated since 3.4.0, has no effect, and will be removed in 4.0.0. The Dead classification starts where the Stale classification ends.
`apicurio.usage.drift-alert-threshold`	int	`2`	Number of versions behind latest before a consumer is flagged with a drift alert.
`apicurio.usage.retention-days`	int	`180`	Number of days to retain raw usage events. Older events are cleaned up automatically.
`apicurio.usage.aggregation.every`	duration	`1h`	How often the usage metrics are refreshed and old events are cleaned up. Uses Quarkus scheduler expression syntax (for example, `1h`, `30m`, or `10s`).
`apicurio.usage.flush.every`	duration	`10s`	How often buffered usage events are written to storage. Uses Quarkus scheduler expression syntax.

Client configuration

The SerDes client exposes the following property to configure usage telemetry reporting.

Table 2. SerDes client property for usage telemetry
Property	Type	Default	Description
`apicurio.registry.usage-telemetry.client-id`	string	(empty)	Identifier for this client application. When set, the SerDes SDK sends `X-Registry-Client-Id` and `X-Registry-Operation` headers on schema fetch requests. When empty (default), no headers are sent and the client is not tracked. Must contain only letters, digits, dots, hyphens, and underscores.

Enabling usage telemetry

You can enable usage telemetry on the registry server and configure a client to report its schema fetches.

Prerequisites

Experimental features are enabled (apicurio.features.experimental.enabled=true), because usage telemetry is an experimental feature.

Procedure

Enable usage telemetry on the server by adding the following properties to its application.properties:
```
apicurio.features.experimental.enabled=true
apicurio.usage.telemetry.enabled=true
```
Configure the Kafka SerDes client with the registry URL and a client identifier:
```
apicurio.registry.url=http://registry:8080/apis/registry/v3
apicurio.registry.usage-telemetry.client-id=payments-svc
```
No additional endpoints to call and no buffers to configure are required.

All usage endpoints are under /admin/usage/ and require apicurio.usage.telemetry.enabled=true. Endpoints return HTTP 409 (Conflict) when telemetry is not enabled.

Get usage summary

Returns global Active, Stale, and Dead counts across all tracked schema versions.

GET /admin/usage/summary

{
  "active": 12,
  "stale": 3,
  "dead": 5
}

Get artifact usage metrics

Returns per-version usage metrics for a specific artifact, including fetch counts, client lists, and classification.

GET /admin/usage/artifacts/{groupId}/{artifactId}

{
  "groupId": "default",
  "artifactId": "UserEvent",
  "versions": [
    {
      "version": "3.0.0",
      "globalId": 42,
      "totalFetches": 1523,
      "uniqueClients": 3,
      "firstFetchedOn": 1710000000000,
      "lastFetchedOn": 1714400000000,
      "clients": ["payments-svc", "analytics", "fraud-detector"],
      "classification": "ACTIVE"
    }
  ]
}

Get consumer version heatmap

Returns a consumer version adoption grid showing which clients use which versions, with version drift detection.

GET /admin/usage/artifacts/{groupId}/{artifactId}/heatmap

{
  "groupId": "default",
  "artifactId": "UserEvent",
  "versions": ["1.0.0", "2.0.0", "3.0.0"],
  "consumers": [
    {
      "clientId": "payments-svc",
      "versions": {"3.0.0": 523},
      "versionsBehind": 0,
      "driftAlert": false
    },
    {
      "clientId": "analytics",
      "versions": {"1.0.0": 102},
      "versionsBehind": 2,
      "driftAlert": true
    }
  ]
}

Get deprecation readiness

Returns which consumers are actively using a specific version and whether it is safe to deprecate.

GET /admin/usage/artifacts/{groupId}/{artifactId}/versions/2024.Q2/deprecation-readiness

{
  "groupId": "default",
  "artifactId": "UserEvent",
  "version": "1.0.0",
  "activeConsumers": [
    {
      "clientId": "analytics",
      "lastFetched": 1714400000000,
      "fetchCount": 102
    }
  ],
  "safeToDeprecate": false
}

Transitioning a version through the deprecation workflow

Apicurio Registry supports a version lifecycle state machine for controlled deprecation.

  ENABLED ──────▶ DEPRECATED ──────▶ SUNSET ──────▶ (delete)
     ▲                │                  │
     │                ▼                  │
     └──────── DISABLED                  │
     ◀───────────────────────────────────┘

ENABLED: Version is active and available for use.
DEPRECATED: Version still works but consumers should migrate. A warning is logged on access.
SUNSET: Migration deadline has passed. The version will be removed soon. Requires transitioning through DEPRECATED first.
DISABLED: Version is blocked from use.

Prerequisites

No active consumers depend on the version, verified by using the deprecation readiness endpoint.

Procedure

Deprecate the version:

curl -X PUT http://localhost:8080/apis/registry/v3/groups/default/artifacts/UserEvent/versions/1.0.0/state \
  -H "Content-Type: application/json" \
  -d '{"state":"DEPRECATED"}'

Sunset the version:

curl -X PUT http://localhost:8080/apis/registry/v3/groups/default/artifacts/UserEvent/versions/1.0.0/state \
  -H "Content-Type: application/json" \
  -d '{"state":"SUNSET"}'

UI features

When usage telemetry is enabled, the registry UI displays the following elements.

Dashboard widget

Shows Active/Stale/Dead counts with colored labels (green/orange/red).

Version badges

Each version in the artifact versions table shows its usage classification badge.

Usage tab

The artifact detail page has a "Usage" tab with the following panels:

Consumer version heatmap table with color-coded fetch counts.
Version drift alerts for consumers using outdated versions.
Deprecation readiness panel (click a version column header to check).

SUNSET badge

Versions in the SUNSET state display a purple badge.

OpenTelemetry metrics export

When OpenTelemetry is enabled (quarkus.otel.sdk.disabled=false), usage classification counts are exported as metrics.

Table 3. OpenTelemetry metrics for usage classification
Metric	Description
`apicurio.usage.schemas.active`	Number of schema versions classified as Active.
`apicurio.usage.schemas.stale`	Number of schema versions classified as Stale.
`apicurio.usage.schemas.dead`	Number of schema versions classified as Dead.

These gauges are updated periodically (configured via apicurio.usage.aggregation.every). When OTel is disabled (the default), no overhead.

Enabling OTel export

You can export usage classification metrics by enabling OpenTelemetry.

Procedure

Set the following properties:

quarkus.otel.sdk.disabled=false
quarkus.otel.exporter.otlp.endpoint=http://otel-collector:4317
quarkus.otel.metrics.enabled=true

Example Grafana query

Use the following PromQL query to chart usage classification metrics in Grafana.

apicurio_usage_schemas_active or apicurio_usage_schemas_stale or apicurio_usage_schemas_dead

Privacy and data collection

Usage telemetry is designed to minimize data collection and give operators and clients control over what is recorded.

Opt-in only: Disabled by default on both server and client.
No message content: Only schema coordinates and the client identifier are recorded.
No user identity: The clientId is a configured application name, not a user or personal identifier.
Client-controlled: Each client application decides whether to set the X-Registry-Client-Id header.
Zero overhead when disabled: No headers are sent, no events are recorded, and no database writes occur.
Automatic cleanup: Events older than the retention period are deleted automatically.

Running the usage telemetry demo script

A demo script is provided to populate realistic usage data for testing.

Procedure

Start the registry with telemetry enabled:

./mvnw quarkus:dev -pl :apicurio-registry-app \
  -Dapicurio.features.experimental.enabled=true \
  -Dapicurio.usage.telemetry.enabled=true

In another terminal, run the demo script:

./examples/usage-telemetry-demo/populate-usage-data.sh

Results

The script creates three artifacts (UserEvent, OrderEvent, PaymentEvent) with multiple versions and simulates usage from five different client applications by fetching schemas with X-Registry-Client-Id headers.