Installing Apicurio Registry using Docker

This chapter explains how to install and run Apicurio Registry using Docker with the following storage options:

Installing Apicurio Registry with in-memory storage
Installing Apicurio Registry with SQL database storage
Installing Apicurio Registry with Apache Kafka storage

Prerequisites

Introduction to Apicurio Registry

You can install more than one replica of Apicurio Registry depending on your environment. The number of replicas depends on your storage option, for example, your Kafka or database cluster configuration, and on the number and type of artifacts stored in Apicurio Registry.

Additional resources

For details on building from source, see https://github.com/Apicurio/apicurio-registry.

Installing Apicurio Registry with in-memory storage

This section explains how to install and run Apicurio Registry with simple in-memory storage from a container image.

The in-memory storage option is suitable for development only. All data is lost when the container image is restarted.

Prerequisites

You must have Docker installed and the Docker daemon running.
Ensure that you can connect to the container repository: https://hub.docker.com/r/apicurio/apicurio-registry.

Procedure

Get the Apicurio Registry container image:
```
$ docker pull apicurio/apicurio-registry:VERSION
```
VERSION is the Apicurio Registry release version, for example, latest-release. For more details, see https://hub.docker.com/r/apicurio/apicurio-registry/tags.

Run the container image:

$ docker run -it -p 8080:8080 apicurio/apicurio-registry:VERSION

All storage variants of Apicurio Registry 3.x now share the same container image of apicurio/apicurio-registry. This is in contrast to the separate container images used by Apicurio Registry 2.x. Choosing a storage variant is now a matter of configuring the single backend/server container image.

Send a test request using the Apicurio Registry REST API. For example, enter the following curl command to create a simple Avro schema artifact for a share price application in the registry:

$ curl -X POST -H "Content-type: application/json; artifactType=AVRO" -H "X-Registry-ArtifactId: share-price" --data '{"type":"record","name":"price","namespace":"com.example","fields":[{"name":"symbol","type":"string"},{"name":"price","type":"string"}]}' http://localhost:8080/apis/registry/v3/groups/my-group/artifacts

Verify that the response includes the expected JSON body to confirm that the Avro schema artifact was created in the registry. For example:

{"name":"price","createdBy":"","createdOn":"2021-03-19T17:48:17+0000","modifiedOn":"2021-03-19T17:48:17+0000","id":"share-price","version":1,"type":"AVRO","globalId":12,"state":"ENABLED","groupId":"my-group","contentId":12}

Additional resources

For more REST API sample requests, see the Apicurio Registry REST API documentation.
For demonstration examples, see https://github.com/Apicurio/apicurio-registry/tree/main/examples.

Installing Apicurio Registry with SQL database storage

This section explains how to install and run Apicurio Registry with storage in a PostgreSQL database from a container image. This storage option is suitable for production environments.

Prerequisites

You must have Docker installed and the Docker daemon running.
Ensure that you can connect to the container repository: https://hub.docker.com/r/apicurio/apicurio-registry.
You must already have a PostgreSQL database server installed and running. For example:
```
$ postgres -D /usr/local/pgsql/data
```
For more details, see https://www.postgresql.org/docs/12/server-start.html

Procedure

Get the Apicurio Registry container image:
```
$ docker pull apicurio/apicurio-registry:VERSION
```
VERSION is the Apicurio Registry release version, for example, latest-release. For more details, see https://hub.docker.com/r/apicurio/apicurio-registry/tags.
Run the container image and specify the following environment variable for your Postgresql system:
- APICURIO_STORAGE_KIND - The storage variant. Set this to sql.
- APICURIO_STORAGE_SQL_KIND - The type of SQL database to use. Set this to postgresql
- APICURIO_DATASOURCE_URL - The database connection URL.
- APICURIO_DATASOURCE_USERNAME, APICURIO_DATASOURCE_PASSWORD - The credentials for the PostgreSQL server.
  $ docker run -it -p 8080:8080 \ -e "APICURIO_STORAGE_KIND=sql" \ -e "APICURIO_STORAGE_SQL_KIND=postgresql" \ -e "APICURIO_DATASOURCE_URL=jdbc:postgresql://postgres/apicurio-registry" \ -e "APICURIO_DATASOURCE_USERNAME=apicurio-registry" \ -e "APICURIO_DATASOURCE_PASSWORD=password" \ apicurio/apicurio-registry:VERSION

Send a test request using the Apicurio Registry REST API. For example, enter the following curl command to create a simple Avro schema artifact for a share price application in the registry:

$ curl -X POST -H "Content-type: application/json" --data '{"artifactId":"share-price","artifactType":"AVRO","name":"Share Price","labels":{"environment":"DEV","category":"finance"},"firstVersion":{"version":"1.0.0","content":{"content":"{\"type\":\"record\",\"name\":\"price\",\"namespace\":\"com.example\",\"fields\":[{\"name\":\"symbol\",\"type\":\"string\"},{\"name\":\"price\",\"type\":\"string\"}]}","contentType":"application/json"}}}' http://localhost:8080/apis/registry/v3/groups/my-group/artifacts

Verify that the response includes the expected JSON body to confirm that the Avro schema artifact was created in the registry. For example:

{"artifact":{"name":"Share Price","owner":"","createdOn":"2024-09-26T16:26:03Z","modifiedBy":"","modifiedOn":"2024-09-26T16:26:03Z","artifactType":"AVRO","labels":{"environment":"DEV","category":"finance"},"groupId":"my-group","artifactId":"share-price"},"version":{"version":"1.0.0","owner":"","createdOn":"2024-09-26T16:26:03Z","artifactType":"AVRO","globalId":1,"state":"ENABLED","groupId":"my-group","contentId":1,"artifactId":"share-price"}}

Additional resources

For more REST API sample requests, see the Apicurio Registry REST API documentation.
For demonstration examples, see https://github.com/Apicurio/apicurio-registry/tree/main/examples.
For more details on PostgreSQL, see the PostgreSQL documentation.

Using file-based secrets for database credentials

This section explains how to use file-based secrets for database credentials with Apicurio Registry when using Docker Compose. This approach follows Docker’s recommended secret management practices and is more secure than passing credentials as environment variables.

Prerequisites

You must have Docker and Docker Compose installed.
You must already have a database server installed and running (PostgreSQL, MySQL, or Microsoft SQL Server).

Overview

Instead of passing credentials and secrets as environment variables (which can be exposed in process lists and container inspection), you can use file-based secrets. Apicurio Registry supports a generic _FILE suffix pattern that works with any configuration property.

Environment variables are automatically converted to configuration properties by Quarkus. For example, APICURIO_DATASOURCE_PASSWORD_FILE becomes the property apicurio.datasource.password.file. The FileBasedSecretsConfigSource reads the file path from these properties and loads the actual values from the files.

How it works

To use file-based secrets for any configuration property:

Take any environment variable name (e.g., APICURIO_DATASOURCE_PASSWORD)
Add _FILE suffix (e.g., APICURIO_DATASOURCE_PASSWORD_FILE)
Set the value to a file path containing the secret
Apicurio Registry reads the file contents and uses them as the actual property value

This works for any configuration property, not just the examples listed below.

Common file-based credential variables

Some commonly used file-based credential environment variables (and their corresponding properties):

Database credentials:

APICURIO_DATASOURCE_USERNAME_FILE (property: apicurio.datasource.username.file)
APICURIO_DATASOURCE_PASSWORD_FILE (property: apicurio.datasource.password.file)
APICURIO_DATASOURCE_BLUE_USERNAME_FILE (property: apicurio.datasource.blue.username.file)
APICURIO_DATASOURCE_BLUE_PASSWORD_FILE (property: apicurio.datasource.blue.password.file)
APICURIO_DATASOURCE_GREEN_USERNAME_FILE (property: apicurio.datasource.green.username.file)
APICURIO_DATASOURCE_GREEN_PASSWORD_FILE (property: apicurio.datasource.green.password.file)

OAuth/OIDC credentials:

QUARKUS_OIDC_CLIENT_SECRET_FILE (property: quarkus.oidc.client-secret.file)
APICURIO_UI_AUTH_OIDC_CLIENT_SECRET_FILE (property: apicurio.ui.auth.oidc.client-secret.file)

Kafka SASL credentials:

APICURIO_KAFKASQL_SECURITY_SASL_CLIENT_ID_FILE (property: apicurio.kafkasql.security.sasl.client-id.file)
APICURIO_KAFKASQL_SECURITY_SASL_CLIENT_SECRET_FILE (property: apicurio.kafkasql.security.sasl.client-secret.file)

Kafka SSL/TLS passwords:

APICURIO_KAFKASQL_SECURITY_SSL_TRUSTSTORE_PASSWORD_FILE (property: apicurio.kafkasql.security.ssl.truststore.password.file)
APICURIO_KAFKASQL_SECURITY_SSL_KEYSTORE_PASSWORD_FILE (property: apicurio.kafkasql.security.ssl.keystore.password.file)
APICURIO_KAFKASQL_SECURITY_SSL_KEY_PASSWORD_FILE (property: apicurio.kafkasql.security.ssl.key.password.file)

Any other property:

The _FILE suffix works with any configuration property. Simply add _FILE to any environment variable name and set the value to a file path.

When a _FILE variant is set, Apicurio Registry reads the value from the specified file instead of using the direct environment variable. Configuration precedence: System properties (400) > File-based secrets (350) > Environment variables (300) > application.properties (250).

Procedure

Create a directory for your secret files:
```
$ mkdir -p secrets
```

Create files containing your database credentials:

$ echo "pguser" > secrets/db_user.txt
$ echo "pgpass" > secrets/db_password.txt

Set restrictive permissions on the secret files:
```
$ chmod 600 secrets/*.txt
```

Create a docker-compose.yml file that uses Docker Compose secrets:

version: '3.8'

secrets:
  registry_db_user:
    file: ./secrets/db_user.txt
  registry_db_password:
    file: ./secrets/db_password.txt

services:
  apicurio-registry:
    image: apicurio/apicurio-registry:VERSION
    environment:
      APICURIO_STORAGE_KIND: "sql"
      APICURIO_STORAGE_SQL_KIND: "postgresql"
      APICURIO_DATASOURCE_URL: "jdbc:postgresql://postgres:5432/registrydb"
      APICURIO_DATASOURCE_USERNAME_FILE: /run/secrets/registry_db_user
      APICURIO_DATASOURCE_PASSWORD_FILE: /run/secrets/registry_db_password
    secrets:
      - registry_db_user
      - registry_db_password
    ports:
      - "8080:8080"

Start the services:
```
$ docker compose up -d
```

File format

Secret files should contain only the credential value in plain text (UTF-8 encoding). Leading and trailing whitespace is automatically trimmed. For example:

mypassword

Security considerations

Ensure secret files have restrictive permissions (e.g., chmod 600)
Never commit secret files to version control (add them to .gitignore)
Apicurio Registry will log a warning if secret files are world-readable
If a _FILE variable is set but the file doesn’t exist or isn’t readable, Apicurio Registry will fail to start with a clear error message

Additional resources

For a complete Docker Compose example with secrets, see PostgreSQL with file-based secrets example.
For more details on Docker Compose secrets, see Docker Compose secrets documentation.

Installing Apicurio Registry with Apache Kafka storage

This topic explains how to install and run Apicurio Registry with Kafka storage from a container image. The kafkasql storage option uses a Kafka topic for storage, along with an in-memory H2 database. This storage option is suitable for production environments.

Prerequisites

You must have Docker installed and the Docker daemon running.
Ensure that you can connect to the container repository: https://hub.docker.com/r/apicurio/apicurio-registry.
Ensure that KAFKA_HOME is set to the location of your Kafka-based system.

You must already have your Kafka broker running. For example:

$KAFKA_HOME/bin/zookeeper-server-start.sh $KAFKA_HOME/config/zookeeper.properties &
$KAFKA_HOME/bin/kafka-server-start.sh $KAFKA_HOME/config/server.properties &

For more details, see https://kafka.apache.org/quickstart.

Procedure

Get the Apicurio Registry container image:
```
$ docker pull apicurio/apicurio-registry:VERSION
```
VERSION is the Apicurio Registry release version, for example, latest-release. For more details, see https://hub.docker.com/r/apicurio/apicurio-registry/tags.
Run the container image and specify the following environment variable for your Kafka system:
- APICURIO_STORAGE_KIND - Set this to kafkasql
- APICURIO_KAFKASQL_BOOTSTRAP_SERVERS - The address(es) of your Kafka broker(s).
  
  For example:
  $ docker run -it -p 8080:8080 -e "APICURIO_STORAGE_KIND=kafkasql" -e "APICURIO_KAFKASQL_BOOTSTRAP_SERVERS=kafka:9092" apicurio/apicurio-registry:latest

Send a test request using the Apicurio Registry REST API. For example, enter the following curl command to create a simple Avro schema artifact for a share price application in the registry:

$ curl -X POST -H "Content-type: application/json" --data '{"artifactId":"share-price","artifactType":"AVRO","name":"Share Price","labels":{"environment":"DEV","category":"finance"},"firstVersion":{"version":"1.0.0","content":{"content":"{\"type\":\"record\",\"name\":\"price\",\"namespace\":\"com.example\",\"fields\":[{\"name\":\"symbol\",\"type\":\"string\"},{\"name\":\"price\",\"type\":\"string\"}]}","contentType":"application/json"}}}' http://localhost:8080/apis/registry/v3/groups/my-group/artifacts

Verify that the response includes the expected JSON body to confirm that the Avro schema artifact was created in the registry. For example:

{"artifact":{"name":"Share Price","owner":"","createdOn":"2024-09-26T16:26:03Z","modifiedBy":"","modifiedOn":"2024-09-26T16:26:03Z","artifactType":"AVRO","labels":{"environment":"DEV","category":"finance"},"groupId":"my-group","artifactId":"share-price"},"version":{"version":"1.0.0","owner":"","createdOn":"2024-09-26T16:26:03Z","artifactType":"AVRO","globalId":1,"state":"ENABLED","groupId":"my-group","contentId":1,"artifactId":"share-price"}}

Additional resources

For more REST API sample requests, see the Apicurio Registry REST API documentation.
For demonstration examples, see https://github.com/Apicurio/apicurio-registry/tree/main/examples.
For more details on Apache Kafka, see https://kafka.apache.org/documentation/.