Apicurio Studio

Migrating an existing Apicurio Registry deployment

This chapter explains how to migrate an existing Apicurio Registry deployment from version 1.3.x to version 2.x.

Migration from Apicurio Registry 1.3.x to 2.x

Apicurio Registry 2.x includes new features with breaking changes from the previous Apicurio Registry 1.3.x release. This section describes the major changes between Apicurio Registry 1.3.x and version 2.x.

Because of the breaking changes in 2.x, there is no automatic upgrade and a migration process is required. This involves moving the data in your existing registry to a new registry. You must also review your existing registry client applications and update their configuration to meet the new requirements.

When migrating to version 2.x, you must take the following major changes into account:

New data storage options

The existing registry data storage options in Apicurio Registry 1.3.x (streams,jpa, and infinispan) have been replaced with new storage options in version 2.x (sql and kafkasql). These new storage options provide more robust, performant, and maintainable Apicurio Registry deployments. The in-memory storage option (mem) is also still available, but is not suitable for production workloads.

For details on how to deploy Apicurio Registry 2.x with your chosen storage option, see Installing Apicurio Registry storage on OpenShift.

New v2 REST API

Apicurio Registry 2.x includes a new REST API with support for artifact groups and improved long term maintainability. Apicurio Registry still supports the original registry v1 REST API and compatibility APIs, for example, Confluent and IBM schema registry APIs. Apicurio Registry now also implements the Schema Registry specification provided in the CNCF Cloud Events specification.

Refactored Java client libraries

  • The Apicurio Registry Java client classes are available in version 2.x in a different Maven module named apicurio-registry-client.

  • The Kakfa client serializer and deserializer (SerDe) classes are available in version 2.x in three different Maven modules, one for each supported data format: Apache Avro, Protobuf, and JSON Schema. You can now use only the module you want without pulling in transitive dependencies that you are not concerned with.

Additional resources

Migrating Apicurio Registry data

Migrating data to Apicurio Registry 2.x requires exporting all data from your existing 1.3.x deployment and importing it into the new 2.x deployment. If you are using Apicurio Registry as a schema registry for Kafka applications, data migration is critical because each Kafka message carries the global identifier for the schema stored in Apicurio Registry. This identifier must be preserved during registry data migration.

Apicurio Registry 2.x provides an API to bulk import/export all data from your registry deployment, which guarantees that all identifiers are kept when importing data from your existing registry. The export API downloads a custom .zip file containing all the information for your artifacts. The import API accepts this .zip and loads all artifacts into the registry in a single batch.

Apicurio Registry 1.3.x does not provide an import/export API. However, version 2.x provides an export tool compatible with Apicurio Registry 1.3.x to export a .zip, which you can import into your 2.x registry. This tool uses common existing APIs to export all content in the registry. However, it is less performant than the 2.x export API, and should only be used when exporting from a 1.3.x registry.

Prerequisites

  • Running Apicurio Registry instances of the 1.3.x server you are exporting from and the 2.x server you are importing into.

  • Download the Apicurio Registry exportV1 tool from GitHub. This is a Java application that you can run on the command line.

Procedure

  1. Export all the data from Apicurio Registry 1.3.x using the exportV1 tool. This generates a registry-export.zip file in your current directory:

    java -jar apicurio-registry-utils-exportV1-2.3.0.Final.jar http://old-registry.my-company.com/api

  2. Import the .zip file into Apicurio Registry 2.x using the import API:

    curl -X POST "http://new-registry.my-company.com/apis/registry/v2/admin/import" \
  -H "Accept: application/json" -H "Content-Type: application/zip" \
  --data-binary @registry-export.zip

  3. Check that all the artifacts have been imported into the new 2.x registry by running these commands and comparing the count field:

    curl "http://old-registry.my-company.com/api/search/artifacts"
    curl "http://new-registry.my-company.com/apis/registry/v2/search/artifacts"
Additional resources

Migrating Apicurio Registry client applications

You must review your existing Apicurio Registry client applications to ensure that the Maven dependencies and Java client configuration meet the new requirements for version 2.x. For example, this includes new Maven dependencies for the Apicurio Registry Java REST client libraries or Kafka client serializer/deserializer (Serdes) libraries. You must also update your Java application configuration with the new registry v2 API path.

Prerequisites

  • Existing Apicurio Registry 1.3.x Java client application or Kafka client producer and consumer Java applications with SerDes

Procedure

  1. If you are using the Apicurio Registry Java REST client, you must change the Maven dependencies for the Apicurio Registry Java client libraries, which have been repackaged in version 2.x:

    <dependency>
    <groupId>io.apicurio</groupId>
    <artifactId>apicurio-registry-client</artifactId>
    <version>2.3.0.Final</version>
</dependency>

  2. In your Java client application, you must change your registry URL configuration, from pointing to the existing v1 API path to the new v2 path. For example:

    public class ClientExample {

    private static final RegistryRestClient client;

     public static void main(String[] args) throws Exception {
        // Create a registry client
        String registryUrl = "https://new-registry.my-company.com/apis/registry/v2";
        RegistryClient client = RegistryClientFactory.create(registryUrl);
    }
}

    You can find more details on the Java client in Managing Apicurio Registry content using a Java client.

  3. If you are using the Apicurio Registry SerDes libraries, you must change the Maven dependencies, which have been repackaged in version 2.x. In Apicurio Registry 1.3.x, the SerDes libraries were all provided with only one Maven dependency:

    <dependency>
    <groupId>io.apicurio</groupId>
    <artifactId>apicurio-registry-utils-serde</artifactId>
    <version>1.3.2.Final</version>
</dependency>

    In Apicurio Registry 2.x, the SerDes libraries have been split into three Maven dependencies, one for each supported data format: avro, protobuf, and json schema, depending on your use cases:

    <dependency>
    <groupId>io.apicurio</groupId>
    <artifactId>apicurio-registry-serdes-avro-serde</artifactId>
    <version>2.3.0.Final</version>
</dependency>
<dependency>
    <groupId>io.apicurio</groupId>
    <artifactId>apicurio-registry-serdes-protobuf-serde</artifactId>
    <version>2.3.0.Final</version>
</dependency>
<dependency>
    <groupId>io.apicurio</groupId>
    <artifactId>apicurio-registry-serdes-jsonschema-serde</artifactId>
    <version>2.3.0.Final</version>
</dependency>

  4. In your Kafka producer and consumer Java applications, you must change your registry URL configuration from pointing to the existing v1 API path to the new v2 path. For example:

    Existing registry v1 API path:

    props.putIfAbsent(AbstractKafkaSerDe.REGISTRY_URL_CONFIG_PARAM, "http://old-registry.my-company.com/api");

    New registry v2 API path:

    props.putIfAbsent(SerdeConfig.REGISTRY_URL, "http://new-registry.my-company.com/apis/registry/v2");

    The refactored SerDes libraries also include other important changes to configuration properties. For more details on SerDes configuration, see Validating schemas using Kafka client serializers/deserializers in Java clients.

Additional resources