--- title: Schema management in Stream Connect description: Import and create schemas to send and receive messages in an Apache Avro format. Using an Avro format can reduce the size of the payload and simplify your integration to your local Kafka instance. locale: en-US canonical_url: https://www.servicenow.com/docs/r/australia/integrate-applications/integration-hub/schema-management.html release: australia product: Integration Hub classification: integration-hub topic_type: concept last_updated: "2026-03-12" reading_time_minutes: 6 breadcrumb: [Using Stream Connect for Apache Kafka, Import and stream data, Integration Hub, Workflow Data Fabric] --- # Schema management in Stream Connect Import and create schemas to send and receive messages in an Apache Avro format. Using an Avro format can reduce the size of the payload and simplify your integration to your local Kafka instance. Avro is an open-source data-serialization system that uses schemas to structure encoded data. With an Avro schema, data can be converted from plain-text JSON to an Avro binary format and back. You can store schemas in ServiceNow, so your Stream Connect producers and consumers can use the schemas to serialize Avro messages. The following image shows an overview of schema management in Stream Connect. Schemas, stored in schema registries, enable messages in producers and consumers to be converted from plain text to an Avro format and back. \[Omitted image "stream-connect-schema-diagram.png"\] Alt text: Diagram showing how Stream Connect uses schemas stored in the schema registries to convert Kafka messages into different formats. ## Schemas You can import a schema from the Confluent registry or create your own standalone schema by uploading a JSON file or entering a schema directly as a JSON-formatted string. After your schema is imported or created, you can see it on the Stream Connect Schemas \[stream\_connect\_schema\] table, which stores both Confluent and standalone schemas. Additionally, Confluent schemas are visible on the Confluent Stream Connect Schema \[confluent\_stream\_connect\_schema\] table. Standalone schemas are on the Standalone Stream Connect Schema \[standalone\_stream\_connect\_schema\] table. All schemas have a schema ID, a globally unique identifier of the schema. For Confluent schemas, the schema ID is imported from the Confluent registry. For standalone schemas, the schema ID is generated locally and is unique on the instance. By default, the generated schema ID value is the next highest available schema ID on the instance. For example, if your schemas have ID numbers one through five, the next schema you create will have a schema ID of six. You can change the default value. Schema IDs are unique per registry. For example, two schemas can both have an ID of one as long as they're in different registries. ## Schema registries Every schema belongs to a registry. There are two types of schema registries in ServiceNow: the Confluent Schema Registry and the Standalone Schema Registry. Both schema registries have an option to **Track in the update set**. When this option is enabled, the schemas in that registry are saved to the update set. Saving the schemas to the update set makes it possible to move them from one environment to another. By default, this option is turned off for the Confluent Schema Registry because schema IDs may change from one environment to another. This option is enabled for the Standalone Schema Registry, because if you're creating schemas manually, the schema ID is less likely to change from one environment to another. To change the default setting for either registry, navigate to **All** > **IntegrationHub** > **Schema Registries**, select the registry, and change the **Track in the update set** option. When you create or update a Stream Producer record with the serialization format set to Avro, ServiceNow automatically generates and maintains an Avro schema for the associated table. If a schema already exists for the table when the Stream Producer record is saved, that schema is reused. If no schema exists, a new one is generated based on the table's field layout. Table metadata is used to detect whether a schema needs to be regenerated. A new schema is only generated when the table structure has changed. Schemas used by Stream Producer are cached in memory. If a message payload contains a field that the cached schema doesn't recognize, a new schema is generated and stored in the schema tables. The updated schema is then used to convert the change data capture \(CDC\) payload to Avro format before the message is sent to Kafka. Stream Producer schemas are stored across two tables in the ServiceNow IntegrationHub Stream Connect Schema \[`com.glide.hub.stream_connect.schema`\] plugin. These tables are read-only for all users. No one has create, update, or delete access.
Table Description
Table Message Schemas \[`sys_table_message_schema`\] Stores the Avro schema content for a table. Extends `stream_connect_schema`. Inherits all fields from the parent table, including the schema content field.
Table Message Schema Evolutions \[`sys_table_message_schema_evolution`\] Tracks schema versions for a table. Each record has the following.- A subject in the format `sn_*table\_name*` \(for example, `sn_incident`\). - The Table Message Schema, which is a reference to the corresponding `sys_table_message_schema` record. - The name of the ServiceNow table this schema is associated with. - The version number of the schema. The version increments each time a new schema is generated for the table.
For reference fields in the Avro payload, both the sys\_id and display value are included. |Role|Description| |----|-----------| |`message_schema_viewer`|Grants read, range, and report\_view access to the schema tables \(`sys_table_message_schema` and `sys_table_message_schema_evolution`\). No user has create, update, or delete access to these tables. If the Stream Connect Core \[com.glide.hub.stream\_connect.common.core\] plugin is active, this role is automatically included in the `stream_connect_viewer` role.| ## Wire-level message format For interoperability, ServiceNow uses a wire-level message format similar to the ones used by other systems. The first byte is set to 0. The next 4 bytes are used for the schema ID. The remaining bytes are used for the data, serialized in an Avro format.
Byte 0 Magic byte.
Byte 1–4 Schema ID.
Remaining bytes Data, serialized in an Avro format.
## Producers and consumers Stream Connect producers and consumers can be configured to use an Avro format. When configuring a producer, simply specify which schema you want to use. Then when you run the producer, the message payload is generated in JSON and automatically converted to an Avro format using the specified schema. For more information on producers, see the or the ProducerV2 API. You can also configure a Stream Producer to send messages in an Avro format. When the serialization format is set to Avro, the Stream Producer uses the auto-generated schema for the selected table to convert CDC payloads to Avro before sending them to Kafka. Configuring a consumer is similar. Specify the serialization format as **Encoded** and select a schema registry. When the consumer receives a message in an Avro format, it's automatically converted to JSON according to the schema for the schema ID received in the message. For more information, see the Kafka Message trigger or the [ETL](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/australia/integrate-applications/integration-hub/configure-etl-consumer.md), [Transform Map](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/australia/integrate-applications/integration-hub/configure-transform-map-consumer.md), or [Script](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/australia/integrate-applications/integration-hub/configure-script-consumer.md) consumers. The Schema Registry REST API exposes your Stream Producer Avro schemas to external systems and consumers. External applications can use the API to retrieve schemas and decode Avro-encoded messages received from Kafka topics. The API is available at the base URI `/api/now/schemaregistry`. It uses standard ServiceNow authentication. ACLs control access to the scripted REST API endpoints. ## Plugin Schema management features require the ServiceNow Stream Connect Installer \[com.glide.hub.stream\_connect.installer\] plugin. Stream Producer schema evolution features requires either the: - ServiceNow Integration Hub Stream Connect Schema \[com.glide.hub.stream\_connect.schema\] plugin, or the - ServiceNow Stream Connect Installer \(Direct Kafka\) \[com.glide.hub.stream\_connect.onprem\_installer\] plugin. If the Stream Connect Core \[com.glide.hub.stream\_connect.common.core\] plugin is installed, the `message_schema_viewer` role is automatically added to the `stream_connect_viewer` role. - **[Import a schema from the Confluent Registry](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/australia/integrate-applications/integration-hub/import-schema-confluent-registry.md)** Import a Schema from the Confluent Registry to enable your Stream Connect producers and consumers to send and receive Kafka messages in an Apache Avro format. - **[Create a standalone schema in Stream Connect](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/australia/integrate-applications/integration-hub/create-standalone-schema.md)** Create a schema to enable your Stream Connect producers and consumers to send and receive Kafka messages in an Apache Avro format. **Parent Topic:**[Using Stream Connect for Apache Kafka](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/australia/integrate-applications/integration-hub/stream-connect-apache-kafka.md)