Setup Guide
Follow these steps to connect Apache Kafka to your Connect AI account:Enter the Bootstrap Servers that the provider uses to establish the initial connection to the Apache Kafka cluster.
(Optional) Toggle on Use SSL to have the provider negotiate SSL/TLS when connecting to the Apache Kafka broker.
Type Detection Scheme
- None
- RowScan
- SchemaRegistry
- SchemaRegistryAggregate
- MessageOnly
Enter the Registry URL, which is the endpoint of the schema registry. When this property is specified, the provider supports reading Avro and JSON schemas from the server.
Select the Registry Service that the provider uses to retrieve key and value schemas for Apache Kafka topics. Options are Confluent and AWSGlue.
Select the Registry Auth Scheme that the provider uses when connecting to the schema registry. Options are None, Basic, and SSLCertificate. If you select Basic, also enter the Registry User and Registry Password used to authenticate to the schema registry.
(Optional) Select the Schema Merge Mode, which specifies how the provider exposes schemas with multiple versions. Options are None and Simple.
(Optional) Select the Non Registry Type Detection Scheme, which specifies the type detection scheme to use for topics that do not have schemas in the schema registry. Options are Disabled, None, RowScan, and MessageOnly.
(Optional) Select the Non Registry Type Detection Scheme, which specifies the type detection scheme to use for topics that do not have schemas in the schema registry. Options are Disabled, None, RowScan, and MessageOnly.
Select the Auth Scheme that the provider uses when connecting to the Apache Kafka broker, then proceed to the Authentication section.
Authentication
- None
- Plain
- SCRAM
- SCRAM-SHA-512
- Kerberos
- SSLCertificate
- KafkaOAuthClient
- AzureAD
- AzureServicePrincipal
- AzureServicePrincipalCert
- OAuthJWT
- AWSWorkloadIdentity
- AzureWorkloadIdentity
At the top of the Connect AI Add Connection page, click Save & Test.
- If the connection test succeeds, a Connection successfully saved message appears, indicating that your connection has been created. The Status on the Edit Connection page also changes to Authenticated. View the data model of your successful connection in the right pane of the Edit Connection page, in the Data Model tab.
- If the connection test fails, ensure that you entered your login information correctly with no stray spaces or other characters. Connect AI displays error messages under the required fields with missing data. Some data sources require that you sign in directly to the source website. If you did not, an error message appears under the Sign in button. Correct the errors and try again.
- Unsuccessful connections are saved as drafts and have a Status of Not Authenticated. You can return to the connection and authenticate it later.
At the top of the Connect AI Add Connection page, click Save & Test.
- If the connection test succeeds, a Connection successfully saved message appears, indicating that your connection has been created. The Status on the Edit Connection page also changes to Authenticated. View the data model of your successful connection in the right pane of the Edit Connection page, in the Data Model tab.
- If the connection test fails, ensure that you entered your login information correctly with no stray spaces or other characters. Connect AI displays error messages under the required fields with missing data. Some data sources require that you sign in directly to the source website. If you did not, an error message appears under the Sign in button. Correct the errors and try again.
- Unsuccessful connections are saved as drafts and have a Status of Not Authenticated. You can return to the connection and authenticate it later.
At the top of the Connect AI Add Connection page, click Save & Test.
- If the connection test succeeds, a Connection successfully saved message appears, indicating that your connection has been created. The Status on the Edit Connection page also changes to Authenticated. View the data model of your successful connection in the right pane of the Edit Connection page, in the Data Model tab.
- If the connection test fails, ensure that you entered your login information correctly with no stray spaces or other characters. Connect AI displays error messages under the required fields with missing data. Some data sources require that you sign in directly to the source website. If you did not, an error message appears under the Sign in button. Correct the errors and try again.
- Unsuccessful connections are saved as drafts and have a Status of Not Authenticated. You can return to the connection and authenticate it later.
At the top of the Connect AI Add Connection page, click Save & Test.
- If the connection test succeeds, a Connection successfully saved message appears, indicating that your connection has been created. The Status on the Edit Connection page also changes to Authenticated. View the data model of your successful connection in the right pane of the Edit Connection page, in the Data Model tab.
- If the connection test fails, ensure that you entered your login information correctly with no stray spaces or other characters. Connect AI displays error messages under the required fields with missing data. Some data sources require that you sign in directly to the source website. If you did not, an error message appears under the Sign in button. Correct the errors and try again.
- Unsuccessful connections are saved as drafts and have a Status of Not Authenticated. You can return to the connection and authenticate it later.
Enter the Kerberos SPN, which is the full Kerberos service principal name (SPN) of the Apache Kafka broker.
(Optional) Enter the Kerberos Keytab File, which is the path to the keytab file that contains the Kerberos principals and encrypted keys used for authentication.
(Optional) Toggle on Use Kerberos Ticket Cache to use the Kerberos ticket cache for authentication instead of a keytab file.
At the top of the Connect AI Add Connection page, click Save & Test.
- If the connection test succeeds, a Connection successfully saved message appears, indicating that your connection has been created. The Status on the Edit Connection page also changes to Authenticated. View the data model of your successful connection in the right pane of the Edit Connection page, in the Data Model tab.
- If the connection test fails, ensure that you entered your login information correctly with no stray spaces or other characters. Connect AI displays error messages under the required fields with missing data. Some data sources require that you sign in directly to the source website. If you did not, an error message appears under the Sign in button. Correct the errors and try again.
- Unsuccessful connections are saved as drafts and have a Status of Not Authenticated. You can return to the connection and authenticate it later.
Select the SSL Client Cert Type, which specifies the format of the SSL client certificate. Options are JKSFILE, PFXFILE, PEMKEY_FILE, and PEMKEY_BLOB.
(Optional) Enter the SSL Client Cert Password used to decrypt the certificate provided in SSL Client Cert.
At the top of the Connect AI Add Connection page, click Save & Test.
- If the connection test succeeds, a Connection successfully saved message appears, indicating that your connection has been created. The Status on the Edit Connection page also changes to Authenticated. View the data model of your successful connection in the right pane of the Edit Connection page, in the Data Model tab.
- If the connection test fails, ensure that you entered your login information correctly with no stray spaces or other characters. Connect AI displays error messages under the required fields with missing data. Some data sources require that you sign in directly to the source website. If you did not, an error message appears under the Sign in button. Correct the errors and try again.
- Unsuccessful connections are saved as drafts and have a Status of Not Authenticated. You can return to the connection and authenticate it later.
Enter the OAuth Client Id (also known as the consumer key) assigned to your custom OAuth application. This ID is required to identify the application to the OAuth authorization server during authentication.
Enter the OAuth Client Secret assigned to your custom OAuth application. This confidential value is used to authenticate the application to the OAuth authorization server.
At the top of the Connect AI Add Connection page, click Save & Test.
- If the connection test succeeds, a Connection successfully saved message appears, indicating that your connection has been created. The Status on the Edit Connection page also changes to Authenticated. View the data model of your successful connection in the right pane of the Edit Connection page, in the Data Model tab.
- If the connection test fails, ensure that you entered your login information correctly with no stray spaces or other characters. Connect AI displays error messages under the required fields with missing data. Some data sources require that you sign in directly to the source website. If you did not, an error message appears under the Sign in button. Correct the errors and try again.
- Unsuccessful connections are saved as drafts and have a Status of Not Authenticated. You can return to the connection and authenticate it later.
Enter the Azure Tenant, which identifies the Apache Kafka tenant being used to access data. This field accepts either the tenant’s domain name (for example, contoso.onmicrosoft.com) or its directory (tenant) ID.
Enter the OAuth Client Id (also known as the consumer key) assigned to your custom OAuth application. This ID is required to identify the application to the OAuth authorization server during authentication.
Enter the OAuth Client Secret assigned to your custom OAuth application. This confidential value is used to authenticate the application to the OAuth authorization server.
At the top of the Connect AI Add Connection page, click Save & Test.
- If the connection test succeeds, a Connection successfully saved message appears, indicating that your connection has been created. The Status on the Edit Connection page also changes to Authenticated. View the data model of your successful connection in the right pane of the Edit Connection page, in the Data Model tab.
- If the connection test fails, ensure that you entered your login information correctly with no stray spaces or other characters. Connect AI displays error messages under the required fields with missing data. Some data sources require that you sign in directly to the source website. If you did not, an error message appears under the Sign in button. Correct the errors and try again.
- Unsuccessful connections are saved as drafts and have a Status of Not Authenticated. You can return to the connection and authenticate it later.
Enter the Azure Tenant, which identifies the Apache Kafka tenant being used to access data. This field accepts either the tenant’s domain name (for example, contoso.onmicrosoft.com) or its directory (tenant) ID.
Enter the OAuth Client Id (also known as the consumer key) assigned to your custom OAuth application. This ID is required to identify the application to the OAuth authorization server during authentication.
Enter the OAuth Client Secret assigned to your custom OAuth application. This confidential value is used to authenticate the application to the OAuth authorization server.
At the top of the Connect AI Add Connection page, click Save & Test.
- If the connection test succeeds, a Connection successfully saved message appears, indicating that your connection has been created. The Status on the Edit Connection page also changes to Authenticated. View the data model of your successful connection in the right pane of the Edit Connection page, in the Data Model tab.
- If the connection test fails, ensure that you entered your login information correctly with no stray spaces or other characters. Connect AI displays error messages under the required fields with missing data. Some data sources require that you sign in directly to the source website. If you did not, an error message appears under the Sign in button. Correct the errors and try again.
- Unsuccessful connections are saved as drafts and have a Status of Not Authenticated. You can return to the connection and authenticate it later.
Enter the Azure Tenant, which identifies the Apache Kafka tenant being used to access data. This field accepts either the tenant’s domain name (for example, contoso.onmicrosoft.com) or its directory (tenant) ID.
Enter the OAuth Client Id (also known as the consumer key) assigned to your custom OAuth application. This ID is required to identify the application to the OAuth authorization server during authentication.
Enter the OAuth Client Secret assigned to your custom OAuth application. This confidential value is used to authenticate the application to the OAuth authorization server.
Select the OAuth JWT Cert Type, which identifies the type of key store containing the JWT certificate. Options are PFXBLOB, JKSBLOB, PEMKEY_BLOB, PUBLIC_KEY_BLOB, SSHPUBLIC_KEY_BLOB, XMLBLOB, BCFKSBLOB, and GOOGLEJSONBLOB.
(Optional) Enter the OAuth JWT Cert Password. This is the password for the OAuth JWT certificate used to access a password-protected certificate store. If the certificate store does not require a password, leave this property blank.
(Optional) Enter the OAuth JWT Cert Subject of the OAuth JWT certificate used to locate a matching certificate in the store. Supports partial matches and the wildcard ’*’ to select the first certificate.
At the top of the Connect AI Add Connection page, click Save & Test.
- If the connection test succeeds, a Connection successfully saved message appears, indicating that your connection has been created. The Status on the Edit Connection page also changes to Authenticated. View the data model of your successful connection in the right pane of the Edit Connection page, in the Data Model tab.
- If the connection test fails, ensure that you entered your login information correctly with no stray spaces or other characters. Connect AI displays error messages under the required fields with missing data. Some data sources require that you sign in directly to the source website. If you did not, an error message appears under the Sign in button. Correct the errors and try again.
- Unsuccessful connections are saved as drafts and have a Status of Not Authenticated. You can return to the connection and authenticate it later.
Select the OAuth JWT Cert Type, which identifies the type of key store containing the JWT certificate. Options are PFXBLOB, JKSBLOB, PEMKEY_BLOB, PUBLIC_KEY_BLOB, SSHPUBLIC_KEY_BLOB, XMLBLOB, BCFKSBLOB, and GOOGLEJSONBLOB.
(Optional) Enter the OAuth JWT Cert Password. This is the password for the OAuth JWT certificate used to access a password-protected certificate store. If the certificate store does not require a password, leave this property blank.
(Optional) Enter the OAuth JWT Cert Subject of the OAuth JWT certificate used to locate a matching certificate in the store. Supports partial matches and the wildcard ’*’ to select the first certificate.
At the top of the Connect AI Add Connection page, click Save & Test.
- If the connection test succeeds, a Connection successfully saved message appears, indicating that your connection has been created. The Status on the Edit Connection page also changes to Authenticated. View the data model of your successful connection in the right pane of the Edit Connection page, in the Data Model tab.
- If the connection test fails, ensure that you entered your login information correctly with no stray spaces or other characters. Connect AI displays error messages under the required fields with missing data. Some data sources require that you sign in directly to the source website. If you did not, an error message appears under the Sign in button. Correct the errors and try again.
- Unsuccessful connections are saved as drafts and have a Status of Not Authenticated. You can return to the connection and authenticate it later.
Enter the AWS Workload Identity Config, which are the configuration properties to provide when using Workload Identity Federation via AWS.
Enter the Workload Project Id, which is the ID of the Google Cloud project that hosts your Workload Identity Federation pool.
Enter the Workload Provider Id, which is the ID of your Workload Identity Federation pool provider.
At the top of the Connect AI Add Connection page, click Save & Test.
- If the connection test succeeds, a Connection successfully saved message appears, indicating that your connection has been created. The Status on the Edit Connection page also changes to Authenticated. View the data model of your successful connection in the right pane of the Edit Connection page, in the Data Model tab.
- If the connection test fails, ensure that you entered your login information correctly with no stray spaces or other characters. Connect AI displays error messages under the required fields with missing data. Some data sources require that you sign in directly to the source website. If you did not, an error message appears under the Sign in button. Correct the errors and try again.
- Unsuccessful connections are saved as drafts and have a Status of Not Authenticated. You can return to the connection and authenticate it later.
Enter the Azure Workload Identity Config, which are the configuration properties to provide when using Workload Identity Federation via Azure.
Enter the Workload Project Id, which is the ID of the Google Cloud project that hosts your Workload Identity Federation pool.
Enter the Workload Provider Id, which is the ID of your Workload Identity Federation pool provider.
At the top of the Connect AI Add Connection page, click Save & Test.
- If the connection test succeeds, a Connection successfully saved message appears, indicating that your connection has been created. The Status on the Edit Connection page also changes to Authenticated. View the data model of your successful connection in the right pane of the Edit Connection page, in the Data Model tab.
- If the connection test fails, ensure that you entered your login information correctly with no stray spaces or other characters. Connect AI displays error messages under the required fields with missing data. Some data sources require that you sign in directly to the source website. If you did not, an error message appears under the Sign in button. Correct the errors and try again.
- Unsuccessful connections are saved as drafts and have a Status of Not Authenticated. You can return to the connection and authenticate it later.