Source Configuration Settings

org.neo4j.connectors.kafka.source.Neo4jConnector

A compatible Kafka Connect converter.

Could be one of;

Schema Registry URL for message keys. Required when a schema registry is required by the configured key.converter.

Should be set to true when key.converter is io.confluent.connect.protobuf.ProtobufConverter.

A compatible Kafka Connect converter.

Could be one of;

Schema Registry URL for message values. Required when a schema registry is required by the configured value.converter.

Should be set to true when value.converter is io.confluent.connect.protobuf.ProtobufConverter.

Neo4j URI to connect to. Multiple URIs can be specified separated by ,.

Neo4j database name to connect to. Recommended to be specified explicitly.

Authentication type to use. One of NONE, BASIC, KERBEROS, BEARER, CUSTOM

Default: BASIC

Username to authenticate with. Required when neo4j.authentication.type is BASIC.

Password to authenticate with. Required when neo4j.authentication.type is BASIC.

Authentication realm to authenticate with, leave empty for default.

Kerberos ticket to establish connection with. Required when neo4j.authentication.type is KERBEROS.

Bearer token to establish connection with. Required when neo4j.authentication.type is BEARER.

Custom authentication scheme to establish connection with. Required when neo4j.authentication.type is CUSTOM.

Custom principal to establish connection with. Required when neo4j.authentication.type is CUSTOM.

Custom credential to establish connection with. Required when neo4j.authentication.type is CUSTOM.

Custom authentication realm to authenticate with, set as required by your custom authentication provider.

TCP connection timeout (valid units are: ms, s, m, h and d; default unit is s).

Default: 30s (Driver default)

Maximum number of connections to keep in the connection pool.

Default: 100 (Driver default)

Maximum duration to wait for acquiring a connection from the connection pool (valid units are: ms, s, m, h and d; default unit is s).

Default: 60s (Driver default)

Duration after which idle connections are tested for liveness (valid units are: ms, s, m, h and d; default unit is s).

Default: no test (Driver default)

Duration after which a connection is dropped from the connection pool (valid units are: `ms, s, m, h and d; default unit is s).

Default: 1h (Driver default)

Maximum duration to retry a transaction.

Default: 30s

Whether encryption is enabled. Only applicable when bolt or neo4j schemes are used inside neo4j.uri. One of true, false

Default: false.

Trust strategy to use for TLS connections. One of TRUST_ALL_CERTIFICATES, TRUST_SYSTEM_CA_SIGNED_CERTIFICATES, TRUST_CUSTOM_CA_SIGNED_CERTIFICATES Required when neo4j.security.encrypted is true.

Default: TRUST_SYSTEM_CA_SIGNED_CERTIFICATES

List of files that contain X509 certificates of CAs to trust. Required when neo4j.security.trust-strategy is TRUST_CUSTOM_CA_SIGNED_CERTIFICATES.

Whether hostname verification is enabled during TLS handshake. One of true, false.

Default: true

Source strategy for this connector. One of CDC, QUERY.

A time anchor to start streaming from. One of EARLIEST, NOW, USER_PROVIDED. Only used on initial run of the connector instance, and ignored when there is already a stored offset in Kafka Connect.

Default: NOW

Custom value to use as a starting offset. Required when neo4j.start-from is set to USER_PROVIDED.

Whether to ignore any offset value retrieved from the offset storage saved by a previous run. One of true, false.

Default: false

Maximum number of change events to publish for each poll cycle.

Default: 1000

Defines the structure of change messages. One of COMPACT, EXTENDED, RAW_JSON_STRING. COMPACT provides simpler messages but faces schema compatibility issues if property types change. EXTENDED includes type information to avoid such issues. RAW_JSON_STRING produces a raw JSON string representation of the data.

Default: EXTENDED. Introduced in 5.1.5

Serialisation strategy for CDC topic key. One of SKIP, ELEMENT_ID, ENTITY_KEYS, WHOLE_VALUE.

Default: WHOLE_VALUE

For guidance on choosing a strategy, see Best practices for Source connector.

Serialisation strategy for CDC topic value. One of CHANGE_EVENT, ENTITY_EVENT.

Default: CHANGE_EVENT

Note: If you’re using a sink connector with one of the CDC strategies, this setting must be configured as CHANGE_EVENT.

Maximum amount of time Kafka Connect poll request will wait for a change to be received from the database (valid units are: ms, s, m, h and d; default unit is s)

Default: 5s

The interval in which the database will be polled for changes during neo4j.cdc.poll-duration (valid units are: ms, s, m, h and d; default unit is s).

Default: 1s

Comma-separated list of graph patterns, with optional event property filters (prefixed with + or - for inclusion or exclusion respectively).

Example setting: "neo4j.cdc.topic.my-topic.patterns": "(:Person {+name}),(:Person)-[:WORKS_FOR]→(:Company),(:Company {-id})". The change event for Company nodes will not include the id property.

For guidance on choosing between this form and the indexed form below, see Choosing between the list and indexed forms.

Indexed graph pattern, with optional event property filters (prefixed with + or - for inclusion or exclusion respectively).

Example: "neo4j.cdc.topic.my-topic.patterns.0.pattern": "(:Company {+name})". The change event for Company nodes will either always include the name property (in CDC FULL enrichment mode) or when the name property has changed (in CDC DIFF enrichment mode).

One of CREATE, UPDATE, DELETE. Operation for which we want to receive events for the corresponding indexed graph pattern.

Example: "neo4j.cdc.topic.my-topic.patterns.0.operation": "DELETE"

Comma-separated list of property names.

Example: "neo4j.cdc.topic.my-topic.patterns.0.changesTo": "name, age" (see how selectors are combined)

User who was authenticated while performing the change.

Example: "neo4j.cdc.topic.my-topic.patterns.0.metadata.authenticatedUser": "userA"

User who was executing the transaction while performing the change. Usually will be same with authenticatedUser, but might change if user impersonation is used.

Example: "neo4j.cdc.topic.my-topic.patterns.0.metadata.executingUser": "userB"

Key-value transaction metadata selector. The key-value pair is matched against the actual transaction metadata (with the key stripped of the aforementioned prefix).

Example: "neo4j.cdc.topic.my-topic.patterns.0.metadata.txMetadata.app": "neo4j-browser"

Whether to enable the last_db_tx_id metric.

Default: false Introduced in 5.3.0

The refresh interval of the last_db_tx_id metric (valid units are: ms, s, m, h and d; default unit is s).

Default: 30s Introduced in 5.3.0

Whether to always run CDC queries against the leader instance of the database, instead of any cluster member.

For guidance on choosing between the two, see Decide whether to pin CDC queries to the leader.

Default: false Introduced in 5.3.0

Kafka topic to publish change events gathered through provided query.

Cypher query to use for gathering changes. Requires both neo4j.query.streaming-property to be in the result set, and $lastCheck query parameter used for tracking changes.

Property name that is both present in the result set of the specified query and used as a filter to query changes from a previous value.

Default: timestamp

Maximum amount of time Kafka Connect poll request will wait for a change to be received from the database (valid units are: ms, s, m, h and d; default unit is s).

Default: 5s

The interval in which the database will be polled for changes during neo4j.query.poll-duration (valid units are: ms, s, m, h and d; default unit is s).

Default: 1s

Maximum amount of time query is allowed to run (valid units are: ms, s, m, h and d; default unit is s).

Default: 0s

This forces the schema encoding of map values to be structs (default: true). Setting the value to false will only affect the schema encoding of maps with homogeneous value types. Maps with heterogeneous value types will remain encoded as structs. Note that the Protobuf converter does not support map message types (map message types can happen in the COMPACT payload mode and when this parameter is set to false).