Installation

Choose the correct version

Make sure to match both the Spark version and the Scala version of your setup. See Compatibility to determine what connector version is suitable for your setup.

The examples below show the installation process using the latest 6.1 connector.

Usage with the Spark shell

Starting from version 5.4.0, the connector is not published to Spark Packages anymore.

The connector is available via Maven Central:

$SPARK_HOME/bin/spark-shell --packages org.neo4j.connectors:spark:6.1.2-s_2.13

$SPARK_HOME/bin/pyspark --packages org.neo4j.connectors:spark:6.1.2-s_2.13

Alternatively, you can download the connector JAR file from the Neo4j Connector Page or from the GitHub releases page and run the following command to launch a Spark interactive shell with the connector included:

$SPARK_HOME/bin/spark-shell --jars neo4j-spark-connector-6.1.2-s_2.13.jar

$SPARK_HOME/bin/pyspark --jars neo4j-spark-connector-6.1.2-s_2.13.jar

Self-contained applications

For non-Python applications:

Include the connector in your application using the application’s build tool.
Package the application.
Use spark-submit to run the application.

For Python applications, run spark-submit directly.

As for the spark-shell, you can run spark-submit via Spark Packages or with a local JAR file. See the Quickstart for code examples.

A minimal build.sbt

name := "Spark App"
version := "1.0"
scalaVersion := "2.13.18"

libraryDependencies += "org.apache.spark" %% "spark-sql" % "4.1.2"
libraryDependencies += "org.neo4j.connectors" %% "spark" % "6.1.2-s_2.13"

If you use the sbt-spark-package plugin, add the following to your build.sbt instead:

scala spDependencies += "org.neo4j.connectors/spark:6.1.2-s_2.13"

A minimal pom.xml

<project>
  <groupId>org.neo4j</groupId>
  <artifactId>spark-app</artifactId>
  <modelVersion>4.0.0</modelVersion>
  <name>Spark App</name>
  <packaging>jar</packaging>
  <version>1.0</version>
  <dependencies>
    <dependency>
      <groupId>org.apache.spark</groupId>
      <artifactId>spark-sql_2.13</artifactId>
      <version>4.1.2</version>
      <scope>provided</scope>
    </dependency>
    <dependency>
      <groupId>org.neo4j.connectors</groupId>
      <artifactId>spark</artifactId>
      <version>6.1.2-s_2.13</version>
    </dependency>
  </dependencies>
</project>

Other build tools

Gradle

dependencies {
    // list of dependencies
    compile "org.neo4j.connectors:spark:6.1.2-s_2.13"
}

Thin JAR
Introduced in 6.1.2

By default, the connector JAR bundles its dependencies, such as Netty, Kotlin and Reactor. If you manage your own classpath, for example by placing each dependency as a separate JAR, the bundled classes can clash with other versions of the same libraries.

In this case, use the thin classifier. The thin JAR contains only the connector, and its dependencies are declared in the Maven POM and resolved as usual.

<dependency>
  <groupId>org.neo4j.connectors</groupId>
  <artifactId>spark</artifactId>
  <version>6.1.2-s_2.13</version>
  <classifier>thin</classifier>
</dependency>

The thin JAR is also included in the release archive, as neo4j-spark-connector-6.1.2-s_2.13-thin.jar. Note that you must then add its dependencies to the classpath yourself.