Getting started

Installation

Neo4j Graph Analytics for Snowflake is delivered as a Native Application. The application can be installed from the Snowflake Marketplace. During the installation, you are required to enable Event sharing. See more about managing event sharing.

Application privileges

The application requires the CREATE COMPUTE POOL and CREATE WAREHOUSE privileges. The easiest way to grant those privileges is via Snowsight by selecting Data Products ⇒ Apps ⇒ Neo4j Graph Analytics ⇒ Privileges ⇒ Grant. Finally, click the Activate button on the same page. This will trigger the application to create internal resources such as compute pools.

When running algorithms, the application needs certain privileges to be able to run. For details, please see administration of application privileges. We also give a Usage example below to show both how to grant these privileges and run an algorithm.

Consumer roles and privileges

Users of the application also require certain privileges. For details please see administration of consumer privileges. The usage example below includes the aspect of consumer privileges.

Introduction to the Algorithm API

Neo4j Graph Analytics offers a catalogue of algorithms, from PageRank over Dijkstra to WCC.

The execution of algorithms is done in three steps: projection, computation, and writing results. The project-compute-write pattern is a common pattern in graph processing, where you first project a graph from your data, then compute some properties or metrics on that graph, and finally write the results back to your data store.

The three steps are covered more extensively in the respective chapters Project, Compute and Write.

Writing results

Immediately after an algorithm finishes, we write data back to your tables for inspection and further processing. Here we require you to specify things like an output table and labels (a Neo4j concept, think table name).

Preparation

Before you run an algorithm, your environment needs to be set up correctly.

We need to make sure to use a role with the required privileges to use the application.

USE ROLE <consumer_user_role>;

Optionally, to run an algorithm without giving the fully qualified endpoint names we can execute the following command.

USE DATABASE Neo4j_Graph_Analytics;

Here we are assuming that the application is installed under the default name, Neo4j_Graph_Analytics.

Usage example

Quickstart script

Fill in the names below and copy the script. It runs top to bottom as a single script, whether you run it in Snowsight with Run All or as a .sql file with the Snowflake CLI.

Part 1 creates example data, so that the algorithm has something to run on. If you already have node and relationship tables or views matching the required column names you can skip this step.

Part 2 grants the application access to your data. It grants your user’s access to the application, runs the algorithm and selects the results.

The prerequisites are that the application is installed and activated, and that you can use a role such as ACCOUNTADMIN.

Quickstart script

-- Neo4j Graph Analytics quickstart for {{app}}
-- Runs top to bottom as a single script.

-- Part 1: the example graph -- six nodes, four relationships, two components.
-- Nothing here is specific to Neo4j Graph Analytics; it just gives the
-- algorithm something to run on. Delete this part if you already have node and
-- relationship tables with the required column names.
USE ROLE ACCOUNTADMIN;

CREATE DATABASE IF NOT EXISTS {{db}};
CREATE SCHEMA IF NOT EXISTS {{db}}.{{schema}};

CREATE OR REPLACE TABLE {{db}}.{{schema}}.NODES (nodeId NUMBER);
INSERT INTO {{db}}.{{schema}}.NODES VALUES (1), (2), (3), (4), (5), (6);

CREATE OR REPLACE TABLE {{db}}.{{schema}}.RELATIONSHIPS (sourceNodeId NUMBER, targetNodeId NUMBER);
INSERT INTO {{db}}.{{schema}}.RELATIONSHIPS VALUES (1, 2), (2, 3), (4, 5), (5, 6);

-- Part 2: privileges, and running the algorithm.

-- 1. A role that may use the application, granted to you
USE ROLE ACCOUNTADMIN;
CREATE ROLE IF NOT EXISTS {{role}};
GRANT APPLICATION ROLE {{app}}.app_user TO ROLE {{role}};
SET quickstart_user = (SELECT CURRENT_USER());
GRANT ROLE {{role}} TO USER IDENTIFIER($quickstart_user);

-- 2. Let the application and the consumer role read your data and write its results back
CREATE DATABASE ROLE IF NOT EXISTS {{db}}.{{dbRole}};
GRANT USAGE ON DATABASE {{db}} TO DATABASE ROLE {{db}}.{{dbRole}};
GRANT USAGE ON SCHEMA {{db}}.{{schema}} TO DATABASE ROLE {{db}}.{{dbRole}};
GRANT SELECT ON ALL TABLES IN SCHEMA {{db}}.{{schema}} TO DATABASE ROLE {{db}}.{{dbRole}};
GRANT SELECT ON FUTURE TABLES IN SCHEMA {{db}}.{{schema}} TO DATABASE ROLE {{db}}.{{dbRole}};
GRANT CREATE TABLE ON SCHEMA {{db}}.{{schema}} TO DATABASE ROLE {{db}}.{{dbRole}};
GRANT DATABASE ROLE {{db}}.{{dbRole}} TO APPLICATION {{app}};
GRANT DATABASE ROLE {{db}}.{{dbRole}} TO ROLE {{role}};

-- 3. Run WCC as the consumer role
USE ROLE {{role}};
CALL {{app}}.graph.wcc('{{pool}}', {
    'project': {
        'nodeTables': ['{{db}}.{{schema}}.NODES'],
        'relationshipTables': {
            '{{db}}.{{schema}}.RELATIONSHIPS': {
                'sourceTable': '{{db}}.{{schema}}.NODES',
                'targetTable': '{{db}}.{{schema}}.NODES',
                'orientation': 'NATURAL'
            }
        }
    },
    'compute': { 'consecutiveIds': true },
    'write': [{
        'nodeLabel': 'NODES',
        'outputTable': '{{db}}.{{schema}}.NODES_COMPONENTS'
    }]
});

-- 4. Two components of three nodes each
SELECT * FROM {{db}}.{{schema}}.NODES_COMPONENTS;

{
  "intro": "Fill in your names and the script below updates as you type. Nothing is sent anywhere; it is assembled in your browser.",
  "groups": [
    { "legend": "Names to use", "fields": ["app", "db", "schema", "role", "dbRole", "pool"] }
  ],
  "fields": {
    "app": {
      "label": "Application name",
      "description": "The name the application is installed under.",
      "default": "Neo4j_Graph_Analytics",
      "pattern": "[A-Za-z0-9_$\"]"
    },
    "db": {
      "label": "Database to create the example in",
      "description": "Created by the script if it does not exist.",
      "default": "EXAMPLE_DB",
      "pattern": "[A-Za-z0-9_$\"]"
    },
    "schema": {
      "label": "Schema to create the example in",
      "description": "Created by the script if it does not exist.",
      "default": "DATA_SCHEMA",
      "pattern": "[A-Za-z0-9_$\"]"
    },
    "role": {
      "label": "Consumer role to create",
      "description": "May use the application. Created by the script and granted to you.",
      "default": "MY_CONSUMER_ROLE",
      "pattern": "[A-Za-z0-9_$\"]"
    },
    "dbRole": {
      "label": "Database role to create",
      "description": "Carries the application's access to the example schema.",
      "default": "EXAMPLE_DB_ROLE",
      "pattern": "[A-Za-z0-9_$\"]"
    },
    "pool": {
      "label": "Compute pool",
      "description": "One of CPU_X64_XS, CPU_X64_M, CPU_X64_L, HIGHMEM_X64_S, HIGHMEM_X64_M, HIGHMEM_X64_L.",
      "default": "CPU_X64_XS",
      "pattern": "[A-Za-z0-9_$\"]"
    }
  }
}

The fields accept identifier characters only; if an identifier needs double quotes in Snowflake, include the quotes in the field. For the grants a real deployment needs, rather than this example, see the permission generator. The same statements are built up step by step in the walkthrough below.

Walkthrough

We will give a more comprehensive example on how to run Neo4j Graph Analytics using the default warehouse configuration and selecting the CPU_X64_XS compute pool.

In the following example we assume that: - the application is installed as Neo4j_Graph_Analytics (default). - the role executing the queries has access to the consumer database objects referred to. - the role executing the queries has granted usage of the app_user application role.

For our example, we create two tables, one for nodes and one for relationships. The graph we want to project consists of six nodes and four relationships. Note the column names nodeId and sourceNodeId, targetNodeId respectively. The application requires these columns to exist in order to extract the data. Optional additional columns are treated as node or relationship properties, respectively.

If node and relationship tables are already available, but do not have the required column names, you can create views on top of them.

-- Use a role with the required privileges
USE ROLE ACCOUNTADMIN;

-- Create a consumer role for users of the Graph Analytics application
CREATE ROLE IF NOT EXISTS MY_CONSUMER_ROLE;
GRANT APPLICATION ROLE Neo4j_Graph_Analytics.app_user TO ROLE MY_CONSUMER_ROLE;
SET MY_USER = (SELECT CURRENT_USER());
GRANT ROLE MY_CONSUMER_ROLE TO USER IDENTIFIER($MY_USER);

USE SCHEMA EXAMPLE_DB.DATA_SCHEMA;
CREATE TABLE NODES (nodeId Number);
INSERT INTO NODES VALUES (1), (2), (3), (4), (5), (6);
CREATE TABLE RELATIONSHIPS (sourceNodeId Number, targetNodeId Number);
INSERT INTO RELATIONSHIPS VALUES (1, 2), (2, 3), (4, 5), (5, 6);

-- Grants needed for the app to read consumer data stored in tables and views, using a database role
USE DATABASE EXAMPLE_DB;
CREATE DATABASE ROLE IF NOT EXISTS MY_DB_ROLE;
GRANT USAGE ON DATABASE EXAMPLE_DB TO DATABASE ROLE MY_DB_ROLE;
GRANT USAGE ON SCHEMA EXAMPLE_DB.DATA_SCHEMA TO DATABASE ROLE MY_DB_ROLE;
GRANT SELECT ON ALL TABLES IN SCHEMA EXAMPLE_DB.DATA_SCHEMA TO DATABASE ROLE MY_DB_ROLE;
GRANT SELECT ON ALL VIEWS IN SCHEMA EXAMPLE_DB.DATA_SCHEMA TO DATABASE ROLE MY_DB_ROLE;
-- Future tables also include tables that are created by the application itself.
-- This is useful as many use-cases require running algorithms in a sequence and using the output of a prior algorithm as input.
GRANT SELECT ON FUTURE TABLES IN SCHEMA EXAMPLE_DB.DATA_SCHEMA TO DATABASE ROLE MY_DB_ROLE;
GRANT SELECT ON FUTURE VIEWS IN SCHEMA EXAMPLE_DB.DATA_SCHEMA TO DATABASE ROLE MY_DB_ROLE;
GRANT CREATE TABLE ON SCHEMA EXAMPLE_DB.DATA_SCHEMA TO DATABASE ROLE MY_DB_ROLE;
GRANT DATABASE ROLE MY_DB_ROLE TO APPLICATION Neo4j_Graph_Analytics;
GRANT DATABASE ROLE MY_DB_ROLE TO ROLE MY_CONSUMER_ROLE;

-- Use the consumer role to run the algorithm and inspect the output
USE ROLE MY_CONSUMER_ROLE;

We capture this in projection configuration like so:

'project': {
    'nodeTables': ['EXAMPLE_DB.DATA_SCHEMA.NODES'],
    'relationshipTables': {
      'EXAMPLE_DB.DATA_SCHEMA.RELATIONSHIPS': {
        'sourceTable': 'EXAMPLE_DB.DATA_SCHEMA.NODES',
        'targetTable': 'EXAMPLE_DB.DATA_SCHEMA.NODES',
        'orientation': 'NATURAL'
      }
    }
  }

Both, nodes and relationships, can be read from multiple tables. Using the nodeTables and relationshipTables configuration parameters, we can specify which tables to read from. The nodeTables configuration parameter specifies an array of tables that contain the nodes. The name of each node table is mapped to a node label in the graph. The relationship tables are specified in a map, where the key is the name of the table and the value is a map of configuration parameters. The name of each relationship table is mapped to a relationship type in the graph. The sourceTable and targetTable configuration parameters specify the node tables that the relationship table refers to.

If we want to project relationships using a different orientation, we can specify that in the configuration. Possible values are NATURAL (default), UNDIRECTED and REVERSE.

In our example, we will use the Weakly Connected Components algorithm (WCC) to find disconnected parts of the graph. We can put together the algorithm computation configuration for WCC by specifying that we want "nice numbers" on the output side:

'compute': { 'consecutiveIds': true }

Once we have computed WCC, we write the results back to a table for further analytics. That table will be created and overridden if it already exists.

'write': [{
    'nodeLabel': 'NODES',
    'outputTable': 'EXAMPLE_DB.DATA_SCHEMA.NODES_COMPONENTS'
  }]

Since this is the first time writing back to our example schema, we also need to consider privileges. We need to grant the CREATE TABLE privilege on the schema to the application.

Finally with all that preamble, we are ready to go!

CALL Neo4j_Graph_Analytics.graph.wcc('CPU_X64_XS', {
    'project': {
        'nodeTables': ['EXAMPLE_DB.DATA_SCHEMA.NODES'],
        'relationshipTables': {
            'EXAMPLE_DB.DATA_SCHEMA.RELATIONSHIPS': {
                'sourceTable': 'EXAMPLE_DB.DATA_SCHEMA.NODES',
                'targetTable': 'EXAMPLE_DB.DATA_SCHEMA.NODES',
                'orientation': 'NATURAL'
            }
        }
    },
    'compute': { 'consecutiveIds': true },
    'write': [{
        'nodeLabel': 'NODES',
        'outputTable': 'EXAMPLE_DB.DATA_SCHEMA.NODES_COMPONENTS'
    }]
});

Please note that we could have called USE DATABASE Neo4j_Graph_Analytics followed by CALL graph.wcc(…) to avoid the fully qualified name.

Once this query ran, we can select the components from the table.

SELECT * FROM EXAMPLE_DB.DATA_SCHEMA.NODES_COMPONENTS;

This will list the component for each node. We can see that the graph consists of two separate components, each containing three nodes.

NODE	VALUE
1	0
2	0
3	0
4	1
5	1
6	1

Most algorithms produce node property results. Some algorithms, like KNN and Node Similarity, produce relationship results. These specific variants will be covered in the algorithms catalogue.

Now results have been written back to your end, and you can inspect and further process them - for example by feeding them into another Neo4j Graph Analytics algorithm.

Larger example

For a larger usage example including preparation of data into the node and relationship table/view format, see Basket analysis example on TPC-H data. It uses the TPC-H sample data available in Snowflake.