Querying with Cypher¶

There are two ways to run Cypher, and most code wants the first:

The examples on this page share this setup:

import pyarrow as pa
from lance_csr import CypherEngine, GraphConfig

config = (
    GraphConfig.builder()
    .node_table("Person", "id")
    .node_table("City", "id")
    .rel_table("KNOWS", "Person", "Person", "src", "dst")
    .rel_table("LIVES_IN", "Person", "City", "src", "dst")
    .build()
)
datasets = {
    "Person": pa.table({
        "id": [1, 2, 3, 4],
        "name": ["Alice", "Bob", "Carol", "David"],
        "age": [28, 34, 29, 42],
    }),
    "City": pa.table({"id": [10, 20], "name": ["London", "Sydney"]}),
    "KNOWS": pa.table({"src": [1, 1, 2, 3], "dst": [2, 3, 4, 4]}),
    "LIVES_IN": pa.table({"src": [1, 2, 3, 4], "dst": [10, 10, 20, 20]}),
}
engine = CypherEngine(config, datasets, use_csr=True)

CypherEngine¶

CypherEngine(config, datasets, use_csr=False)

The constructor validates the datasets against the config, registers them with a DataFusion session, and, with use_csr=True, builds the CSR index. After that, every query reuses the same catalog and session.

execute(query, params=None)¶

Runs the query and returns the whole result as a pyarrow.Table:

table = engine.execute("""
    MATCH (p:Person)-[:LIVES_IN]->(c:City)
    RETURN c.name AS city, collect(p.name) AS people, avg(p.age) AS avg_age
    ORDER BY city
""")
print(table.to_pylist())
# [{'city': 'London', 'people': ['Alice', 'Bob'], 'avg_age': 31.0},
#  {'city': 'Sydney', 'people': ['Carol', 'David'], 'avg_age': 35.5}]

Output columns are named by their AS alias, or by the expression text (p.name) when there is none.

Parameters¶

Write $name in the query and pass a dict as the second argument. Parameter values can be numbers, strings, booleans, None, lists, string-keyed maps, and datetime.datetime or datetime.date values:

import datetime

engine.execute(
    "MATCH (p:Person) WHERE p.age >= $min_age AND p.name <> $skip RETURN p.name AS name",
    {"min_age": 30, "skip": "Bob"},
).to_pylist()  # [{'name': 'David'}]

Lists retain integer IDs, and maps can contain nested lists/maps and nulls:

engine.execute(
    "MATCH (p:Person) WHERE p.id IN $ids RETURN p.name AS name ORDER BY name",
    {"ids": [1, 3]},
).to_pylist()  # [{'name': 'Alice'}, {'name': 'Carol'}]

Pagination also accepts parameters, for example RETURN 1 AS n LIMIT $take with {"take": 1}. WITH ... LIMIT $take uses the same rules. Missing, null, boolean, fractional, negative and overflowing counts raise ValueError; arbitrary expressions and WITH SKIP remain unsupported.

Map keys must be strings, integer parameters must fit signed 64-bit values, and nesting is limited to 128 levels. Cyclic containers and unsupported Python objects raise ValueError. Lists follow literal-list coercion: mixed string/numeric values become strings because Arrow lists have one element type. Containers are expression values; inline node/relationship property filters still require scalar parameters.

Datetimes compare against timestamp columns, and dates against date columns. Parameters are substituted before planning, so they are also allowed in inline property maps: (p:Person {id: $id}).

execute_batches(query, params=None)¶

Streams the result as a pyarrow.RecordBatchReader instead of building one table:

reader = engine.execute_batches("MATCH (p:Person) RETURN p.id AS id, p.name AS name")
rows = 0
for batch in reader:  # each batch is a pyarrow.RecordBatch
    rows += batch.num_rows
print(rows)  # 4

• Batches are computed as you read them. Scans, filters and projections stream with bounded memory; joins, sorts and aggregations may still buffer their inputs.
• Planning errors (a syntax error, an unknown label) raise immediately. Execution errors raise while reading.
• The reader keeps its snapshot alive, so a concurrent refresh() doesn't affect it.
• Dropping the reader cancels the remaining work.

explain(query, params=None)¶

Returns the plan as text: the graph logical plan, DataFusion's logical plan and the physical plan. It's the quickest way to check whether a hop used the CSR (CsrExpandExec) or a join (HashJoinExec), and whether filters reached the Lance scan:

plan = engine.explain("MATCH (a:Person)-[:KNOWS]->(b:Person) RETURN b.name")
print("CsrExpandExec" in plan)  # True

Other methods¶

CypherQuery¶

CypherQuery parses a query up front and runs it against whatever datasets you hand it. Each builder method returns a new query:

from lance_csr import CypherQuery

query = (
    CypherQuery("MATCH (p:Person) WHERE p.age > $age RETURN p.name AS name ORDER BY name")
    .with_config(config)
    .with_parameter("age", 30)
)
print(query.execute(datasets).to_pydict())  # {'name': ['Bob', 'David']}

strategy takes an ExecutionStrategy. ExecutionStrategy.DataFusion is the default and the only one implemented; ExecutionStrategy.LanceNative is reserved and raises an error.

Supported Cypher¶

The Graph Data Council (GDC), formerly known as the Linked Data Benchmark Council (LDBC), maintains the GDC SNB Interactive complex queries. lance-csr implements a large read-only subset of openCypher. The tables below specify its supported surface; relationship-path operations needed by the unchanged IC14 query are not yet supported.

Clauses¶

Integer list literals retain signed 64-bit values: RETURN [1, 2, 3] AS ids returns integers, and UNWIND [1131, 19791209300775] AS id RETURN id preserves both IDs exactly. Numeric literals with a decimal point or exponent, such as [0.1, 1e-3], form float32 vectors. Whole-number lists can also be used as query vectors; conversion to float32 happens inside the vector operation. A configured engine is still required for standalone RETURN queries.

Patterns¶

// Direction: outgoing, incoming or either
MATCH (a:Person)-[:KNOWS]->(b)     // outgoing
MATCH (a:Person)<-[:KNOWS]-(b)     // incoming
MATCH (a:Person)-[:KNOWS]-(b)      // either direction

// Variable length: *, *n, *n..m, *..m, *n..
MATCH (a:Person {id: 1})-[:KNOWS*1..3]->(b:Person)

// Alternation, also with variable length
MATCH (p:Person)-[:LIKES_POST|LIKES_COMMENT]->(m)

// Relationship variables and properties
MATCH (a:Person)-[k:KNOWS]->(b:Person) WHERE k.since > 2020

// Shortest paths (one relationship type, both ends bound)
MATCH p = shortestPath((a:Person {id: 1})-[:KNOWS*]-(b:Person {id: 4}))
RETURN length(p), nodes(p)

• Variable length defaults to a minimum of 1 and a maximum of 20 hops; a maximum above 20 is rejected, and zero-length paths (*0..) are not supported. Each length becomes one branch of a union.
• Undirected variable-length paths skip immediate backtracking (a-b-a).
• shortestPath and allShortestPaths need exactly one relationship type, no relationship variable, and named end nodes. Their length is unbounded unless you give a maximum. nodes(p) returns the node keys. With no path, MATCH drops the row and OPTIONAL MATCH returns null.
• Named paths (p = ...) are only supported with shortestPath and allShortestPaths.

Expressions and predicates¶

Functions¶

The string and list functions follow the Cypher v9 reference, within the list representation described above. size counts Unicode characters in a string or the outer elements of a list. left(text, length), right(text, length) and substring(text, start [, length]) count characters; the substring start is zero-based. Lengths and starts must be non-negative integers. A null input string returns null; a null start or length on a non-null string raises an error. trim, lTrim and rTrim remove whitespace, and replace replaces all literal matches. split preserves empty fields; an empty delimiter splits into characters. Splitting an empty string returns [''].

last([]) returns null; tail([]) and reverse([]) return empty lists. reverse also accepts strings. range(start, end [, step]) includes the end when reached, defaults to step 1, and accepts negative steps. A step pointing away from the end gives an empty list; zero steps and non-integer arguments raise an error. Unsigned Arrow arguments outside Int64 range also raise an error; they are not converted to null. Null arguments return null.

result = engine.execute("""
    RETURN substring('a猫😀z', 1, 2) AS text,
           reverse(range(1, 5, 2)) AS numbers
""")
print(result.to_pylist())  # [{'text': '猫😀', 'numbers': [5, 3, 1]}]

abs preserves integer results; the other math functions return floating point values. Integer abs raises an error on absolute-value overflow or unsigned inputs outside Int64 range. The other math functions convert numeric inputs, including UInt64, to Float64. round accepts one argument, with ties toward positive infinity (round(-1.5) is -1.0), as specified in the Neo4j 4.4 numeric reference. sqrt of a negative number returns NaN. These functions return null for null input and reject non-numeric values, including strings and booleans.

result = engine.execute("RETURN abs(-5) AS magnitude, round(-1.5) AS rounded, sqrt(9) AS root")
print(result.to_pylist())  # [{'magnitude': 5, 'rounded': -1.0, 'root': 3.0}]

keys returns the field names in a map, including keys with null values. For a node or relationship, it returns its exposed, non-null property names, including the ID or endpoint columns already exposed by the engine. Property names use the engine's normalized column spelling. Key order is not guaranteed. keys({}) returns an empty list. labels(node) returns the canonical label configured for that node, even if the query spells it differently. Both functions preserve aliases carried through WITH and return null for a missing entity from OPTIONAL MATCH.

result = engine.execute("""
    MATCH (p:Person {name: 'Alice'}) WITH p AS person
    RETURN labels(person) AS labels, size(keys(person)) AS properties
""")
print(result.to_pylist())  # [{'labels': ['Person'], 'properties': 3}]

Any other function raises ValueError with Cypher function 'name' is not implemented. This is a bounded function set, not a claim of full Cypher conformance. relationships(p) is not supported.

Ordered reduce and list predicates¶

reduce(total = initial, item IN list | expression) evaluates the expression once per element in list order. Each step sees the previous accumulator value. An empty list returns the initial value; a null list returns null. Null elements are passed to the expression, so use coalesce if you want to replace them. The accumulator and iterator names are local to the body: the initial value and list still refer to outer variables. The accumulator must keep a compatible Arrow type; integer initial values can promote to floating point when the expression requires it.

result = engine.execute("""
    RETURN reduce(total = 0, item IN [1, 2, 3] | total * 10 + item) AS ordered,
           all(item IN [1, 2, 3] WHERE item > 0) AS positive,
           single(item IN [1, 2, 3] WHERE item = 2) AS exactly_one
""")
print(result.to_pylist())  # [{'ordered': 123, 'positive': True, 'exactly_one': True}]

The list predicates use function(item IN list WHERE predicate) syntax. all requires every predicate result to be true, any requires at least one, none requires no true results, and single requires exactly one. For an empty list, all and none are true; any and single are false. A null list returns null. A null predicate result makes the answer null unless a known result already decides it: false decides all, true decides any and none, and two true results decide single. Predicates can inspect null elements explicitly with item IS NULL.

Both forms can capture outer values and nest scalar folds or predicates. List predicates also support correlated graph predicates such as EXISTS { MATCH ... }. A reduce input can be a list comprehension, pattern comprehension, or collect(...); its body supports scalar expressions but rejects graph patterns, comprehensions, and aggregation. Compute such a list before the body. These semantics follow the pinned Cypher 4.4 list reference and predicate reference.

Example: a little of everything¶

result = engine.execute("""
    MATCH (p:Person)
    OPTIONAL MATCH (p)-[:KNOWS]->(f:Person)
    WITH p, count(f) AS friends
    WHERE EXISTS { MATCH (p)-[:LIVES_IN]->(:City {name: 'London'}) }
    RETURN p.name AS name,
           friends,
           CASE WHEN p.age >= 30 THEN 'senior' ELSE 'junior' END AS band
    ORDER BY name
""")
print(result.to_pylist())
# [{'name': 'Alice', 'friends': 2, 'band': 'junior'},
#  {'name': 'Bob', 'friends': 1, 'band': 'senior'}]

Case sensitivity¶

Keywords, function names, variables, labels, relationship types and property names are all case-insensitive. match (P:person) return p.NAME works, and the output column keeps the spelling you wrote.

An output alias may reuse an input variable's name. Expressions in the same RETURN still use the input variable; the following ORDER BY uses the output alias. For example, RETURN p.name AS p, p.id AS id ORDER BY p returns names and IDs, sorted by name. This also works with aggregation and OPTIONAL MATCH.

Errors¶