Python Bindings

██████╗ ██╗   ██╗████████╗██╗  ██╗ ██████╗ ███╗   ██╗
██╔══██╗╚██╗ ██╔╝╚══██╔══╝██║  ██║██╔═══██╗████╗  ██║
██████╔╝ ╚████╔╝    ██║   ███████║██║   ██║██╔██╗ ██║
██╔═══╝   ╚██╔╝     ██║   ██╔══██║██║   ██║██║╚██╗██║
██║        ██║      ██║   ██║  ██║╚██████╔╝██║ ╚████║
╚═╝        ╚═╝      ╚═╝   ╚═╝  ╚═╝ ╚═════╝ ╚═╝  ╚═══╝

Maelys DL exposes a native Python binding for in-process use — no browser, no WASM runtime. It targets tooling that wants direct, native-speed access to the engine from a research pipeline, an offline analysis script, or a long-running service: real Python exceptions instead of sentinel return codes, real object handles instead of a single implicit session per module instance, and the full native int64 range for integer terms.

The underlying concepts — EDB facts, predicate kinds, solving, querying — are the same as in C and WASM, documented once at the C level. See Bindings overview for the concept-to-page map, how this binding compares to WASM, and where each fits relative to the shared engine.

This page covers the Python surface only: installation, domain registration, object lifecycle, term encoding, and the API reference.

Installation

Python 3.10 or later is required. The binding uses the public X | Y union syntax and built-in generic types throughout its annotations; older Python versions are not supported.

The binding lives in bindings/python/ inside the engine repository — it is not published as a package yet. Build the native shared libraries with CMake, then build the cffi extension, then verify the import:

cmake -S . -B build/python-small -DMAELYS_DATALOG_BUILD_PYTHON_BINDING=ON
cmake --build build/python-small --target maelys_py_bind
 
python bindings/python/build_cffi.py
 
PYTHONPATH=bindings/python python -c "import maelys_datalog; print(maelys_datalog.Engine().limits)"
# BuildLimits(max_symbols=..., ...)

build_cffi.py takes no build-directory argument — it always links against whatever .so/.dylib files CMake's build step already copied into bindings/python/maelys_datalog/ (see Native library discovery below). There is no flag or environment variable to tell it which CMake profile to use; if you switch profiles, rebuild both steps in order before re-running build_cffi.py — a stale SMALL/LARGE mix is not caught by the cffi build step itself, only by Engine().limits disagreeing with what you expect at runtime.

Native library discovery

A compiled Python extension is itself a dynamic library, and it depends on two others: libmaelys_datalog_shared (the engine) and libmaelys_py_bind (the shim). At import time, the OS has to find those two files on disk — and by default it only looks in a handful of system locations, not wherever you happen to have built them.

The build solves this by copying the two .so/.dylib files into the same folder as the Python package:

bindings/python/maelys_datalog/
├── __init__.py
├── _maelys_cffi....so          ← the compiled extension
├── libmaelys_datalog_shared.so ← copied here by the build
└── libmaelys_py_bind.so        ← copied here by the build

Without anything else, the extension still would not find its neighbors — you would have to tell the OS where to look, every time, with an environment variable. This is a separate problem from Python finding the maelys_datalog package in the first place — that part is ordinary Python package discovery (PYTHONPATH, or running from inside bindings/python/), nothing to do with $ORIGIN. $ORIGIN only kicks in after Python has already found and loaded the compiled extension, to help that file find its two native dependencies.

To see the $ORIGIN difference specifically, keep the Python-package-discovery part constant (PYTHONPATH, works from any directory) and vary only the native-library part:

# check_import.py — save anywhere, does not need to live in bindings/python/
import maelys_datalog
print("import OK")

# The annoying way, without $ORIGIN/@loader_path: PYTHONPATH lets Python find
# the package, but the compiled extension still can't find its two native
# libraries on its own — fails regardless of which directory you run from
PYTHONPATH=/path/to/bindings/python python check_import.py
# ImportError: libmaelys_datalog_shared.so: cannot open shared object file
 
# ...only works once you ALSO set LD_LIBRARY_PATH, by hand, every time
PYTHONPATH=/path/to/bindings/python \
LD_LIBRARY_PATH=/path/to/bindings/python/maelys_datalog \
python check_import.py
# import OK — but only because both variables were set on this exact command

$ORIGIN (Linux) and @loader_path (macOS) remove the second variable entirely: they are special tokens baked into the compiled .so itself at build time, meaning "look in my own folder" — not a fixed path, whichever folder the file happens to be in right now. Because all three files were copied together, each one's "own folder" is the same folder, so they always find each other as long as they stay together. PYTHONPATH is still needed (that is ordinary Python, unrelated to this mechanism) — LD_LIBRARY_PATH is not, from any directory:

# The actual way — only PYTHONPATH, no LD_LIBRARY_PATH, from anywhere
PYTHONPATH=/path/to/bindings/python python check_import.py
# import OK

Loading path

There are two ways to get a policy loaded: the Manifest path (JSON manifest, SHA-256 verification, Public Query Whitelist) and the Inline path (raw .dl text, no manifest, no whitelist). Each path has two variants:

This binding supports exactly one of these four: Inline, dynamic. Neither Manifest variant is supported — no SHA-256 verification, no Public Query Whitelist. See Registries — Loading paths for the full model behind all four.

ruleset = engine.load_inline_ruleset(domain, ruleset_id, source) calls the shim's maelys_py_load_inline_ruleset, which wraps maelys_datalog_load_policy_inline directly. engine.register_domain(...) calls maelys_datalog_domain_registry_register — the registration step the dynamic variant expects to have happened beforehand. Together, the two Python calls are the dynamic variant, unchanged from its C shape: register, then load.

Why not the static variant too? load_policy_inline_with_static_domain collapses registration and loading into one call, specifically for a domain already written as a compile-time C array (see Registries — Method 1). That solves a C-specific problem: skipping a second call when the table is already sitting in C source. In Python, register_domain(name, predicates) from a plain list is already the one-call ergonomic — there is no compile-time C array to skip past, so adding the combined function would just reimplement what two Python calls already do.

Why not the Manifest path? SHA-256 verification and a Public Query Whitelist are real, missing capabilities — not something the current model already covers under a different name. Adding either variant is a bounded change (a new shim export mirroring load_inline_ruleset, calling maelys_datalog_manifest_load_from_text for the buffer form, with the manifest JSON and SHA-256 built in Python via hashlib) — deferred because no current caller of this binding loads policy source from an untrusted or external source that would need integrity verification before execution.

Session lifecycle

Ruleset owns its own policy_set and symbol table — not Engine, which only holds the build limits and the last load diagnostic. This is why multiple rulesets can be loaded and closed independently within one process, unlike the domain registry below, which is shared.

Python API walkthrough

Four classes wrap the shim. Engine holds the build limits and the process-wide domain registry; Ruleset owns a loaded policy and its symbol table; Edb holds one evaluation's input facts; SolveResult holds one solve's derived facts. This section walks through them in the order you use them.

Factory

from maelys_datalog import Engine
 
engine = Engine()

Engine() loads the native library and returns a ready instance. Unlike the WASM binding's MaelysPlayground.create(), there is no factory module or WASM URL to pass — the shared libraries are already resolved through $ORIGIN/@loader_path (see Installation) before Python code runs at all.

More precisely: import maelys_datalog is what loads the compiled cffi extension and its two native dependencies. At import it also runs a one-time ABI self-check: it asks the compiled shim for its real struct sizes, field offsets, and enum values (maelys_py_get_abi_layout / maelys_py_get_abi_constants) and compares them against what the Python-side cffi definitions expect. A shim rebuilt out of sync with the Python package therefore fails loudly at import rather than silently misreading fields later. All of that has already happened by the time Engine() is even called. Engine() itself only creates the native engine handle and reads the build limits into .limits (see Build limits below); it does not repeat any loading or ABI work.

Build limits

engine = Engine()
engine.limits
# BuildLimits(max_symbols=..., string_pool_bytes=..., max_predicates=...,
#             max_rules=..., max_arity=..., max_body_literals=...,
#             max_depth=..., max_edb_facts=..., max_idb_facts=...,
#             max_facts_per_pred=...)

Same ten fields as WASM's buildLimits(), read once at Engine() construction — the active build profile's observable capacities, compile-time constants, not runtime occupancy. Unlike WASM, there is no separate call to make later: .limits is already populated by the time Engine() returns.

Predicate kind flags

from maelys_datalog import PRED_EDB, PRED_IDB, PRED_QUERY, PRED_POLICY_FACT
 
PRED_EDB         # 1 — runtime fact supplied by the caller
PRED_IDB         # 2 — derived by Datalog rules
PRED_QUERY       # 4 — inspectable through the query API
PRED_POLICY_FACT # 8 — declared by the policy source

Same bit values as WASM's PredKind. These constants are read from the engine's ABI constants at import time, not written as literals in the Python source — so if the engine ever changed a flag value, the Python constant would track it automatically instead of silently disagreeing with the native side. Combine with |: PRED_IDB | PRED_QUERY.

Domain definition

Unlike the WASM dynamic builder — which is scoped to one MaelysPlayground instance (see WASM bindings — Domain definition) — the native domain registry is process-wide. Every Engine instance in the same Python process shares it. Creating a second Engine() does not give you a second, isolated domain space — unlike WASM, where a new MaelysPlayground.create() gets its own WASM module and its own linear memory, so the same domain name can be declared again independently:

from maelys_datalog import Engine, Predicate, PRED_EDB, PRED_IDB, PRED_QUERY
 
engine1 = Engine()
engine1.register_domain('document_access', [        # writes into the process-wide registry
    Predicate('owns',  2, PRED_EDB),
    Predicate('allow', 2, PRED_IDB | PRED_QUERY),
])
 
engine2 = Engine()                                  # same process, same registry
engine2.register_domain('document_access', [        # 'document_access' already exists:
    Predicate('owns',  2, PRED_EDB),                # a safe no-op when the schema is
    Predicate('allow', 2, PRED_IDB | PRED_QUERY),   # identical — nothing new is created
])

register_domain declares predicates — their name, arity, and kind — but not the string atoms that appear as arguments inside rules. So a rule that matches on a literal string, such as classification(Document, "confidential"), depends on the atom "confidential" already being interned by a domain that knows it; register_domain alone does not make that atom available, and the rule can silently fail to match. For low-cardinality categories, prefer a boolean predicate over a string literal — write is_confidential(Document) rather than classification(Document, "confidential").

In practice this rarely bites, because the two common cases are already fine:

# Case 1: re-running the same cell — by far the most common notebook
# workflow. Safe no-op, nothing to worry about.
engine.register_domain('document_access', [
    Predicate('owns', 2, PRED_EDB),
    Predicate('allow', 2, PRED_IDB | PRED_QUERY),
])
engine.register_domain('document_access', [
    Predicate('owns', 2, PRED_EDB),
    Predicate('allow', 2, PRED_IDB | PRED_QUERY),
])
# no error — identical table, safe no-op
 
# Case 2: changing the schema while iterating — raises, but the fix is
# the same "Restart Kernel" you would already reach for after changing
# imports or clearing corrupted variable state.
engine.register_domain('document_access', [
    Predicate('owns', 2, PRED_EDB),
    Predicate('allow', 3, PRED_IDB | PRED_QUERY),   # arity changed 2 -> 3
])
# DomainAlreadyRegisteredError — restart the kernel, then re-run

The one real friction case, narrower than either of these: a long-running kernel or service process, never restarted, where the same domain name is re-registered with a different schema several times over its lifetime without a restart in between. That is the scenario this cap is worth planning around — not ordinary notebook re-execution.

Ruleset loading

ruleset = engine.load_inline_ruleset(
    'document_access', 'doc.main',
    'allow(User, Document) :- owns(User, Document).\n')

load_inline_ruleset(domain, ruleset_id, source) compiles the Datalog rules in source — the .dl program text, e.g. 'allow(User, Document) :- owns(User, Document).' — against the predicates declared for domain, and returns a Ruleset. Raises MaelysDatalogError (see Diagnostics) on a parse or validation failure — no lengthBytesUTF8 bookkeeping to get right, Python strings already carry their own length. The source accepts the full policy source language, including the bounded or disjunction in rule bodies.

EDB: adding facts

The binding exports four type aliases that describe the public term boundary:

from maelys_datalog import Fact, InputTerm, RawFact, ResolvedTerm
 
# InputTerm    = str | int | bool | Term
# ResolvedTerm = str | int | bool
# Fact         = tuple[ResolvedTerm, ...]
# RawFact      = tuple[Term, ...]

ruleset.edb() returns a fresh, independent EDB object, and Edb.add_fact accepts a Sequence[InputTerm]. EDB facts take Python values directly — add_fact converts str, int, and bool for you. Use Term explicitly only when you need to disambiguate (for example, forcing a small integer to be treated as a pre-interned symbol ID rather than a fresh string to intern — see Inserting by pre-interned symbol ID below):

edb = ruleset.edb()
edb.add_fact('owns', ['alice', 'roadmap.pdf'])    # str, str -> both interned as symbols
edb.add_fact('quota_exceeded', ['alice', 1024])   # str + int
edb.add_fact('has_flag', ['alice', True])         # bool

An add_fact call can fail in two very different ways, from two different layers — a wrongly shaped call caught in Python, or a well-shaped call the engine rejects. They raise different exception types, so it is worth keeping them apart.

Shape errors — Python TypeError / ValueError

Before any native call, the wrapper checks the shape of the arguments on the Python side and raises an ordinary built-in Python exception when it is wrong:

max_arity here is the build profile's global ceiling, not each predicate's declared arity — the Python check only rejects an impossible arity early. A shape error almost always means a bug in the calling code, so you normally fix the call rather than catch these at runtime.

Engine errors — MaelysDatalogError

Once the call is well-shaped, the engine itself can still reject it. That surfaces as a MaelysDatalogError — the binding's single exception type, and a subclass of RuntimeError — carrying .code (an int), .message, and .hint (see Diagnostics). Two conditions both raise .code == ERR_INVALID_FIELD:

• an explicit Term.symbol_id(id) that is not in this ruleset's symbol table — the id is not a valid field value;
• an arity within the build limit that does not match the predicate's declared arity — predicate lookup matches on (name, arity) together, so a wrong arity is not a registered predicate.

It can also raise ERR_FORBIDDEN (the predicate is a POLICY_FACT kind) or ERR_PAYLOAD_TOO_LARGE (fact caps). To act on a specific code, compare .code:

from maelys_datalog import C, MaelysDatalogError
 
try:
    edb.add_fact('owns', [Term.symbol_id(bad_id), Term.symbol_id(doc)])
except MaelysDatalogError as exc:
    if exc.code == C.ERR_INVALID_FIELD:
        ...   # unknown symbol id, or a wrong (predicate, arity) pair

exc.code holds a maelys_result_t return code — the MAELYS_ERR_* values in the Error codes table (MAELYS_ERR_INVALID_FIELD is -2). Compare it against the matching constant from C, a namespace of engine constants the package exports (from maelys_datalog import C). Note the asymmetry: the predicate-kind flags have plain top-level names — you import PRED_EDB directly — but the error codes live only on C, under names with the MAELYS_ prefix dropped, so it is C.ERR_INVALID_FIELD (there is no top-level ERR_INVALID_FIELD to import).

Catching both

The two kinds come from different layers and share no common base below Exception: MaelysDatalogError is a RuntimeError, while TypeError and ValueError are not. So a single except MaelysDatalogError clause does not catch a shape error. To guard both at once, give each its own clause:

try:
    edb.add_fact(predicate, terms)
except (TypeError, ValueError) as exc:
    # caller passed the wrong shape (bad predicate/arity/terms type) —
    # normally a bug in your code to fix, not a runtime condition
    raise
except MaelysDatalogError as exc:
    # the engine rejected a well-shaped call — inspect the code
    if exc.code == C.ERR_INVALID_FIELD:
        ...   # unknown symbol id, or a wrong (predicate, arity) pair

In practice the first clause is a guard rail (shape bugs surface loudly during development), and the second is where the real runtime conditions land.

Inserting by pre-interned symbol ID

add_fact with a str value is ergonomic sugar: internally it calls ruleset.intern_symbol(value) and wraps the result in Term.symbol_id(...) for you, on every call. Use the ID path directly when the same symbol is inserted many times (dense graphs, repeated nodes): intern once, reuse the integer handle, and skip the redundant intern lookup on every fact.

from maelys_datalog import Term
 
alice = ruleset.intern_symbol('alice')      # -> positive int handle
doc   = ruleset.intern_symbol('roadmap.pdf')
 
edb = ruleset.edb()
edb.add_fact('owns', [Term.symbol_id(alice), Term.symbol_id(doc)])

intern_symbol is idempotent — interning 'alice' twice returns the same id — and, unlike WASM's internRuntimeSymbol, is not scoped to one EDB session: it reads and writes the ruleset's own symbol table directly, so the same handle stays valid across every Edb created from that Ruleset, including ones created after the intern call.

Batch insertion

There is no batch equivalent of WASM's addSymbolIdFacts/ addRuntimeSymbolFacts in this binding — add_fact inserts one fact per call, always. WASM's batch methods exist specifically to avoid paying a JS/WASM boundary crossing per fact; a native Python call through cffi is a single in-process function call, which is much cheaper than that boundary crossing — but not free. A cffi call per fact is still measurable overhead at high volume (tens of thousands of facts per evaluation), so this is a deliberately simple first API, not a claim that per-fact calls are costless. If profiling shows meaningful time spent crossing the Python/C boundary rather than solving, a shim-level batch export (looping over an array of maelys_py_term_t rows in C) would be a bounded addition, unchanged at the engine level — not something built ahead of a measured need.

Solve

edb = ruleset.edb()
edb.add_fact('owns', ['alice', 'roadmap.pdf'])
result = ruleset.solve(edb)
 
result.enumerate_predicate_facts('allow', 2)   # [('alice', 'roadmap.pdf')]

Unlike WASM's edbBegin(), which resets a session inside one MaelysPlayground instance, ruleset.edb() returns a new, independent object every time — nothing to reset, no session to carry state across by accident. ruleset.solve(edb) finalizes edb exactly once before calling the native solver; after a successful solve, edb is closed for mutation — a later add_fact() raises RuntimeError before reaching C. Calling solve() again on the same, already-finalized edb is allowed and reuses the finalized snapshot; it does not require adding facts again.

Enumerating results (resolved vs raw)

SolveResult.enumerate_predicate_facts resolves symbol terms to strings automatically — each resolution is one native cffi call, cheap but not free. Each returned tuple mixes resolved strings, native int, and bool, one Python object per term, ready to use without a decode step.

The WASM binding can enumerate the same derived IDB facts with enumeratePredicateFacts(predicate, arity), but its return shape is designed for the JavaScript boundary: discriminated term objects, raw symbolId handles, and a separate symbolText(symbolId) lookup. It does not provide Python's resolved-tuple convenience form.

enumerate_predicate_facts_raw returns the same facts without that resolution step. The difference is in what each tuple holds:

resolved: list[Fact] = result.enumerate_predicate_facts('allow', 2)
# [('alice', 'roadmap.pdf')]                   -> plain str / int / bool, ready to use
 
raw: list[RawFact] = result.enumerate_predicate_facts_raw('allow', 2)
# [(Term.symbol_id(7), Term.symbol_id(12))]    -> opaque, ruleset-local integer handles

Both methods enumerate the same already-derived facts in the same order; they differ only in the final conversion. The resolved form is what you usually want. Reach for the raw form to skip the per-term resolution cost, or to compare and group facts by identity — but those integer handles belong to the originating Ruleset: they are neither stable across rulesets nor portable identifiers.

Ground query vs enumeration

A ground query asks one yes/no question — "is allow("alice", "roadmap.pdf") true?" SolveResult.contains_fact(predicate, terms) answers it directly, the same capability C exposes as maelys_datalog_query_solved_ground_fact and WASM as querySymbol:

result.contains_fact('allow', ['alice', 'roadmap.pdf'])   # -> True or False

terms is a Sequence[InputTerm]. String terms are resolved read-only under the owning Ruleset lock — an unknown string returns False without interning or mutating the symbol table. The predicate is validated before any term is resolved, so an invalid predicate raises rather than masquerading as a False:

• a forbidden, unknown, wrong-arity, or non-PRED_QUERY predicate raises MaelysDatalogError (even when a string term is also unknown);
• an explicit Term.symbol_id(n) not valid in this ruleset raises MaelysDatalogError with .code == ERR_INVALID_FIELD — distinct from an unknown string, which is a plain False;
• a non-string predicate, a top-level str / bytes instead of a term sequence, or an unsupported term value raises TypeError.

Scope differs from enumeration — the two are not symmetric:

• contains_fact() searches query-capable policy facts, the finalized EDB snapshot, and derived IDB facts;
• enumerate_predicate_facts() (and its raw variant) enumerate derived IDB facts only.

So a query-capable policy or EDB fact can make contains_fact() return True while enumerate_predicate_facts() returns no matching row. Neither API derives new facts.

contains_fact is a ground check — every term must be a concrete value. For a pattern query with a variable ("all X such that allow(X, "roadmap.pdf")"), enumerate the predicate and filter on the Python side:

facts = result.enumerate_predicate_facts('allow', 2)
users = [user for user, doc in facts if doc == 'roadmap.pdf']

Diagnostics

from maelys_datalog import MaelysDatalogError
 
try:
    ruleset = engine.load_inline_ruleset('document_access', 'doc.main', bad_source)
except MaelysDatalogError as exc:
    exc.code      # int — last error code
    exc.message   # str — last error message
    exc.hint      # str — corrective hint, empty string if none

hint mirrors the C-level hint field of maelys_datalog_diagnostic_t — see Errors — Diagnostic codes. Unlike WASM's pg.diagMessage/diagHint/diagCode (read separately, after the fact, from mutable properties on the instance), Python raises the diagnostic directly as the exception itself — nothing to read afterwards, no stale value left over from a previous call to accidentally reuse.

Derived fact count

result = ruleset.solve(edb)
result.derived_fact_count()   # int — total IDB facts derived by this solve

Same meaning as WASM's derivedFactCount() — a property of one solve, not a session constant. A SolveResult corresponds to exactly one solve() call, so there is no "current EDB session" to read it against; call it on the SolveResult returned by the solve() you care about.

Complete example

from maelys_datalog import Engine, Predicate, PRED_EDB, PRED_IDB, PRED_QUERY
 
with Engine() as engine:
    engine.register_domain('document_access', [
        Predicate('owns',  2, PRED_EDB),
        Predicate('allow', 2, PRED_IDB | PRED_QUERY),
    ])
 
    ruleset = engine.load_inline_ruleset(
        'document_access', 'doc.main',
        'allow(User, Document) :- owns(User, Document).\n')
 
    edb = ruleset.edb()
    edb.add_fact('owns', ['alice', 'roadmap.pdf'])
    result = ruleset.solve(edb)
 
    print(result.enumerate_predicate_facts('allow', 2))
    # [('alice', 'roadmap.pdf')]
 
    # Next evaluation — same ruleset, fresh EDB
    edb2 = ruleset.edb()
    edb2.add_fact('owns', ['bob', 'roadmap.pdf'])
    result2 = ruleset.solve(edb2)
    print(result2.enumerate_predicate_facts('allow', 2))
    # [('bob', 'roadmap.pdf')]

with Engine() closes the engine — and every ruleset, EDB, and solve result still open underneath it — on exit.

Threading model

cffi releases the GIL during every lib.foo(...) call — confirmed against the official cffi documentation, not assumed. That means two Python threads can genuinely be inside the C shim at the same time, which is exactly why register_domain and load_inline_ruleset take a process-wide Python lock around the native domain registry: that registry is a single, unsynchronized C array shared by the whole process, so without the lock two threads registering domains concurrently could corrupt it.

Symbol-table access uses a separate threading.RLock per Ruleset. The wrapper serializes intern_symbol, symbol_text, the complete Edb.add_fact operation, and the symbol-resolution loop of enumerate_predicate_facts. Two distinct Edb objects belonging to the same Ruleset therefore cannot run those symbol-table paths simultaneously. Different rulesets keep independent locks; this is not a process-wide lock.

enumerate_predicate_facts_raw does not acquire the symbol lock: it copies already-derived term kinds and values without reading the ruleset's symbol table.

These locks do not make Ruleset, Edb, or SolveResult generally thread-safe. Concurrent solves on one shared Ruleset are not a documented guarantee. Neither are sharing one Edb or SolveResult between threads, nor calling close() while another thread uses that object or one of its children. The locks establish a narrow wrapper guarantee around symbol interning/resolution and add_fact; they do not establish general thread-safety for the C engine.

One pattern that avoids the question rather than managing it: give each thread — or each request — its own Ruleset (load_inline_ruleset can be called repeatedly against the same already-registered domain, and each resulting Ruleset has its own, unshared symbol table).

API reference

Verification scope