Skip to content

Latest commit

 

History

History
298 lines (228 loc) · 12 KB

TESTING.md

File metadata and controls

298 lines (228 loc) · 12 KB

Testing Guide

Test Strategy

go-typeql uses a two-tier testing approach:

  • Unit tests — test all packages without a database connection. Use mock implementations of Conn and Tx interfaces.
  • Integration tests — test against a live TypeDB instance. Gated by -tags integration and require the Rust FFI library.

Running Tests

# Unit tests (738 tests, no DB required)
make test-unit
# Or directly:
go test ./ast/... ./given/... ./gotype/... ./tqlgen/...

# Integration tests (requires TypeDB + Rust library)
docker compose up -d
make test-integration
# Or directly:
TEST_DB_ADDRESS=localhost:1730 TYPEDB_GO_COMPOSE_PORT_MAP=1 go test -tags "cgo,typedb,integration" ./driver/... ./gotype/...

# Rust FFI and direct TypeQL parser tests
make test-rust

# Lint
make lint

# Benchmarks (records results in benchmarks/benchmarks.sqlite)
make bench

Test Organization

Package Test File(s) What's Tested
ast/ *_test.go AST node compilation, literal formatting
gotype/ *_test.go Tags, models, registry, schema gen, CRUD, queries, filters, migration
tqlgen/ *_test.go Schema parsing, code generation, naming, functions, structs, annotations
driver/ *_test.go Connection, transactions, queries (integration only)

Benchmark History

Benchmark runs are persisted in benchmarks/benchmarks.sqlite. The recorder appends:

  • benchmark metrics (ns/op, B/op, allocs/op)
  • git commit and dirty state
  • Go version, OS/arch, hostname, and CPU label
  • raw go test output for the run

Use make bench for an explicit performance checkpoint, or make test-all to run unit tests and then record a benchmark run. For opt-in driver decoding, bulk writes, typed reads, projections, and completed close lifecycles, see benchmark groups. go run ./cmd/benchdb -group=<name> is exploratory and does not alter the tracked SQLite history; pass -db explicitly only for a reviewed baseline.

To find where the time of one operation goes, trace the integration tests with make perf-trace. See Performance Tracing. Benchmarks do not use tracing.

When using the repo docker-compose.yml, TypeDB is exposed on host port 1730 by default even though the server listens on container port 1729. The tests default to localhost:1729, so set TEST_DB_ADDRESS=localhost:1730 when running against the compose-managed instance. Because the container advertises its internal 127.0.0.1:1729 address, also set TYPEDB_GO_COMPOSE_PORT_MAP=1 so the driver maps the dialed localhost:1730 back to the advertised address (the driver no longer rewrites localhost ports implicitly; make test-integration sets both variables by default).

Mock Patterns

Unit tests use mock implementations of the Conn and Tx interfaces defined in gotype/session.go. The mocks use a sequence-based pattern where each transaction gets preset responses:

// mockTx records queries and returns responses in sequence
type mockTx struct {
    queries   []string              // All queries executed (recorded)
    responses [][]map[string]any    // One response per query, in order
    committed bool
    err       error
}

func (m *mockTx) Query(query string) ([]map[string]any, error) {
    m.queries = append(m.queries, query)
    if m.err != nil {
        return nil, m.err
    }
    idx := len(m.queries) - 1
    if idx < len(m.responses) {
        return m.responses[idx], nil
    }
    return nil, nil
}

func (m *mockTx) Commit() error  { m.committed = true; return nil }
func (m *mockTx) Rollback() error { return nil }
func (m *mockTx) Close()          {}
func (m *mockTx) IsOpen() bool    { return true }

// mockConn returns transactions in sequence
type mockConn struct {
    txs []*mockTx // One tx per Transaction() call, in order
}

func (m *mockConn) Transaction(dbName string, txType int) (gotype.Tx, error) {
    idx := /* tracks call count */
    return m.txs[idx], nil
}

func (m *mockConn) Schema(dbName string) (string, error) { return "", nil }
func (m *mockConn) Close()                               {}
func (m *mockConn) IsOpen() bool                         { return true }

This pattern allows testing multi-query operations (like Insert which queries then fetches IID) by providing responses in the order they'll be consumed.

Registry in Tests

The global type registry is shared across tests. Each test that registers types should clear the registry first:

func TestSomething(t *testing.T) {
    gotype.ClearRegistry()
    gotype.Register[Person]()
    // ... test code
}

This prevents interference from other tests that may have registered (or cleared) different types. A helper pattern used in many tests:

func registerTestTypes(t *testing.T) {
    t.Helper()
    gotype.ClearRegistry()
    gotype.Register[testPerson]()
    gotype.Register[testCompany]()
    gotype.Register[testEmployment]()
}

TypeQL Syntax Validation

Everything this library emits is TypeQL text, so syntax bugs historically surfaced only as cryptic server errors far from the code that generated them. The test suite now validates generated TypeQL with the official typeql-check CLI from typedb/typedb-tools — the same parser the server uses.

# One-time install (into ~/go/bin, version pinned in the Makefile)
make install-typeql-check

The pinned checker and test server use 3.14.0-rc0 (release candidate). After an upgrade, run make install-typeql-check again.

How it works:

  • internal/typeqlcheck locates the binary ($TYPEQL_CHECK env override → $PATH → ~/go/bin) and exposes AssertValid(t, label, query) / Validate(query).
  • Dedicated syntax batteries pipe real generator output through the tool:
    • ast/typeql_syntax_test.go — compiler and fluent-layer output
    • gotype/typeql_syntax_test.go — CRUD queries (via mocks), GenerateSchema, migration statements
    • tqlgen/typeql_syntax_test.go — parser conformance: schemas the official grammar accepts should parse in tqlgen too
  • Soft dependency: without the binary, assertions print a one-time warning and pass, so contributors are never blocked. check.sh detects the binary and sets TYPEQL_CHECK_REQUIRED=1, which turns a missing binary into a hard failure — so on a machine with the tool installed (and in CI), the syntax gates are mandatory.
  • Known-invalid outputs (open bugs from the 2026-07 code review, e.g. #23, #24, #32) are marked with a knownIssue in the test tables: they skip while the bug exists and fail loudly once the output becomes valid, forcing the marker's removal alongside the fix.

When adding a new query generator or changing emitted TypeQL, add the output to the matching syntax battery — a typeql-check failure at unit-test time beats a [TQL03] server error at integration time.

Filter Compiler Tests

The filter compiler and its variable allocator (issue #138) have three kinds of tests in gotype:

  • compile_test.go has one test for each fault and rule of the issue.
  • compile_property_test.go compiles random filter trees. It checks that each variable is a valid TypeQL variable, that each variable is bound once, that the text is deterministic, and that typeql-check accepts the query.
  • compile_diff_test.go compares the Go compiler with the Lean model of the design (formal/lean/Scopes.lean). It compares which occurrences share a variable, the scope of each variable, and the bindings, after it normalizes both outputs. The variable names can differ.

The differential test needs the model binary. It is a soft dependency: when the binary is not built, the test skips. To build it, install Lean with elan, then run:

cd formal/lean && lake build diffmodel

When you change the compiler, run the differential test. If you change a rule of the design, change the Lean model first, then the Go code.

Test Fixtures

Common test types used across the test suite:

type testPerson struct {
    gotype.BaseEntity
    Name  string `typedb:"name,key"`
    Email string `typedb:"email,unique"`
    Age   *int   `typedb:"age"`
}

type testCompany struct {
    gotype.BaseEntity
    Name string `typedb:"name,key"`
}

type testEmployment struct {
    gotype.BaseRelation
    Employee *testPerson  `typedb:"role:employee"`
    Employer *testCompany `typedb:"role:employer"`
}

Integration Test Infrastructure

Integration tests live in gotype/integ_*_test.go files, gated by //go:build integration && cgo && typedb. They use a shared TestMain that creates/deletes a test database.

Helpers (integ_helpers_test.go)

Test helpers for unique data and common assertions:

// uniqueSuffix returns a 6-char hex string for test isolation
func uniqueSuffix() string

// makeName returns "prefix-abc123" with a unique suffix
func makeName(prefix string) string

// assertEntityExists fetches by filter and fails if not found
func assertEntityExists[T Entity](t *testing.T, mgr *Manager[T], filter Filter) T

// assertEntityCount checks that the count matches expected
func assertEntityCount[T Entity](t *testing.T, mgr *Manager[T], expected int)

Use uniqueSuffix() / makeName() to prevent test collisions when tests run against a shared database.

Per-Test Schema Setup

Each integration test registers its own types and syncs the schema. Use t.Cleanup to clear the registry:

func TestIntegration_SomeFeature(t *testing.T) {
    gotype.ClearRegistry()
    gotype.Register[myEntity]()
    // Sync schema to DB...
    t.Cleanup(func() { gotype.ClearRegistry() })
}

Python Test Parity

The Python type-bridge has 698 integration tests across 80 files. Go currently covers the core paths (~83 ported). The main unported areas are:

Area Python Tests Status
Multi-value attributes ~30 Not yet supported (slice CRUD)
Date/Duration/Decimal ~20 Go supports 5 of 9 TypeDB value types
Multi-role players ~13 Not yet supported
Constraint enforcement ~18 @key/@unique/@card/@regex/@range enforcement
Relations-as-roles ~15 Relations playing roles in other relations
Domain scenarios ~80 Bookstore, STIX, Social, Drug Discovery, IAM
Role player queries ~40 Filter by role player attributes (partially done)

Writing New Tests

  1. Clear and re-register types at the start of each test.
  2. Use mockConn/mockTx to control query results with the sequence-based pattern.
  3. Verify generated TypeQL by inspecting mockTx.queries.
  4. For schema/migration tests, use IntrospectSchemaFromString with a TypeQL string.
  5. Integration tests should use the integration build tag and clean up test databases.
  6. Use uniqueSuffix() in integration tests to avoid data collisions.

Example: Testing a Multi-Query Operation

func TestManager_Insert(t *testing.T) {
    registerTestTypes(t)

    // Insert executes 2 queries: the insert, then a key match to fetch IID
    tx := &mockTx{
        responses: [][]map[string]any{
            nil, // insert returns nothing
            {{"_iid": map[string]any{"value": "0x123"}}}, // key match returns IID
        },
    }
    mock := &mockConn{txs: []*mockTx{tx}}
    db := gotype.NewDatabase(mock, "testdb")
    mgr := gotype.MustNewManager[testPerson](db)

    p := &testPerson{Name: "Alice"}
    err := mgr.Insert(context.Background(), p)
    if err != nil {
        t.Fatal(err)
    }
    // Verify IID was set
    if p.GetIID() != "0x123" {
        t.Errorf("expected IID 0x123, got %s", p.GetIID())
    }
}