Testing Guide

The core repository tests the C++ core, WASM and Python bindings, native CLI, MCP oracle, examples, and cross-binding parity. The Go binding is maintained and tested in its own repository.

Test Architecture

Layer	Framework	Location	Description
C++ Unit/Integration	Google Test	tests/**/*.cpp	Core library, dictionary, grammar, normalization
Data-Driven	JSON + Google Test	tests/data/tokenization/*.json	Tokenization correctness (auto-discovered)
WASM	Vitest	bindings/wasm/tests/	JS/C API, memory layout, generated ABI compatibility
Python	pytest	bindings/python/tests/	Analyze/tags API, errors, ABI layout
CLI	Built-in	test / test benchmark	Single/batch tests and benchmarks
Go	Go test	go-suzume	cgo API, ownership, dictionaries, and concurrency

Running Tests

C++ Tests

# Build dictionaries and run the native tests
make native-test

# Or manually:
cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build --parallel
cmake --build build --target build-dict   # Required: build dictionaries first
ctest --test-dir build --output-on-failure

Run specific tests by name pattern:

ctest --test-dir build -R "ConjugationTest" --output-on-failure
ctest --test-dir build -R "UserDict" --verbose

WASM Tests

# Build WASM and run tests
make wasm-test

# Or run tests only (if WASM is already built)
(cd bindings/wasm && yarn test)
(cd bindings/wasm && yarn test:watch)      # Watch mode
(cd bindings/wasm && yarn test:coverage)   # With coverage report

Python Tests

make python-test
# Run pytest only after the Rye environment is provisioned:
(cd bindings/python && rye run pytest -q)

make python-test also builds the binding and dictionaries, then runs Ruff and mypy.

Go Tests

Run the binding tests from the separate go-suzume repository:

make test
make test-race

The first command builds the native library and runs go test ./... -count=1; the second runs the race-detector suite.

CLI Test Command

# Test single input
suzume-cli test "東京スカイツリー" --expect "東京,スカイツリー"

# Run tests from file
suzume-cli test -f tests.tsv

# With user dictionary
suzume-cli test -f tests.tsv -d user.dic

Adding Tests

Data-Driven Tokenization Tests (Recommended)

Expectations in tests/data/tokenization/*.json are generated from the reference-analyzer normalization pipeline and auto-discovered by universal_tokenization_test.cpp.

Do not edit generated fixtures directly

Both tests/data/tokenization/*.json and data/**/*.tsv are tooling-managed; repository hooks block direct writes. Do not change an expected token merely to match current Suzume output. Fix the analyzer or the normalization rule, then regenerate through the tooling.

With the repository MCP server configured, the standard workflow is:

test_show(input_text="問題文")
# Fix and rebuild the analyzer or normalization rule.
test_show(input_text="問題文")
test_add(input_text="問題文", file="verb_example.json")

After changing normalization rules under scripts/mcp/src/suzume_mcp/core/, synchronize affected expectations with test_needs_suzume_update(apply=True). See the repository CONTRIBUTING.md and AGENTS.md for the current workflow and tool reference.

POS labels in fixtures

The pos values in these JSON fixtures use a Title-case reference taxonomy (Noun, Particle, …). This is a separate label set from the runtime Morpheme.pos values returned by the library, which are UPPERCASE English (NOUN, PARTICLE, …). Don't assume the two match verbatim.

Existing Test Files

The suite grows over time and is organized by linguistic category. A representative selection:

Category	Description
basic.json	Basic tokenization, single words
adjective*.json	i-adjectives, na-adjectives, compounds
verb*.json	Ichidan, godan, suru, passive, causative
particle*.json	Case, topic, binding particles
usecase_*.json	Real-world texts: news, business, casual
pattern_*.json	Linguistic patterns

C++ Unit Tests

For testing internal modules directly:

1. Create tests/category/new_test.cpp
2. Add to TEST_SOURCES in tests/CMakeLists.txt

#include <gtest/gtest.h>
#include "module_header.h"

TEST(ModuleTest, SpecificBehavior) {
    // Arrange
    auto input = ...;

    // Act
    auto result = module.process(input);

    // Assert
    EXPECT_EQ(result.field, expected_value);
}

WASM Tests

For testing the WebAssembly bindings:

Create bindings/wasm/tests/feature.test.ts:

import { afterAll, beforeAll, describe, expect, it } from 'vitest'
import { Suzume } from '../dist/index.js'

describe('Feature', () => {
  let suzume: Suzume

  beforeAll(async () => {
    suzume = await Suzume.create()
  })

  afterAll(() => {
    suzume.destroy()
  })

  it('should behave correctly', () => {
    const morphemes = suzume.analyze('テスト')
    expect(morphemes[0].surface).toBe('テスト')
  })
})

CLI Test Files (TSV)

For batch testing via the CLI:

# Comments start with #
東京スカイツリーに行きました	東京,スカイツリー,行く
美しい花が咲いている	美しい,咲く

Run with:

suzume-cli test -f tests.tsv

Output shows per-test results and a summary with pass/fail counts.

Benchmarks

The CLI includes a built-in benchmark command:

# Built-in test texts with the default measurement settings
suzume-cli test benchmark

# Control steady iterations, statistical samples, and warmup
suzume-cli test benchmark --iterations=5 --samples=3 --warmup=2

# With custom corpus
suzume-cli test benchmark -f corpus.txt

Metrics reported include initialization time, first-analysis latency, median steady-state latency, byte throughput, per-text latency, and peak RSS. Multiple samples make the steady-state median less sensitive to one-off noise.

Debug Builds

With Sanitizers

# AddressSanitizer
cmake -B build-asan -DCMAKE_BUILD_TYPE=Debug -DENABLE_SANITIZER=ON -DENABLE_ASAN=ON
cmake --build build-asan && ctest --test-dir build-asan

# UndefinedBehaviorSanitizer
cmake -B build-ubsan -DCMAKE_BUILD_TYPE=Debug -DENABLE_SANITIZER=ON -DENABLE_UBSAN=ON
cmake --build build-ubsan && ctest --test-dir build-ubsan

# ThreadSanitizer
cmake -B build-tsan -DCMAKE_BUILD_TYPE=Debug -DENABLE_SANITIZER=ON -DENABLE_TSAN=ON
cmake --build build-tsan && ctest --test-dir build-tsan

make asan is the standard aggregate for AddressSanitizer, LeakSanitizer, and UndefinedBehaviorSanitizer.

With Coverage

cmake -B build -DCMAKE_BUILD_TYPE=Debug -DENABLE_COVERAGE=ON
cmake --build build
ctest --test-dir build
# Coverage files generated in build/

CI

For changes outside the documentation-only ignore paths, GitHub Actions runs on pushes to main and develop, and on pull requests targeting main. The workflow defines seven jobs; python-binding is skipped only for a direct push to develop.

Job	What it checks
lint	WASM lint plus Ruff checks for the MCP server and repository Python scripts
guardrails	Generated-file, mirrored-enum, oracle, compound, and version-consistency checks
mcp-tests	MCP/oracle tests against MeCab + IPADIC and golden-expectation synchronization
sanitizers	Native ASan, LSan, and UBSan checks
manylinux-toolchain	Compile the shared core with the oldest supported manylinux toolchain
build-and-test	Native tests and examples with coverage, WASM build/size/tests, and binding parity
python-binding	ruff check / ruff format --check, mypy, and pytest for the Python binding

make consumer-smoke runs in build-and-test. make format-check (including C++ clang-format) remains a local verification target and is not wired into CI.

Makefile Targets

Target	Description
make test	Run native, MCP, Python, WASM, example, consumer, parity, and guardrail checks
make build	Build the project
make dict	Build the project, then compile dictionaries
make wasm-test	Build WASM + run WASM tests
make python-test	Build and test the Python binding, including lint/type checks
make examples	Build the in-tree C and C++ examples
make consumer-smoke	Test an installed package through find_package
make version-check	Verify versions across binding manifests
make format	Format/lint C++, MCP, WASM, and Python sources
make format-check	Check formatting across all languages