Name: Suzume
Author: libraz

Question 1

What is Suzume?

Accepted Answer

Suzume is a lightweight, feature-driven Japanese tokenizer that runs on WebAssembly. Unlike dictionary-based analyzers like MeCab, it works without large dictionary files and is robust to unknown words. It runs in browsers, Node.js, Deno, and Bun.

Question 2

How does Suzume handle unknown words?

Accepted Answer

Suzume generates candidates from character patterns (kanji sequences, katakana sequences, alphanumeric compounds) and evaluates them alongside dictionary entries using Viterbi algorithm. This makes it robust to neologisms and domain-specific terms.

Question 3

Can I use Suzume in the browser?

Accepted Answer

Yes, Suzume runs entirely in the browser via WebAssembly. No server required. You can load it from npm or directly from a CDN like esm.sh. The entire package is under 350KB gzipped.

Question 4

How do I add custom words to Suzume?

Accepted Answer

Use loadUserDictionary() to add custom words at runtime. Format: "word,pos" (e.g., "ChatGPT,noun"). You can add brand names, technical terms, or domain-specific vocabulary without rebuilding the dictionary.

Question 5

What is the difference between Suzume and MeCab?

Accepted Answer

MeCab requires large dictionary files (50MB+) and server-side installation. Suzume uses feature-based analysis with a minimal dictionary, runs on WebAssembly in the browser, and handles unknown words gracefully. Choose Suzume for client-side processing without server infrastructure.

Question 6

How is Suzume different from kuromoji.js?

Accepted Answer

kuromoji.js requires downloading a 20MB+ dictionary on first load, causing slow initial page loads. Suzume is under 350KB gzipped and loads instantly. Suzume also handles unknown words better and has a simpler API.

Question 7

Can I use Suzume for SEO keyword extraction?

Accepted Answer

Yes, Suzume can extract nouns and compound words from Japanese text, making it ideal for auto-tagging blog posts, generating hashtags, or building keyword analysis tools - all without server infrastructure.

Question 8

Is Suzume suitable for production use?

Accepted Answer

Yes, Suzume is production-ready. It is compiled from C++ to WebAssembly for near-native performance, includes full TypeScript support, and works in all modern browsers, Node.js, Deno, and Bun.

Question 9

Does Suzume work offline?

Accepted Answer

Yes, once loaded, Suzume works completely offline. All processing happens locally in the browser or runtime. No API calls or internet connection required after initial load.

Question 10

How do I install Suzume?

Accepted Answer

Install via npm: npm install @libraz/suzume. Then import and use: const { Suzume } = await import("@libraz/suzume"); const suzume = await Suzume.create(); const result = suzume.analyze("日本語テキスト");

	MeCab	Suzume
Approach	Dictionary-driven	Feature-driven
Dictionary	50MB+ (required)	Minimal (~300KB total with WASM)
Unknown words	Falls back to character types	Pattern-based candidate generation
Compound handling	Splits per dictionary	Merges by character type heuristics
Target	Server-side, academic	Browser, real-time, client-side

MeCab	Suzume
名詞,一般	NOUN
名詞,固有名詞,地域	NOUN
名詞,サ変接続	NOUN
名詞,副詞可能	NOUN
動詞,自立	VERB
動詞,非自立	VERB

Use Case	Recommendation
Browser / client-side apps	Suzume — no server required
Search indexing / tag extraction	Suzume — compound merging is often desirable
Academic research / corpus analysis	MeCab — maximum accuracy and POS detail
Real-time UI (input-as-you-type)	Suzume — fast, no network latency
Precise compound word splitting	MeCab — dictionary-driven boundaries
Handling unknown / modern words	Suzume — robust to unseen vocabulary

Differences from MeCab

Design Philosophy

Intentional Differences

1. Kanji Compound Merging

2. Katakana Compound Merging

3. Number + Unit Merging

4. Date Merging

5. Nai-Adjective Handling

6. Slang/Modern Adjective Support

Where Suzume Improves on MeCab

POS Correction

Slang and Modern Words

Particle Misclassification

Constraints

Cannot Split Merged Compounds

Context-Dependent POS Classification

POS Granularity

When to Use Which

Differences from MeCab ​

Design Philosophy ​

Intentional Differences ​

1. Kanji Compound Merging ​

2. Katakana Compound Merging ​

3. Number + Unit Merging ​

4. Date Merging ​

5. Nai-Adjective Handling ​

6. Slang/Modern Adjective Support ​

Where Suzume Improves on MeCab ​

POS Correction ​

Slang and Modern Words ​

Particle Misclassification ​

Constraints ​

Cannot Split Merged Compounds ​

Context-Dependent POS Classification ​

POS Granularity ​

When to Use Which ​