A pip-installable Python toolkit that takes text and speech to annotated output across the whole language family, and a free online book on the methods behind it.
A Stanza-style modular pipeline with seven backend families: rule-based, Apertium FSTs, Stanza, multilingual Glot500 models, NLLB-200, Omnilingual ASR, and any LLM you choose.
It handles Latin, Cyrillic, Perso-Arabic and Old Turkic runes natively. Text in a script without models is transliterated automatically and mapped back.
Natural Language Processing Across Turkic Languages: Methods and Applications explains how to build and evaluate language technology for the family. It covers scripts and corpora, finite-state and neural morphology, syntax, MT, speech, language models and cross-lingual transfer.
Each chapter pairs linguistic analysis with a practical section and exercises.
Fifteen pipeline processors with a backend switch on each. Pick what you need, chain it together, and export to CoNLL-U or JSON.
Rule-based and Stanza tokenizers for every script, with automatic script detection and transparent transliteration for scripts that lack models.
Apertium FSTs for 20 languages through Python-native HFST, plus Glot500 neural UPOS, features and lemmas for 22.
A hybrid FST + neural tokenizer that splits words into labelled morphemes, plus morpheme-aware SentencePiece for training language models.
Neural taggers and parsers for 15 languages: official and custom-trained Stanza models, and multilingual Glot500. NER for Turkish and Kazakh, and for every language through the LLM backend.
Omnilingual ASR for 20 Turkic languages, with transcripts in each language's script. Pipeline.from_audio runs speech → text → analysis.
Run morphology, NER, translation, sentiment and spelling correction on vLLM, Ollama, OpenAI, Claude or a Hugging Face model. Answers are validated, cached and logged.
One NLLB-200 model gives translation and 1024-dimensional sentence embeddings for 11 Turkic languages.
GlotLID with script-aware labels, restricted to Turkic languages by default to separate close relatives.
Cyrillic ↔ Latin ↔ Perso-Arabic, Old Turkic runes, and the Common Turkic Alphabet for 23 languages.
The family shares much of its grammar, yet data, tools and evaluations differ greatly from one language and task to the next. This book shows how to build and assess language technology with both facts in view, around three questions: how morphology affects a task, what changes when the script changes, and what transfers between related languages.
From well-resourced Turkish to endangered Siberian varieties. Labels show the trained components. The LLM backend adds morphology, NER, translation, sentiment and spelling for every language.
"Full pipeline" means tokenization, FST and neural morphology, POS, lemmas, dependency parsing, NLLB translation and embeddings, and speech recognition.
The Turkic NLP community spans researchers, practitioners and language enthusiasts around the world. Native speakers can help most by checking analyses in their language. The roadmap lists open tasks.
The toolkit and the book are free and open. If this work is useful to you for research, teaching or building products, please consider supporting its development.
Funds go directly toward model training, infrastructure and new language coverage.