Skip to content

Latest commit

ย 

History

6 Commits

Folders and files

NameName
Last commit message
Last commit date
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 

Repository files navigation

๐Ÿ‡ฎ๐Ÿ‡ณ pizza-analysis-hindi

Hindi text analysis plugin for INFINI Pizza

Crate License


Overview

Hindi/Devanagari language analysis with Indic script normalization, Hindi-specific normalization, light stemming, and stop words.

Components

Type Name Description
TokenFilter indic_normalization Normalize Indic script diacritics
TokenFilter hindi_normalization Hindi-specific character equivalences
TokenFilter hindi_stem Hindi light stemmer (suffix removal)
TokenFilter hindi_stop Hindi stop words (225 entries)
Analyzer hindi Full pipeline: lowercase โ†’ indic_norm โ†’ hindi_norm โ†’ stem โ†’ stop

Normalization

  • Indic: Handles Nukta composites, visarga, and cross-script canonical equivalents
  • Hindi: Normalizes Devanagari variants (chandrabindu, anusvara equivalents)

Example

use pizza_engine::analysis::AnalysisFactory;

let mut factory = AnalysisFactory::new();
pizza_analysis_hindi::register_all(&mut factory);

let analyzer = factory.get_analyzer("hindi").unwrap();

Installation

[dependencies]
pizza-analysis-hindi = "0.1"

Or via pizza-analysis-all:

[dependencies]
pizza-analysis-all = { version = "0.1", features = ["hindi"] }

License

MIT


Part of the INFINI Pizza ecosystem

About

๐Ÿ‡ฎ๐Ÿ‡ณ Hindi text analysis plugin for INFINI Pizza

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages