hezar

Hezar (meaning thousand in Persian) is a multipurpose AI library built to make AI easy for the Persian community!

Hezar is a library that:

• brings together all the best works in AI for Persian
• makes using AI models as easy as a couple of lines of code
• seamlessly integrates with Hugging Face Hub for all of its models
• has a highly developer-friendly interface
• has a task-based model interface which is more convenient for general users.
• is packed with additional tools like word embeddings, tokenizers, feature extractors, etc.
• comes with a lot of supplementary ML tools for deployment, benchmarking, optimization, etc.
• and more!

Installation

Hezar is available on PyPI and can be installed with pip (Python 3.10 and later):

pip install hezar

Note that Hezar is a collection of models and tools, hence having different installation variants:

pip install hezar[all]  # For a full installation
pip install hezar[nlp]  # For NLP
pip install hezar[vision]  # For computer vision models
pip install hezar[audio]  # For audio and speech
pip install hezar[embeddings]  # For word embedding models

You can also install the latest version from the source:

git clone https://github.com/hezarai/hezar.git
pip install ./hezar

Documentation

Explore Hezar to learn more on the docs page or explore the key concepts:

• Getting Started
• Quick Tour
• Tutorials
• Developer Guides
• Contribution
• Reference API

Quick Tour

Models

There's a bunch of ready to use trained models for different tasks on the Hub!

🤗Hugging Face Hub Page: https://huggingface.co/hezarai

Let's walk you through some examples!

• Text Classification (sentiment analysis, categorization, etc)

from hezar.models import Model

example = ["هزار، کتابخانه‌ای کامل برای به کارگیری آسان هوش مصنوعی"]
model = Model.load("hezarai/bert-fa-sentiment-dksf")
outputs = model.predict(example)
print(outputs)

[[{'label': 'positive', 'score': 0.812910258769989}]]

• Sequence Labeling (POS, NER, etc.)

from hezar.models import Model

pos_model = Model.load("hezarai/bert-fa-pos-lscp-500k")  # Part-of-speech
ner_model = Model.load("hezarai/bert-fa-ner-arman")  # Named entity recognition
inputs = ["شرکت هوش مصنوعی هزار"]
pos_outputs = pos_model.predict(inputs)
ner_outputs = ner_model.predict(inputs)
print(f"POS: {pos_outputs}")
print(f"NER: {ner_outputs}")

POS: [[{'token': 'شرکت', 'label': 'Ne'}, {'token': 'هوش', 'label': 'Ne'}, {'token': 'مصنوعی', 'label': 'AJe'}, {'token': 'هزار', 'label': 'NUM'}]]
NER: [[{'token': 'شرکت', 'label': 'B-org'}, {'token': 'هوش', 'label': 'I-org'}, {'token': 'مصنوعی', 'label': 'I-org'}, {'token': 'هزار', 'label': 'I-org'}]]

• Mask Filling

from hezar.models import Model

model = Model.load("hezarai/roberta-fa-mask-filling")
inputs = ["سلام بچه ها حالتون