Helping businesses grow, scale and digitize.
Language AI & NLP
Language technology should work for more than the world’s highest-resource languages. We build language resources, NLP infrastructure and evaluation for regional and under-resourced languages, starting with Dhatki, Sindhi, Urdu, Punjabi and Saraiki.
sndurdpnbskrmkiISO 639-3 codes shown for reference
Why it matters
Millions of people in Pakistan speak and write languages that general-purpose AI handles poorly. The problems are practical, and they start long before the model.
Web text is scarce, noisy and often mixed with other languages. Useful corpora need collection, cleaning and normalization first.
Tokenizers trained mostly on English split words in Perso-Arabic scripts into many fragments, which raises cost and lowers quality.
Without local test sets, nobody can say how well a model really works for a Sindhi or Saraiki speaker.
How we work
Each stage can be a project on its own, or part of an end-to-end engagement.
Where each language stands
Readiness differs a lot between languages. This is where our own work stands, so you know what a project would start from.
| Language | Status | What exists from our work | Typical project work |
|---|---|---|---|
| Sindhi | Most developed | An open-source Python NLP library, a cleaned web text corpus, a trained tokenizer and compact language-model research. | Corpus extension, annotation, fine-tuning, evaluation, tools and chatbots. |
| Urdu | Research and annotation | Urdu and Urdu-English annotation research, corpus auditing and classification experiments. | Annotation, evaluation, classification and multilingual processing. |
| Punjabi | Early resources | Corpus and treebank auditing in collaborative research work. | Data collection, annotation and evaluation. |
| Saraiki | Early resources | Corpus and treebank auditing in collaborative research work. | Data collection, annotation and evaluation. |
| Dhatki | Groundwork needed | No public resources from us yet. | Text collection, writing-convention and normalization groundwork, first datasets. |
Status as of 2026. We update this as resources are released.
Dataset categories
We collect, clean, annotate and validate datasets in these categories, scoped to your language, domain and model.
Large, cleaned and deduplicated text collections for training language models and tokenizers.
Prompt and response pairs in ChatML or similar formats for fine-tuning assistants in a local language.
Small, carefully checked test sets that show how a model really performs for local users.
Labelled news and social text for opinion, sentiment and discourse analysis.
Entity, morphology and part-of-speech labels prepared in annotation tools such as Doccano.
Aligned resources across related Perso-Arabic languages for transfer learning.
Open research
Built in the open by our founder, Aakash Meghwar, and published on Hugging Face. These are the foundations our client work starts from.
sindhi-corpus-cleanCleaned Sindhi corpus, about 529K documents, filtered by Arabic-script ratio with NFKC normalization.
sindhi-corpus-langid-cleanLanguage-ID filtered variant, about 492K documents.
sindhi-corpus-505mSource compilation of about 742K documents (~505M tokens) from 11 public Sindhi datasets: news, literature, legal, religious and encyclopedic text.
sindhi-tokenized-505mPre-tokenized version of the compiled corpus for faster training.
Sindhi-Intelligence-Core-SFT361,225 instruction rows in ChatML format covering grammar, factual knowledge and reasoning.
Sindhi-Intelligence-Core-SFT-v2A refined instruction set of about 46K rows.
Sindhi-Factual-QAA small factual question-answer set for checking model answers.
aurat-march-processed-newsAbout 5K processed news items for discourse and sentiment research.
english-aurat-march-newsEnglish-language news coverage set of 390 items.
SindhiNLTKSindhi NLP toolkit: tokenizer, stemmer, stopwords and sentiment.
SindhiLM-Tokenizer v1, v2, v3Morphology-aware Unigram and BPE tokenizers for Sindhi.
SindhiLMGPT-2 based Sindhi text generation model.
SindhiLM-Qwen-0.5B-v2Qwen2.5 0.5B fine-tuned for Sindhi instructions.
SindhiFormer-ValidationArchitecture validation for morpheme-boundary-aware attention.
IndoAryan-Shared-ManifoldZero-shot transfer and embedding alignment across Urdu, Punjabi, Sindhi, Saraiki and Hindko.
aurat-march-sentiment-minilmMultilingual MiniLM fine-tuned on Urdu news for sentiment.
Who it is for
Teams that need to know how their model performs in regional languages before launch.
Groups adding Sindhi, Urdu, Punjabi, Saraiki or Dhatki to an existing product.
Labs that need corpora, annotation or evaluation support for low-resource work.
Organizations that need language datasets built and checked by native speakers.
Questions
No. Sindhi has the most existing resources from our work. For Urdu, Punjabi, Saraiki and Dhatki, most projects include data collection or annotation. We will tell you plainly what exists and what would need to be built.
Yes, on a project basis. We design test sets around your use case, run the evaluation and explain where the model succeeds and where it fails.
Yes. Much of our work started in research, and we are glad to support academic projects with data, annotation and evaluation.
Data handling, confidentiality and ownership are agreed in writing for each project before any data is shared.
Get started
Tell us the language, the task and what you have today. We will suggest a practical first step.