Bharat-Tiny-LLM v3 vs. LLaMA 3 8B: Indic & Hindi Benchmark
Devanagari Token Fertility, Edge CPU Latency, and GSM8K Hindi Math Reasoning. Direct empirical comparison between Eulogik’s Bharat-Tiny-LLM v3 (1.7B) powered by the Brahmi Tokenizer and Meta’s LLaMA 3 8B on Hindi NLP, token compression, and CPU runtime execution.
EXECUTIVE SUMMARY & EMPIRICAL VERDICT
For Indic and Hindi enterprise tasks on local hardware, Bharat-Tiny-LLM v3 delivers 58% fewer tokens per sentence, 5.9× lower memory usage, and zero cloud API fees compared to deploying LLaMA 3 8B.
EULOGIK EDGE MODELFeatured
Bharat-Tiny-LLM
v3 (Qwen3-1.7B Base) · 1.7 Billion
Target HardwareCommodity CPU / Apple Silicon / WebGPU
Latency / Speed45 tokens/sec (Apple MLX) · 18 tok/s (x86 CPU)
RAM / Memory880 MB (4-bit quant)
Fertility / Size1.83 tokens/word (Brahmi Tokenizer)
- ✓ Proprietary Brahmi Tokenizer eliminates byte fallback
- ✓ Runs 100% offline inside web browsers via WebGPU
- ✓ GSM8K Hindi math reasoning tuned for school curriculums
- ✓ Zero cloud dependency; air-gapped DPDP compliant
INDUSTRY BASELINEOpen Weights
LLaMA 3
8B Instruct · 8.0 Billion
Target HardwareHigh-end GPU (RTX 4090 / A10G) required for SLA
Latency / Speed8–12 tokens/sec (CPU) · 55 tok/s (GPU)
RAM / Memory5.2 GB (4-bit quant)
Fertility / Size4.33 tokens/word (Standard Tiktoken BPE)
- • Broad general English capability
- • Heavy byte-level fragmentation on Devanagari script
- • Cannot execute smoothly on standard CPU or mobile
- • Requires substantial enterprise cloud infrastructure
METRIC SPECIFICATION MATRIX
| METRIC | BHARAT-TINY-LLM | LLAMA 3 |
|---|---|---|
| Devanagari Token Fertility | 1.83 tok/word ★ | 4.33 tok/word |
| Byte Fragmentation on Hindi | 0 Bytes Dropped ★ | High UTF-8 splits |
| Memory Footprint (4-bit) | 880 MB ★ | 5.2 GB |
| In-Browser WebGPU Execution | Native (Offline) ★ | Unsupported (Too large) |
| Inference Speed on Mac/CPU | 45 tok/sec ★ | 8 tok/sec |
| General English MMLU | 61.4% | 68.4% ★ |
REPRODUCIBLE CODE SNIPPET
Execute this benchmark locally
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
# Load Eulogik Bharat-Tiny-LLM v3
tokenizer = AutoTokenizer.from_pretrained("eulogik/Bharat-Tiny-LLM-v3")
model = AutoModelForCausalLM.from_pretrained("eulogik/Bharat-Tiny-LLM-v3", torch_dtype=torch.float16)
prompt = "भारतीय रिज़र्व बैंक की मौद्रिक नीति समिति ने क्या निर्णय लिया?"
inputs = tokenizer(prompt, return_tensors="pt")
print(f"Token Count: {inputs.input_ids.shape[1]}") # 58% fewer tokens than LLaMA-3
outputs = model.generate(**inputs, max_new_tokens=64)
print(tokenizer.decode(outputs[0]))Interested in deploying Bharat-Tiny-LLM?
We assist enterprises with custom fine-tuning, hardware sizing, and on-premise air-gapped installation.