AI Engineer

Beas%20consultancy%20and%20services%20pvt%20ltd; Kolkata District; On-site; INR 1,200,000 - 2,400,000; Full time

Job summary

Beas Consultancy and Services Pvt Ltd in Kolkata invites a Senior AI/NLP Engineer to advance multimodal LLMs for document understanding. You will fine-tune models, design prompts, and ensure JSON-structured outputs.

You will optimize inference with quantization and LoRA adapters, integrate OCR tools, and collaborate with backend teams to deploy scalable pipelines on GPUs.

Qualifications

• 5+ years in ML/NLP with multimodal models.
• Strong Python, PyTorch, Transformers (Hugging Face).
• Experience with OCR tools (PaddleOCR/Tesseract/EasyOCR).

Responsibilities

• Fine-tune multimodal LLMs for domain-specific document understanding.
• Design prompts and instruction templates to improve JSON outputs.
• Fine-tune across datasets to ensure generalization.
• Implement evaluation metrics and validation datasets.
• Optimize inference with quantization, LoRA adapters, Ray Serve/vLLM/Unsloth.
• Collaborate with backend teams to integrate models into production pipelines.
• Build monitoring for model confidence, token usage, and latency.

Skills

Python

PyTorch

Transformers

HuggingFace

OCR tools

JSON schema

LoRA/PEFT

Docker

CUDA

vLLM/Ray Serve

Tools

PaddleOCR

Tesseract

EasyOCR

Unsloth

Ray Serve

vLLM

CUDA

Docker

Job description

• Years Of Experience: 5+ Years
• Location: Kolkata
• Level: Senior
• Duration: 4 Months
• Vacancy: 1
• Type: Full Time

Key Responsibilities

• Fine-tune and adapt multimodal LLMs (e.g., Qwen-VL, LLaVA, or similar) for domain-specific document understanding.
• Design prompt templates and instruction sets to improve JSON-structured output quality.
• Perform different fine-tuning and incremental learning across multiple datasets to ensure generalization.
• Implement evaluation metrics and validation datasets to track model accuracy and performance.
• Optimize inference performance using quantization, LoRA adapters, and Ray Serve / vLLM / Unsloth frameworks.
• Collaborate with backend teams to integrate the model into production pipelines.
• Build monitoring tools to log model confidence, token usage, and inference latency

Required Skills

• Strong Python programming skills with experience in PyTorch and Transformers (Hugging Face).
• Experience working with OCR tools such as PaddleOCR, Tesseract, or EasyOCR.
• Hands-on experience fine-tuning or serving LLMs / VLMs (e.g., Qwen, LLaVA, Mistral, or Vicuna).
• Knowledge of LoRA / QLoRA / PEFT adapters for efficient model training.
• Familiarity with JSON schema generation, prompt engineering, and structured data extraction.
• Experience using Unsloth, Ray Serve, or vLLM for large-scale inference.
• Comfortable working with Docker, CUDA, and NVIDIA GPUs for model deployment.
• Strong understanding of tokenization, attention mechanisms, and quantization (4-bit / 8-bit )