AI/ML & Data Engineer

Congruent Software Inc.; Chennai District; On-site; INR 800,000 - 1,200,000; Full time

Job summary

Congruent Software Inc. is seeking candidates for an ML/NLP position in Chennai. The role involves building data pipelines for processing unstructured data and fine-tuning LLMs for tasks like summarization and classification.

Successful applicants will work with technologies like Python, Hugging Face transformers, and various ML tools. Applicants should have 4-6 years of experience in ML/NLP, specifically in document-heavy domains.

Qualifications

• 4-6 years of experience in ML/NLP, preferably in document-heavy domains.

Responsibilities

• Build and maintain data pipelines for unstructured data ingestion.
• Fine-tune LLMs for summarization and classification.
• Design entity relationships and implement retrieval with graph DBs.
• Build RAG pipelines for natural language querying.
• Annotate training data by collaborating with SMEs.
• Build CI/CD for model retraining and versioning.
• Expose ML models as REST APIs for product integration.

Skills

Python

SQL

Hugging Face transformers

OpenAI API

SpaCy

Scikit-Learn

LangChain

Pinecone

Weaviate

FAISS

Airflow

Kafka

Tesseract

pdfminer

MLflow

Docker

Job description

Experience

4~6 years in ML/NLP, preferably in document-heavy domains (finance, legal, policy).

Key Responsibilities

• Data Ingestion and Preprocessing: build and maintain data pipelines to ingest unstructured data from PDFs, gazettes, HTML circulars etc., and process data extraction, parsing, and normalization.
• NLP & LLM Modeling: fine-tune or prompt-tune LLMs for summarization, classification, and change detection in regulations; develop embeddings for semantic similarity.
• Knowledge Graph Engineering: design entity relationships (regulation, control, policy) and implement retrieval over Neo4j or similar graph DBs.
• Information Retrieval (RAG): build RAG pipelines for natural language querying of regulations.
• Annotation and Validation: annotate training data by collaborating with SMEs and validate model outputs.
• MLOps: build CI/CD for model retraining, versioning, and evaluation (precision, recall, BLEU, etc.).
• API and Integration: expose ML models as REST APIs (FastAPI) for integration with product frontend.

Skills

Languages: Python, SQL

AI/ML/NLP: Hugging Face transformers, OpenAI API, SpaCy, Scikit-Learn, LangChain, RAG, LLM prompt-tuning, LLM fine-tuning

Vector Search: Pinecone, Weaviate, FAISS

Data Engineering: Airflow, Kafka, OCR (Tesseract, pdfminer)

MLOps: MLflow, Docker