AI MLOPS LLMOps Engineer

EXL; Gurugram District; On-site; INR 1,800,000 - 4,000,000; Full time

Job summary

EXL in India is seeking a Data Engineer / AI Engineer to build and operationalize large-scale AI and NLP solutions on cloud platforms, handling multilingual unstructured text and complex JSON. You will integrate LLMs into production workflows and design scalable data pipelines.

You will work with Airflow orchestration, AWS services, vector search, and ML lifecycle tools like MLflow and Databricks, focusing on quality, monitoring, and production readiness.

Qualifications

• Hands-on experience building and deploying large-scale AI/NLP solutions on cloud platforms.
• Experience integrating AI/LLM models into production workflows.
• Strong in Python/SQL and developing scalable data pipelines.
• Familiarity with multilingual unstructured text processing and document processing systems.

Responsibilities

• Integrate and adjust NLP inference pipelines including document classification, entity extraction, de-identification, and LLM-based trend detection
• Link AI modules into end-to-end production workflows with Airflow DAGs on AWS EKS
• Design hybrid search pipelines using multilingual embeddings with GIN-based search on Aurora PostgreSQL
• Deploy and monitor models using MLflow and Databricks; manage schema migrations with Liquibase
• Scale data pipelines across AWS services (S3, Athena, Glue, Fargate, SQS, Step Functions) to handle hundreds of millions of text chunks

Skills

Python

SQL

Apache Airflow

AWS services

vector search

embeddings

MLflow

Databricks / Azure Databricks

GitHub / CI/CD

schema management

Education

Bachelor's degree in Computer Science, Information Technology, Data Science, AI, Statistics, Mathematics

Master's degree in Data Science, AI/ML, Computer Science, or Analytics (preferred)

Cloud or data engineering certifications (advantageous)

Tools

PostgreSQL

Aurora

pgvector

GIN indexes

Liquibase

OpenAI API

S3

Athena

Glue

Job description

Job Description:

Seeking a strong Data Engineer / AI Engineer with expertise in building and operationalizing large-scale AI and NLP solutions on cloud platforms. The ideal candidate should have hands‑on experience integrating AI/LLM models into production workflows, developing scalable data pipelines, and processing large volumes of multilingual unstructured text.

Key strengths should include:

• Proficiency in Python and SQL with experience deploying AI/NLP solutions such as document classification, entity extraction, NER, PII masking, de-identification, hybrid search, and LLM integrations.
• Strong knowledge of Apache Airflow for orchestrating end-to-end data pipelines and automating batch processing workflows.
• Experience working with AWS services including S3, Athena, Glue, Fargate, EKS, SQS, and Step Functions.
• Capability to design and maintain large-scale document processing systems handling complex JSON structures, embedded documents, and multilingual content.
• Familiarity with vector search and retrieval systems, including embeddings, pgvector, PostgreSQL/Aurora, GIN indexes, and full-text search.
• Experience with ML lifecycle management using MLflow, Databricks/Azure Databricks, model deployment, monitoring, and evaluation frameworks.
• Strong DevOps practices including GitHub-based development, CI/CD pipelines, schema management, and production support.

Responsibilities

What You Will Do

AI Module Integration & Inference Pipelines

• Integrate and adjust inference pipelines for NLP modules including document classification, entity extraction, de-identification (DEID), and LLM-based early trend detection
• Connect DS-coded AI modules into end-to-end production workflows via Airflow DAGs on AWS EKS
• Build and tune hybrid search pipelines combining GTE multilingual dense embeddings with GIN lexical search on Aurora PostgreSQL
• Integrate with OpenAI-based API platform for multilingual query expansion and LLM-driven trend detection

Document Processing & Parsing

• Design and maintain document preprocessing pipelines that parse deeply nested JSON structures (emails with attachments, embedded PDFs) from S3/DataLake
• Handle multilingual unstructured text (English, Spanish, Portuguese, German, Dutch, French, Italian) across 300 GB of claim notes and documents
• Build chunking strategies and metadata extraction for downstream embedding and retrieval workflows

Data Pipeline Engineering

• Author and maintain Airflow DAGs for batch processing (monthly entity refresh, trend detection, DEID pipeline)
• Manage data flow across AWS services: S3, Athena, Glue, Fargate, SQS, Step Functions
• Scale pipelines to handle 500K+ claims and hundreds of millions of text chunks

Production Deployment & Quality

• Deploy and version models using MLflow and Databricks
• Manage schema evolution and migrations using Liquibase on Aurora PostgreSQL
• Instrument pipelines with logging, monitoring, and evaluation scoring for retrieval quality

Qualifications

• Languages – Python (primary), SQL
• AI / NLP – LLM API integration, multilingual embeddings (e.g., GTE), hybrid search, text classification, entity extraction, NER, PII masking
• Data Pipelines – Apache Airflow, batch orchestration, large-scale unstructured data processing
• Cloud & Infrastructure – AWS (S3, Athena, Glue, Fargate, EKS, SQS, Step Functions)
• Databases – PostgreSQL / Aurora, pgvector, GIN indexes, full-text search
• ML Platform – MLflow, Databricks / Azure Databricks
• DevOps – GitHub, CI/CD pipelines

Education

• Bachelors degree in Computer Science, Information Technology, Data Science, Artificial Intelligence, Statistics, Mathematics, or a related field.
• Masters degree in Data Science, AI/ML, Computer Science, or Analytics is preferred but not mandatory.
• Relevant cloud or data engineering certifications are advantageous.

Requirements: