Senior AI Data Engineer

Luxoft; Chennai District; On-site; INR 2,400,000 - 3,200,000; Full time

Job summary

Luxoft is seeking AI Engineers in Chennai to drive a strategic initiative around Agentic AI for Trading Surveillance. The role focuses on PySpark-based data processing, building end-to-end AI pipelines, RAG systems, and scalable data infrastructure.

You will implement APIs, CI/CD for AI workloads, and optimize performance across full-stack AI solutions. The ideal candidate has strong PySpark experience with high-volume data and hands-on experience in multi-agent designs, LLM observability, and

Qualifications

• Minimum 5 years of AI/ML solution implementation.
• Minimum 3 years of PySpark processing of high-volume data.
• Experience in multi-agent system design and orchestration.
• Full-stack AI integration from frontend to backend.

Responsibilities

• Design, develop, and maintain Python applications with modern frontends.
• Build RAG systems for AI applications and knowledge stores.
• Create APIs connecting frontend to AI services.
• Implement and maintain CI/CD pipelines for AI apps.
• Monitor performance and troubleshoot production issues.
• Design and maintain data pipelines and ETL/ELT workflows.
• Optimize PySpark jobs and data architectures.

Skills

Pyspark processing

AI/ML solution design

Full-stack AI integration

LLMOps observability

Prompt engineering

Data pipeline design

Spark tuning

Git / GitHub Copilot

Tools

GitHub Copilot

Git

Job description

Project description

We are looking for AI Engineers for a strategic initiative to build Agentic AI use cases around Trading Surveillance. Ideal candidate is someone with strong pyspark background coupled with proven experience build Agentic AI pipelines for code conversion from Q KDB to Pyspark.

Responsibilities

• Design, develop, and maintain full-stack Python applications with modern frontend frameworks


• Build and optimize RAG (Retrieval-Augmented Generation) systems for AI applications


• Create and implement efficient vector databases and knowledge stores


• Develop APIs that connect frontend interfaces with backend AI services


• Implement and maintain CI/CD pipelines for AI applications


• Monitor application performance and troubleshoot issues in production


• Design and maintain data pipelines, ETL/ELT workflows, and data infrastructure to support AI and analytics workloads

SKILLS

Must have

• Minimum of 5 years of experience in AI/ML solution implementation (mandatory)


• Minimum of 3 years of experience in Pyspark processing high volume data (mandatory)


• AI & ML Skills:Multi-Agent System Design & OrchestrationFull-stack AI Integration (Frontend to Backend)Advanced Prompt EngineeringLLM Observability, Evaluation & Monitoring (LLMOps)


• Data Engineering Skills:Proficient in Pyspark coding and concepts. Example: Lazy evaluation, window functions, spark architecture and performance finetuning (ex: data skew reduction (salting etc), joins (broadcast, merge sort joins ect)SQL proficiency and query optimization


• Main Technologies:PysparkGit hub Copilot

Nice to have

--