Senior AI Data Engineer

Luxoft; Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

Luxoft is seeking AI Engineers for a strategic initiative to build Agentic AI use cases around Trading Surveillance in Singapore. The ideal candidate will have a strong PySpark background and hands-on experience constructing Agentic AI pipelines for code conversion from Q KDB to PySpark.

The role involves designing, developing, and maintaining full-stack Python applications, building RAG systems, and implementing scalable data infrastructure to support AI workloads.

Qualifications

• 5+ years of AI/ML solution implementation
• 3+ years of Pyspark for high-volume data processing
• Proficient in Pyspark coding, lazy evaluation and window functions
• SQL proficiency and query optimization

Responsibilities

• Design, develop, and maintain full-stack Python applications with modern frontend frameworks
• Build and optimize RAG systems for AI applications
• Create and implement vector databases and knowledge stores
• Develop APIs that connect frontend interfaces with backend AI services
• Implement and maintain CI/CD pipelines for AI applications
• Monitor application performance and troubleshoot issues in production
• Design and maintain data pipelines, ETL/ELT workflows, and data infrastructure

Skills

Pyspark

AI/ML solution design

Full-stack AI integration

LLMOps observability

SQL proficiency

Data engineering

GitHub Copilot

Tools

GitHub Copilot

Pyspark coding

Job description

Project description

We are looking for AI Engineers for a strategic initiative to build Agentic AI use cases around Trading Surveillance. Ideal candidate is someone with strong pyspark background coupled with proven experience build Agentic AI pipelines for code conversion from Q KDB to Pyspark.

Responsibilities

• Design, develop, and maintain full-stack Python applications with modern frontend frameworks
• Build and optimize RAG (Retrieval-Augmented Generation) systems for AI applications
• Create and implement efficient vector databases and knowledge stores
• Develop APIs that connect frontend interfaces with backend AI services
• Implement and maintain CI/CD pipelines for AI applications
• Monitor application performance and troubleshoot issues in production
• Design and maintain data pipelines, ETL/ELT workflows, and data infrastructure to support AI and analytics workloads

SKILLS

Must have

• Minimum of 5 years of experience in AI/ML solution implementation (mandatory)
• Minimum of 3 years of experience in Pyspark processing high volume data (mandatory)
• AI & ML Skills:Multi-Agent System Design & OrchestrationFull-stack AI Integration (Frontend to Backend)Advanced Prompt EngineeringLLM Observability, Evaluation & Monitoring (LLMOps)
• Data Engineering Skills:Proficient in Pyspark coding and concepts. Example: Lazy evaluation, window functions, spark architecture and performance finetuning (ex: data skew reduction (salting etc), joins (broadcast, merge sort joins ect)SQL proficiency and query optimization
• Main Technologies:PysparkGit hub Copilot

Nice to have

--