Lead Data & AI Engineer: Production-Ready Pipelines

Luxoft; United States; Remote; USD 180,000 - 240,000; Full time

Job summary

Luxoft is seeking a senior data engineer to design, build, and optimize scalable data pipelines and integrations supporting AI, analytics, and business apps. You will drive data quality, lineage, and monitoring while leveraging a modern AI-first approach to design, test, and deploy solutions.

You will collaborate with business users, product managers, and engineering teams to translate objectives into robust technical implementations, working within an agile delivery environment and cloud-based

Qualifications

• 10+ years of professional software development, including 5+ years of data engineering experience.
• Advanced proficiency in Python, including experience building scalable backend services and data processing solutions.
• Strong hands-on experience designing and implementing data pipelines, data integration solutions, and large-scale data processing architectures.
• Experience working with relational databases and modern data storage technologies, including graph, vector, or analytical data platforms.
• Practical experience building and deploying AI/LLM-enabled applications in production environments.
• Experience with LangGraph, LangChain, agentic workflows, or orchestration frameworks for AI applications; evaluations.
• Familiarity with modern AI tooling and workflows, including and AI-assisted software engineering practices.
• Experience operating in agile delivery environments.
• Strong communication skills and ability to interact efficiently with various stakeholder groups including business users.

Responsibilities

• Design, build, and enhance data pipelines and integration frameworks supporting AI, analytics, and business applications.
• Improve data quality, reliability, lineage, and monitoring.
• Implement AI first approach: use AI agents as a primary tool to design, build, test, and support solutions.
• Collaborate closely with business stakeholders, product managers, and engineering teams to translate business objectives into technical solutions.

Skills

Senior software dev

Python expertise

Data pipelines

Databases

AI in production

AI tooling

Agile

Communication

Cloud concepts

Software design

Tools

Databricks

Spark

Delta Lake

Aurora

Neptune

AWS (ECS Fargate, Lambda, SQS)

Terraform / IaC

LangChain

Job description

Project description

Build reusable platform capabilities and data products that let quantitative researchers discover, access, and consume data across a complex, disparate landscape, and then accelerate the good prototypes to production.

Responsibilities

• - Design, build, and enhance data pipelines and integration frameworks supporting AI, analytics, and business applications.- Improve data quality, reliability, lineage, and monitoring.- Implement AI first approach: use AI agents as a primary tool to design, build, test, and support solutions- Collaborate closely with business stakeholders, product managers, and engineering teams to translate business objectives into technical solutions.

SKILLS

Must have

• - 10+ years of professional software development, including 5+ years of data engineering experience.- Advanced proficiency in Python, including experience building scalable backend services and data processing solutions.- Strong hands-on experience designing and implementing data pipelines, data integration solutions, and large-scale data processing architectures- Experience working with relational databases and modern data storage technologies, including graph, vector, or analytical data platforms.- Practical experience building and deploying AI/LLM-enabled applications in production environments.- Experience with LangGraph, LangChain, agentic workflows, or orchestration frameworks for AI applications; evaluations.- Familiarity with modern AI tooling and workflows, including and AI-assisted software engineering practices.- Experience operating in agile delivery environments.- Strong communication skills and ability to interact efficiently with various stakeholder groups including business users.

Nice to have

- Experience with Databricks, Spark, Delta Lake, or similar modern data engineering platforms.- Experience designing enterprise data lakes, lakehouses, or large-scale analytical platforms.- Domain experience within Investment Management, Asset Management, Capital Markets, or Financial Services.- Strong understanding of Fixed Income products and related investment workflows. Candidates with significant Fixed Income domain expertise will be considered highly valuable.- Experience with Aurora, Neptune, and other AWS data services.Strong hands-on experience designing and implementing data pipelines, data integration solutions, and large-scale data processing architectures.- Experience architecting, provisioning, and managing production infrastructure on AWS, including services such as ECS Fargate, Lambda, SQS, and related cloud-native technologies.- Experience defining cloud architecture programmatically using Infrastructure-as-Code (IaC) tools such as AWS CDK or Terraform.