Senior Software Engineer (AI Systems Performance)

Luxoft; United States; Remote; USD 180,000 - 240,000; Full time

Job summary

Luxoft seeks a software engineer focused on distributed inferencing on GPUs to optimize performance of AI workloads across multi-GPU, multi-node clusters. You will join a core team of specialists exploring the latest hardware and software technology.

You will collaborate with GPU library teams, implement scalable strategies, and help build benchmarking pipelines and monitoring dashboards for performance, accuracy, and reliability.

Qualifications

• Hands-on experience with AI inference or serving frameworks such as vLLM, SGLang, and Llama.cpp.
• Understanding KV cache transfer mechanisms and expert parallelization approaches (e.g., DeepEP, MORI, PPLX-Garden).
• Strong C/C++ and Python skills, with experience in software design, debugging, performance analysis, and test development.
• Experience running AI workloads on large-scale, heterogeneous compute clusters.
• Familiarity with cluster management and orchestration platforms such as SLURM and Kubernetes (K8s).
• Experience with GitHub, Jenkins, or similar CI/CD tools and modern development workflows.

Responsibilities

• Enable and benchmark AI models on large-scale distributed systems to evaluate performance, accuracy, and scalability.
• Optimize AI workloads across scale-up, scale-out, and distributed configurations.
• Collaborate closely with internal GPU library teams to analyze and optimize distributed workloads for high throughput and low latency.
• Develop and apply optimal parallelization strategies for AI workloads to achieve best-in-class performance across diverse system configurations.
• Contribute to distributed model management systems, model zoos, benchmarking pipelines, and technical documentation.
• Build and maintain real-time dashboards reporting performance, accuracy, and reliability metrics for internal stakeholders and external users.

Skills

AI framework engineering

KV cache & expert parallelization

C/C++ & Python

Large-scale distributed systems

Cluster & orchestration (SLURM/K8s)

CI/CD tools (GitHub/Jenkins)

Education

Master’s or PhD in CS/CE or related field

Job description

Project description

A leading semiconductor and high-performance computing company is looking for a software engineer who is passionate about Distributed Inferencing on GPUs and improving the performance of key applications and benchmarks. You will be a member of a core team of incredibly talented industry specialists and will work with the very latest hardware and software technology

Responsibilities

• We are seeking a software engineer with strong technical expertise in C++/ Python development, solving performance and investigating scalability on multi-GPU, multi-node clusters. He is also passionate about quality assurance, benchmarking, and automation in the AI/ML space. The ideal candidate thrives in both collaborative and independent environments, demonstrates excellent problem-solving skills, and takes ownership in defining goals and delivering impactful solutions.KEY RESPONSIBILITIES:Distributed AI Enablement and Benchmarking: Enable and benchmark AI models on large-scale distributed systems to evaluate performance, accuracy, and scalability.Scalable Systems Optimization: Optimize AI workloads across scale-up (multi-GPU), scale-out (multi-node), and scale-across distributed system configurations.Cross-Team Collaboration: Collaborate closely with internal GPU library teams to analyze and optimize distributed workloads for high throughput and low latency.Parallelization Strategies: Develop and apply optimal parallelization strategies for AI workloads to achieve best-in-class performance across diverse system configurations.Model Infrastructure and Management: Contribute to distributed model management systems, model zoos, monitoring frameworks, benchmarking pipelines, and technical documentation.Performance Monitoring and Visualization: Build and maintain real-time dashboards reporting performance, accuracy, and reliability metrics for internal stakeholders and external users.

SKILLS

Must have

• PREFERRED EXPERIENCE:AI Framework Engineering: Hands-on experience with AI inference or serving frameworks such as vLLM, SGLang, and Llama.cpp.KV Cache and Expert Parallelization: Understanding KV cache transfer mechanisms and technologies (e.g., Mooncake, NIXL/RIXL) and expert parallelization approaches (e.g., DeepEP, MORI, PPLX-Garden).Programming and Software Design: Strong C/C++ and Python skills, with experience in software design, debugging, performance analysis, and test development.Large-Scale Distributed Systems: Experience running AI workloads on large-scale, heterogeneous compute clusters.Cluster and Orchestration Systems: Familiarity with cluster management and orchestration platforms such as SLURM and Kubernetes (K8s).Development Tools and Workflows: Experience with GitHub, Jenkins, or similar CI/CD tools and modern development workflows.

Nice to have

ACADEMIC CREDENTIALS:Master’s or PhD degree in Computer Science, Computer Engineering, or a related field, or equivalent practical experience.