Machine Learning Performance Engineer, Training

towerresearchcapital

New York (NY)

Sur place

USD 140 000 - 230 000

Plein temps

14 jours+

Obtenez une réponse de cet employeur

Votre CV et votre lettre de motivation, adaptés exactement à ce que l’employeur recherche.

Passez les filtres ATS

Résumé du poste

Tower Research Capital in New York is seeking an experienced ML Training Engineer to bridge the gap between quantitative research and high-performance computing, building and optimizing the systems used to train machine learning models at scale.

You will accelerate the end-to-end training lifecycle—from data ingestion and distributed execution to kernel performance and hardware utilization—enabling researchers to iterate more quickly on increasingly complex models and datasets.

Qualifications

  • 3+ years of experience optimizing ML training workloads in high-performance, distributed, or large-scale computing environments.
  • Deep knowledge of ML frameworks such as PyTorch or JAX, including their execution models, compilation paths, autograd systems, and distributed-training capabilities.
  • Strong programming skills in Python and C++, with experience developing or optimizing performance-critical systems.
  • Proven experience with GPU kernel development and optimization using technologies such as CUDA, Triton, CUTLASS, cuBLAS, cuDNN, or related libraries.
  • Strong understanding of GPU architecture, including streaming multiprocessor execution, warp scheduling, tensor cores, and the memory hierarchy from registers through HBM.

Responsabilités

  • Benchmark model-training workloads across CPUs, GPUs, and other accelerator platforms to identify bottlenecks and guide Tower's compute infrastructure decisions.
  • Design and optimize distributed training strategies, including data, tensor, pipeline, and model parallelism.
  • Improve communication efficiency across multi-GPU and multi-node environments by optimizing collective operations, topology awareness, and computation-communication overlap.
  • Analyze and improve the full training pipeline, including data loading, preprocessing, memory management, forward and backward passes, optimizer execution, checkpointing, and experiment recovery.
  • Identify bottlenecks across compute, memory, storage, networking, and interconnects to increase accelerator utilization and researcher productivity.
  • Develop and optimize GPU kernels and performance-critical framework components for quantitative machine learning workloads.
  • Apply techniques such as mixed-precision training, gradient accumulation, activation checkpointing, operator fusion, and memory-efficient optimizers.
  • Partner with HPC and infrastructure teams to optimize workload scheduling, resource allocation, observability, fault tolerance, and reproducibility across shared compute environments.
  • Work closely with ML Researchers, Quantitative Researchers, HPC Engineers, Systems Engineers, and hardware specialists to translate research requirements into highly efficient training systems.

Description du poste

Tower Research Capital is a leading quantitative trading firm founded in 1998. Tower has built its business on a high-performance platform and independent trading teams. We have a 25+ year track record of innovation and a reputation for discovering unique market opportunities.

Tower is home to some of the world's best systematic trading and engineering talent. We empower portfolio managers to build their teams and strategies independently while providing the economies of scale that come from a large, global organization.

Engineers thrive at Tower while developing electronic trading infrastructure at a world class level. Our engineers solve challenging problems in the realms of low-latency programming, FPGA technology, hardware acceleration and machine learning. Our ongoing investment in top engineering talent and technology ensures our platform remains unmatched in terms of functionality, scalability and performance.

At Tower, every employee plays a role in our success. Our Business Support teams are essential to building and maintaining the platform that powers everything we do - combining market access, data, compute, and research infrastructure with risk management, compliance, and a full suite of business services. Our Business Support teams enable our trading and engineering teams to perform at their best.

At Tower, employees will find a stimulating, results-oriented environment where highly intelligent and motivated colleagues inspire each other to reach their greatest potential.

Summary

You will bridge the gap between quantitative research and high-performance computing, building and optimizing the systems used to train machine learning models at scale. You will focus on accelerating the end-to-end training lifecycle-from data ingestion and distributed execution to kernel performance and hardware utilization-enabling researchers to iterate more quickly across increasingly complex models and datasets.

Responsibilities
Training Performance and Benchmarking

Benchmark model-training workloads across CPUs, GPUs, and other accelerator platforms to identify bottlenecks and guide Tower's compute infrastructure decisions.

Develop performance models and standardized benchmarks for measuring throughput, utilization, scalability, and time to convergence.

Distributed Training Optimization

Design and optimize distributed training strategies, including data, tensor, pipeline, and model parallelism.

Improve communication efficiency across multi-GPU and multi-node environments by optimizing collective operations, topology awareness, and computation-communication overlap.

End-to-End Training Efficiency

Analyze and improve the full training pipeline, including data loading, preprocessing, memory management, forward and backward passes, optimizer execution, checkpointing, and experiment recovery.

Identify bottlenecks across compute, memory, storage, networking, and interconnects to increase accelerator utilization and researcher productivity.

GPU Kernel and Framework Development

Develop and optimize GPU kernels and performance-critical framework components for quantitative machine learning workloads.

Integrate specialized libraries, compilers, and execution techniques to improve throughput, memory efficiency, and numerical performance.

Model and Numerical Optimization

Apply techniques such as mixed-precision training, gradient accumulation, activation checkpointing, operator fusion, and memory-efficient optimizers.

Evaluate tradeoffs among training speed, numerical stability, reproducibility, model quality, and infrastructure cost.

Training Infrastructure

Partner with HPC and infrastructure teams to optimize workload scheduling, resource allocation, observability, fault tolerance, and reproducibility across shared compute environments.

Help define the architecture and tooling required to support large-scale experimentation across on-premises and cloud-based infrastructure.

Cross-Functional Collaboration

Work closely with ML Researchers, Quantitative Researchers, HPC Engineers, Systems Engineers, and hardware specialists to translate research requirements into highly efficient training systems.

Qualifications
  • 3+ years of experience optimizing machine learning training workloads in high-performance, distributed, or large-scale computing environments.
  • Deep knowledge of machine learning frameworks such as PyTorch or JAX, including their execution models, compilation paths, autograd systems, and distributed-training capabilities.
  • Strong programming skills in Python and C++, with experience developing or optimizing performance-critical systems.
  • Proven experience with GPU kernel development and optimization using technologies such as CUDA, Triton, CUTLASS, cuBLAS, cuDNN, or related libraries.
  • Strong understanding of GPU architecture, including streaming multiprocessor execution, warp scheduling, tensor cores, and the memory hierarchy from registers through HBM.
  • Experience with distributed-trai
Obtenez votre examen gratuit et confidentiel de votre CV.

ou faites glisser et déposez votre fichier ici.

Similar jobs

Postes similaires à comparer

ML Training Performance Engineer - HPC & GPUs
ML Training Performance Engineer - HPC & GPUs

towerresearchcapital • New York (NY)

Sur place
USD 140 000 - 230 000
Distributed Training & Performance Engineer - Vice President
Distributed Training & Performance Engineer - Vice President

Testing Site - JPMC • New York (NY)

Sur place
USD 180 000 - 240 000
Machine Learning Infrastructure Engineer, Kernel Optimization
Machine Learning Infrastructure Engineer, Kernel Optimization

Efference • San Francisco (CA)

Sur place
USD 180 000 - 240 000
Training / AI Infrastructure
Training / AI Infrastructure

Genesis • San Carlos (CA)

Sur place
USD 180 000 - 240 000
ML Systems Engineer — Performance & Scale
ML Systems Engineer — Performance & Scale

Veeda Innovation • California

Hybride
USD 150 000 - 230 000
Machine Learning Performance Engineer (Inference)
Machine Learning Performance Engineer (Inference)

Priorslab • New York (NY)

Hybride
USD 200 000 - 300 000
Hybrid working opportunities
Free breakfast, lunch, and snacks
Wellness reimbursement
+2
Training / AI Infrastructure
Training / AI Infrastructure

Genesis AI • San Francisco (CA)

Sur place
USD 180 000 - 260 000
Training / AI Infrastructure
Training / AI Infrastructure

Genesis AI • San Carlos (CA)

Sur place
USD 180 000 - 240 000
Training / AI Infrastructure
Training / AI Infrastructure

Genesis AI • États-Unis

Hybride
USD 190 000 - 240 000
Machine Learning Performance Engineer (Inference) Tower Research Capital · New York, United States a week ago
Machine Learning Performance Engineer (Inference) Tower Research Capital · New York, United States a week ago

Tradermath • New York (NY)

Hybride
USD 200 000 - 300 000
Generous paid time off policies
Savings plans and other financial well
Hybrid working opportunities
+5