Machine Learning Engineer (Training & Inference Systems)

Jobzhr; New York (NY); On-site; USD 180,000 - 280,000; Full time

Job summary

Jobzhr, a leading global trading firm, is expanding its ML infrastructure in New York. We are hiring engineers to build distributed training and low-latency inference systems that move models from research into production.

You will work closely with researchers and traders, operating in a fast-paced, IC-heavy environment near the trading floor. You should have 3+ years of hands-on experience with PyTorch, distributed training, and production-grade ML pipelines.

Qualifications

• Build and scale distributed training systems for large models.
• Develop low-latency inference pipelines to production.
• Collaborate directly with researchers and traders to deploy models.
• 3+ years of relevant experience; IC-heavy role with hands-on work.

Responsibilities

• Design and implement scalable training infrastructure for multi-GPU/multi-node setups.
• Create production-ready inference pipelines with low latency.
• Collaborate with researchers and traders to deliver end-to-end systems.
• Manage growing GPU footprint and ensure reliability in production.

Skills

PyTorch

Distributed systems

Performance engineering

Production ML

Collaboration with researchers and qun

Education

Bachelor's degree in Computer Science or related field

Tools

CUDA

Docker

Job description

A leading global trading firm is building out its machine learning function and hiring engineers to develop the distributed training and inference systems that take models from research into production. This is the most urgent hire on a team that sits unusually close to the trading desk — you'll feel more like part of the trading floor than a traditional ML function.

What you'll do

• Build and scale distributed training systems for large models
• Develop low-latency inference pipelines to get models into production
• Work directly with researchers training the models and the traders/quants relying on them
• Operate on a rapidly growing GPU footprint (low thousands today, scaling into the tens of thousands over the next few years)

What we're looking for

• Strong experience building training and/or inference infrastructure at scale
• Deep PyTorch fluency and strong performance-engineering instincts
• Distributed systems experience — multi-GPU/multi-node training, model serving
• A track record of hands-on building (this is an IC-heavy role, not a strategy seat)
• Background at a top tech firm, trading firm, or a strong big-tech + startup combination
• 3+ years of relevant experience; flexible on level for the right person

Why it's exciting

• Real impact, fast. Your systems directly enable production models that drive trading decisions — no research-to-nowhere, no shipping into a void.
• Scale that's actually scaling. A GPU footprint growing 10x+ over the next two to three years, with the mandate and budget to match.
• Proximity to the work that matters. You'll sit alongside researchers and traders, not three layers removed. Tight feedback loops, smart people, genuine ownership.
• Top-of-market comp for engineers who can build at this level.