Research Engineer, Large-Scale Training

Together AI; San Francisco (CA); On-site; USD 200,000 - 290,000; Full time

Benefits offered by this job

Health insurance

Startup equity

Job summary

Together AI in San Francisco is seeking a Research Engineer to turn cutting-edge research on efficient foundation model training into robust, high-performance systems. You will optimize the training infrastructure, profile bottlenecks, and implement state-of-the-art techniques for production environments.

You will work closely with Research Scientists to deploy open-source models on the platform and build experimental infrastructure that accelerates research while delivering reliable, scalable

Qualifications

• Independently take ambiguous performance or infrastructure problems from investigation through deployment.
• Strong programming in Python and PyTorch, with efficient, maintainable code.
• Hands-on experience training or fine-tuning large neural networks in multi-GPU or multi-node environments.
• Solid understanding of ML systems fundamentals, including GPU architecture, mixed-precision training, and distributed training paradigms.
• Strong communication skills and the ability to collaborate with researchers and engineers.
• Passion for staying current with advances in AI research and applying them to production systems.
• Excitement translating cutting-edge research into production systems that deliver customer impact.

Responsibilities

• Design, implement, and optimize core components of Together's large-scale training infrastructure.
• Integrate new model architectures, validate training correctness and convergence, and optimize performance for production fine-tuning workloads.
• Profile distributed training workloads to identify bottlenecks across compute, memory, and communication.
• Design and execute experiments to validate performance hypotheses and benchmark new approaches against state-of-the-art methods.
• Partner closely with Research Scientists to productionize novel training methods and contribute to publications and open-source releases.
• Rapidly enable support for newly released open-source foundation models on the Together platform.
• Build and maintain experimental infrastructure that accelerates research while ensuring production-quality reliability and scalability.

Skills

Python

PyTorch

Distributed training

GPU optimization

Research-to-production

Communication

Tools

CUDA

Triton

NCCL

NVSHMEM

DeepSpeed

Megatron-LM

Job description

The Model Shaping team at Together AI works on products and research for tailoring open foundation models to downstream applications. We build services that allow machine learning developers to choose the best models for their tasks and further improve these models using domain-specific data. In addition, we develop new methods for more efficient model training and evaluation, drawing inspiration from a broad spectrum of ideas across machine learning, natural language processing, and ML systems.

As a Research Engineer on the Scaling Team within Model Shaping, you will turn cutting‑edge research on efficient foundation model training into robust, high‑performance systems. You will profile and optimize Together's training infrastructure, identify performance bottlenecks across the stack, and implement state‑of‑the‑art techniques from both the research literature and our own scientists in production environments.

Your work will directly shape the fine‑tuning experience of Together's customers. You will rapidly bring newly released open‑source models onto the Model Shaping platform, ensuring they train efficiently and reliably across diverse customer workloads. Working closely with Research Scientists, you will also build the experimental infrastructure that accelerates research and enables validated ideas to be deployed reliably at scale.

Responsibilities

• Design, implement, and optimize core components of Together's large‑scale training infrastructure.
• Integrate new model architectures, validate training correctness and convergence, and optimize performance for production fine‑tuning workloads.
• Profile distributed training workloads to identify and eliminate bottlenecks across compute, memory, and communication.
• Design and execute experiments to validate performance hypotheses and benchmark new approaches against state‑of‑the‑art methods.
• Partner closely with Research Scientists to productionize novel training methods and contribute to publications and open‑source releases.
• Rapidly enable support for newly released open‑source foundation models on the Together platform.
• Build and maintain experimental infrastructure that accelerates research while ensuring production‑quality reliability and scalability.

Requirements

• Demonstrated ability to independently take ambiguous performance or infrastructure problems from investigation through deployment.
• Strong programming skills in Python and PyTorch, with an emphasis on writing efficient, maintainable code.
• Hands‑on experience training or fine‑tuning large neural networks in multi‑GPU or multi‑node environments.
• Solid understanding of ML systems fundamentals, including GPU architecture, mixed‑precision training, and distributed training paradigms such as data, tensor, pipeline, or expert parallelism.
• Strong communication skills and the ability to collaborate effectively with both researchers and engineers.
• Passion for staying current with advances in AI research and applying them to real‑world systems.
• Excitement about translating cutting‑edge research into production systems that deliver customer impact.

Nice to Have

• Experience writing optimized NVIDIA GPU kernels using CUDA or Triton, or implementing communication collectives with technologies such as NCCL or NVSHMEM.
• Experience with large‑scale training frameworks such as FSDP, DeepSpeed, Megatron‑LM, or custom distributed training systems.
• Experience optimizing distributed training for compute efficiency, memory efficiency, or scalability.
• Experience running and managing large‑scale GPU experiments, including scheduling, monitoring, and fault tolerance.
• Contributions to widely used open‑source ML or ML systems projects.
• Experience building or operating ML products or managed services used by external customers.

About Together AI

Together AI, the AI Native Cloud, is purpose‑built for AI engineers. AI application developers get high‑performance inference that scales reliably, fine‑tuning and reinforcement learning for creating frontier‑level specialized models, and pre‑training at massive scale for fully custom intelligence, all around a marketplace of leading open models that teams can run, adapt, and own. Trusted by Cursor, Decagon, ElevenLabs, Salesforce, and Zoom, Together serves 400+ trillion tokens a month.

Compensation

We offer competitive compensation, startup equity, health insurance, and other benefits. The US base salary range for this full‑time position is $200,000 - $290,000. Our salary ranges are determined by location, level and role. Individual compensation will be determined by experience, skills, and job‑related knowledge.

Together AI is an Equal Opportunity Employer and is proud to offer equal employment opportunity to everyone regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity, veteran status, and more.

U.S. Standard Demographic Questions

We invite applicants to share their demographic background. If you choose to complete this survey, your responses may be used to identify areas of improvement in our hiring process.

How would you describe your gender identity? (mark all that apply) Select...

How would you describe your racial/ethnic background? (mark all that apply) Select...

How would you describe your sexual orientation? (mark all that apply) Select...

Do you identify as transgender? (select one) Select...

Do you have a disability or chronic condition (physical, visual, auditory, cognitive, mental, emotional, or other) that substantially limits one or more of your major life activities, including mobility, communication (seeing, hearing, speaking), and learning? (select one) Select...

Are you a veteran or active member of the United States Armed Forces? (select one) Select...

Voluntary Self‑Identification

For government reporting purposes, we ask candidates to respond to the below self‑identification survey.Completion of the form is entirely voluntary. Whatever your decision, it will not be considered in the hiringprocess or thereafter. Any information that you do provide will be recorded and maintained in aconfidential file.

As set forth in Together AI’s Equal Employment Opportunity policy,we do not discriminate on the basis of any protected group status under any applicable law.

If you believe you belong to any of the categories of protected veterans listed below, please indicate by making the appropriate selection.As a government contractor subject to the Vietnam Era Veterans Readjustment Assistance Act (VEVRAA), we request this information in order to measurethe effectiveness of the outreach and positive recruitment efforts we undertake pursuant to VEVRAA. Classification of protected categoriesis as follows:

A \"disabled veteran\" is one of the following: a veteran of the U.S. military, ground, naval or air service who is entitled to compensation (or who but for the receipt of military retired pay would be entitled to compensation) under laws administered by the Secretary of Veterans Affairs; or a person who was discharged or released from active duty because of a service‑connected disability.

A \"recently separated veteran\" means any veteran during the three‑year period beginning on the date of such veteran's discharge or release from active duty in the U.S. military, ground, naval, or air service.

An \"active duty wartime or campaign badge veteran\" means a veteran who served on active duty in the U.S. military, ground, naval or air service during a war, or in a campaign or expedition for which a campaign badge has been authorized under the laws administered by the Department of Defense.

An \"Armed forces service medal veteran\" means a veteran who, while serving on active duty in the U.S. military, ground, naval or air service, participated in a United States military operation for which an Armed Forces service medal was awarded pursuant to Executive Order 12985.