Innovative Model Architectures Research Engineer

Bison; San Francisco (CA); On-site; USD 150,000 - 230,000; Full time

Benefits offered by this job

Comprehensive medical, dental, vision,

FSA plans

Competitive compensation and 401(k)

Relocation and immigration support

In-office snacks and meals provided

Unlimited PTO and company holidays

In-person team in San Francisco

Job summary

Zyphra, an artificial intelligence company based in San Francisco, seeks a Research Engineer - Model Architectures to design and test novel model architectures and training methods. You will work closely with the pre-training team to translate insights into next-generation models.

You will push for improvements in core modeling capabilities, such as loss per FLOP, and address bottlenecks in state-of-the-art models, while maintaining rigorous experimental practices and rapid prototyping.

Qualifications

• Strong research taste and intuition.
• The ability to work through a research project from conception to execution to write-up.
• Strong implementation and prototyping ability to move from idea to experiment quickly.
• Ability to work well in a high-paced research setting.
• Curiosity and joy in understanding intelligence.

Responsibilities

• Design and rigorously test novel model architectures and training methodologies.
• Collaborate closely with the pre-training team to integrate insights into next-gen models.
• Push on bottlenecks like loss per FLOP and efficiency of architectures.
• Demonstrate strong experimental methodology with ablations and thorough evaluation.
• Contribute to a culture of inventing and evaluating 'crazy' ideas with rigor.

Skills

Research intuition

End-to-end research

Prototyping

Team collaboration

Curiosity in AI

RAG retrieval

Reinforcement learning

Control theory

Signal processing

Training pipelines

Ablation experiments

PyTorch

Code comprehension

Publications

Advanced degree

Education

Postgraduate degree in CS/EE/Math/Physics

Job description

Zyphra, an artificial intelligence company based in San Francisco, seeks a Research Engineer - Model Architectures to design and test novel model architectures and training methods. You will work closely with the pre-training team to translate insights into next-generation models.

You will push for improvements in core modeling capabilities, such as loss per FLOP, and address bottlenecks in state-of-the-art models, while maintaining rigorous experimental practices and rapid prototyping.