AI Alignment Engineer — Research & Experimentation

Zizy Inc.; New York, Northern (NY, KY); Hybrid; USD 120,000 - 220,000; Full time

Benefits offered by this job

Mental health support

Generous time off

Parental leave and family-planning

Annual learning & development stipend

Job summary

Zizy Inc. is seeking research engineers and research scientists to design and implement experiments for alignment research, focusing on scalable training signals for AGI alignment with human intent and safety.

You will design experiments to measure scalable oversight, study generalization, manage large datasets from interpretability experiments, and develop LLM-based alignment methods in a fast-paced, collaborative team environment.

Qualifications

• Experience designing and running ML experiments.
• Interest in alignment research and AI safety.
• Ability to work in fast-paced, collaborative research environments.

Responsibilities

• Design experiments to measure scalable oversight techniques (AI-assisted feedback and debate).
• Study generalization to see if models trained on easy problems solve hard ones.
• Manage large datasets from interpretability experiments and create visualizations.
• Develop experiments to test chain-of-thought reasoning reflecting model cognition.
• Investigate how training against a reward signal affects outputs.
• Explore methods to understand/predict model behaviors, incl. anomalous circuits or catastrophic outputs.
• Design novel approaches for using LLMs in alignment research.

Skills

ML algorithms

PyTorch

Job description

Zizy Inc. is seeking research engineers and research scientists to design and implement experiments for alignment research, focusing on scalable training signals for AGI alignment with human intent and safety.

You will design experiments to measure scalable oversight, study generalization, manage large datasets from interpretability experiments, and develop LLM-based alignment methods in a fast-paced, collaborative team environment.