Research Scientist (Intern)

Xterraai; San Francisco (CA); On-site; USD 27,552 - 55,104; Full time

Job summary

Xterraai is seeking interns for research positions focused on machine learning and reinforcement learning in San Francisco. This role involves significant autonomy and the opportunity to tackle high-impact projects.

Ideal candidates are pursuing graduate degrees or have substantial research experience in AI. Responsibilities include designing innovative reasoning systems and contributing to advanced research methodologies in geology.

Interns will enjoy a collaborative environment that promotes independent research and hands-on experimentation.

Qualifications

• Hands-on experience training large models, preferably language models.
• Demonstrated experience with reinforcement learning in various domains.
• Publication record in relevant areas is advantageous.

Responsibilities

• Design and train reasoning systems using reinforcement learning.
• Develop reward models and verifiers for improved reasoning.
• Contribute to research and experimentation in geological reasoning.

Skills

Machine learning fundamentals

Reinforcement learning

Debugging distributed training jobs

Search and planning techniques

AI alignment techniques

Education

Graduate degree in machine learning, AI, or related field

Job description

What You’ll Work On

• Reasoning via reinforcement learning: Designing and training reasoning systems using RLHF, RLAIF, and reward modeling approaches, applied to geological hypothesis generation and evaluation.
• Process reward models and verifiers: Developing fine‑grained supervision over intermediate reasoning steps - not just final answers - so the system learns to reason well, not just get lucky.
• Search and planning: Exploring chain-of-thought strategies, search-time compute (e.g., Monte Carlo Tree Search), and other techniques that enable deeper, more deliberate reasoning over geological evidence.
• Scalable oversight: Contributing to alignment and oversight research - figuring out how to reliably supervise models on geological tasks where ground truth is expensive, delayed, or ambiguous.
• Infrastructure and experimentation: Building robust training pipelines, running large-scale experiments, and iterating quickly across the research-to-production lifecycle.
• Evaluation: Contributing to meaningful benchmarks and evaluation methods for geological reasoning capabilities.

What We’re Looking For

• Strong fundamentals in machine learning, with hands-on experience training large models (LLMs preferred but not required).
• Demonstrated experience with reinforcement learning - ideally applied to language models, but strong RL backgrounds from other domains (robotics, game-playing, scientific discovery) are valued.
• Comfort working across the research-engineering spectrum: you can write a paper and you can debug a distributed training job.
• Familiarity with at least some of: reward modeling, RLHF/RLAIF pipelines, search and planning methods, or AI alignment techniques.
• Publication record is a plus but not a strict requirement - we care more about the quality of your thinking and what you’ve built.

For Interns

We welcome outstanding PhD and Masters students (and exceptional undergraduates) for research internships typically lasting 12-16 weeks. Interns work on the same problems as full-time researchers, embedded in a team and owning a meaningful project from day one. What we look for in intern candidates:

• Currently pursuing a graduate degree (PhD or Masters) in machine learning, AI, or a related field - or an undergraduate with significant research experience.
• Coursework or research experience in reinforcement learning, NLP, or deep learning.
• A strong project portfolio or publications demonstrating independent research ability.
• Eagerness to tackle open-ended problems and ship real experiments on real data.

At More Senior Levels, We’d Also Expect

• A track record of identifying and driving high-impact research directions independently.
• Experience mentoring other researchers and influencing technical strategy.
• Deep expertise in one or more of the core technical areas listed above.