Research Engineer Intern: ML Evaluation & Data Pipelines

Thinklance LLP; San Francisco (CA); On-site; USD 100,000 - 145,000; Full time

Benefits offered by this job

Meals stipend

Equinox gym membership

Laundry reimbursement

Wellness reimbursement

Job summary

Thinklance LLP in San Francisco is seeking a focused researcher to advance post-training evaluation pipelines and RLVR experiments. You will analyze how data, rewards, and training strategies influence model performance and develop scalable evaluation systems.

Responsibilities include designing reward-shaping experiments, improving algorithms, and quantifying data quality against benchmarks. Collaboration with AI researchers and data teams is essential in a fast-paced, high-ownership environment.

Qualifications

• Strong applied research background focused on post-training and/or model evaluation.
• Strong coding proficiency and hands-on experience with machine learning models.
• Solid understanding of data structures, algorithms, backend systems, and core engineering fundamentals.
• Familiarity with APIs, SQL/NoSQL databases, and cloud platforms.
• Ability to reason deeply about model behavior, experimental results, and data quality.
• Willingness to work in person in San Francisco five days a week, with optional remote Saturdays.

Responsibilities

• Work on post-training and RLVR pipelines to analyze how datasets, rewards, and training strategies affect model performance.
• Design and conduct reward-shaping experiments and algorithmic improvements to enhance LLM tool use, agentic behavior, and real-world reasoning.
• Quantify data usability, quality, and performance improvements on key benchmarks.
• Build and maintain scalable data generation and augmentation pipelines aligned with training needs.
• Create and refine rubrics, evaluators, and scoring frameworks to guide training and evaluation decisions.
• Develop and operate large language model evaluation systems, benchmarks, and metrics at scale.
• Collaborate closely with AI researchers, applied AI teams, and experts producing training data.
• Operate in a fast-paced, experimental research environment with rapid iteration cycles and high ownership.

Skills

Applied research

ML model evaluation

Coding proficiency

Data structures & algorithms

Cloud platforms

Job description

Thinklance LLP in San Francisco is seeking a focused researcher to advance post-training evaluation pipelines and RLVR experiments. You will analyze how data, rewards, and training strategies influence model performance and develop scalable evaluation systems.

Responsibilities include designing reward-shaping experiments, improving algorithms, and quantifying data quality against benchmarks. Collaboration with AI researchers and data teams is essential in a fast-paced, high-ownership environment.