Senior/Staff Data Scientist, Applied AI

Clear Street; United States; Remote; USD 180,000 - 250,000; Full time

Job summary

Clear Street is seeking an experienced senior software engineer to own the reliability and quality bar for an AI copilot embedded in a sophisticated trading platform. You will design evaluation systems, benchmarks, and automated quality gates to ensure safety, latency, and correctness across complex workflows.

You will lead model improvement loops, including data collection, labeling/QA, offline experimentation, and tuning.

Qualifications

• Eight+ years of experience shipping production software.
• Strong programming skills across languages and platforms.
• Strong knowledge of CS fundamentals, testing methodologies, and systems design.
• Experience building evaluation frameworks, test harnesses, and benchmark suites for complex systems.
• Experience running model improvement cycles: dataset curation, labeling/QA, and offline experimentation.
• Ability to define metrics and build measurement pipelines.
• Comfort working across the stack, including debugging model/tooling failures.
• High degree of self-motivation.

Responsibilities

• Own the reliability and quality bar for an AI copilot embedded in a trading platform used by sophisticated investors.
• Design and build evaluation systems that measure correctness, safety, latency, and regression risk.
• Develop and maintain benchmarks, including curated golden sets and scenario suites.
• Build automated quality gates and regression workflows.
• Partner with engineering and product to define safe tool/action contracts.
• Own model improvement loops tied to evals, including data collection and fine-tuning.
• Design and operate monitoring and incident response for AI.
• Develop a deep understanding of trading concepts to express them accurately to users.

Skills

Production software

Strong programming

CS fundamentals

Evaluation frameworks

Model evaluation loops

Metrics pipelines

Cross-stack debugging

Self-motivation

Tools

Rust

Test harnesses

Benchmarks

Job description

Responsibilities

• Own the reliability and quality bar for an AI copilot embedded in a trading platform used by sophisticated investors
• Design and build evaluation systems that measure correctness, safety, latency, and regression risk
• Develop and maintain benchmarks, including curated golden sets and scenario suites
• Build automated quality gates and regression workflows
• Partner with engineering and product to define safe tool/action contracts
• Own model improvement loops tied to evals, including data collection and fine-tuning
• Design and operate monitoring and incident response for AI
• Develop a deep understanding of trading concepts to express them accurately to users

Requirements

• At least eight (8) years of experience shipping production software
• Strong proficiency with any programming language
• Strong knowledge of computer science fundamentals, testing methodology, and systems design
• Experience building evaluation frameworks, test harnesses, and benchmark suites for complex systems
• Experience running model improvement cycles: dataset curation, labeling/QA, and offline experimentation
• Ability to define metrics and build measurement pipelines
• Comfort working across the stack, including debugging model/tooling failures
• High degree of self-motivation

Preferred Qualifications

• Experience with fine-tuning, preference optimization, or distillation
• Experience creating domain-specific benchmarks and adversarial suites
• Deep experience with trading across asset classes
• Experience with Rust and performance-sensitive services
• Experience designing incident response and SLOs for ML/AI systems

About the Company

Clear Street is a diversified financial services firm modernizing the brokerage ecosystem by building a cloud-native clearing and custody system designed for today's complex, global market.