AI Developer Experience Engineer at Crosby | New York, NY

Neolife Updates; New York (NY); On-site; USD 200,000 - 360,000; Full time

Benefits offered by this job

Unlimited PTO

Office meals provided

Medical insurance

HSA/FSA

Dental & vision

One Medical membership

401(k) match

Desk stipend

Job summary

Crosby is seeking a Member of Technical Staff, AI Developer Experience in New York, NY. You will own internal systems that empower Crosby's engineers, data scientists, and researchers to develop, evaluate, and ship AI features quickly and reliably.

We value 3+ years of software engineering, strong Python skills, and experience with LLMs and AI workflows. You will build developer tooling, data pipelines, and evaluation frameworks that scale across teams in a fast-moving startup environment.

Qualifications

• 3+ years of software engineering experience, ideally on developer tools.
• Strong proficiency in Python and backend tooling.
• Experience with LLMs and AI workflows (OpenAI/Anthropic).
• Proven track record building experimentation or data pipelines.

Responsibilities

• Build AI developer tooling for prompt management, evaluation, experimentation, and model iteration.
• Accelerate iteration speed with rapid testing, debugging, and comparison of model outputs across approaches.
• Own evaluation infrastructure for benchmarking, regression testing, and monitoring AI system performance in production.
• Streamline data workflows with data collection, labeling, and feedback loops for model improvement.
• Support production systems by ensuring reliability, observability, and maintainability at scale.
• Identify bottlenecks and build systems to increase developer velocity and cross-team efficiency.

Skills

Python

LLMs

Developer tooling

Backend systems

Data pipelines

System design

Communication

Tools

OpenAI API

Anthropic API

Job description

Build the Engine Behind Crosby's AI-Native Legal Platform

Crosby is the first AI-native law firm, and it needs engineers who can make its entire team ship faster. The Member of Technical Staff, AI Developer Experience role at Crosby in New York, NY is for someone who finds satisfaction in building the internal systems — prompt management tooling, evaluation frameworks, data pipelines — that multiply what other engineers can accomplish. If you are driven by leverage and want your work to directly accelerate AI feature delivery in a high-stakes industry, this role is worth your attention.

About Crosby

Crosby combines human legal expertise with cutting-edge AI to help ambitious companies like Cursor, Ramp, and Cognition close commercial contracts faster. The firm operates on the conviction that the best legal work comes from amplifying human judgment with the right technology — not replacing it. Founded by Ryan (Stanford Law, Cooley, startup GC) and John (Penn, first 15 engineers at Ramp), Crosby builds proprietary tools and human-in-the-loop workflows that own outcomes alongside corporate legal teams.

What You Will Do

In this role, you will own the internal systems that enable Crosby's engineers, data scientists, and researchers to develop, evaluate, and ship AI features quickly and reliably. Your work will center on three areas: the developer experience around LLMs, data workflows, and experimentation.

• Build AI developer tooling — Design and implement internal tools for prompt management, evaluation, experimentation, and model iteration.
• Accelerate iteration speed — Create systems that enable rapid testing, debugging, and comparison of model outputs across different approaches.
• Own evaluation infrastructure — Develop frameworks for benchmarking, regression testing, and monitoring AI system performance in production.
• Streamline data workflows — Build pipelines and tooling to support data collection, labeling, and feedback loops for continuous model improvement.
• Support production systems — Partner with engineering teams to ensure AI features are reliable, observable, and maintainable at scale.
• Enable the broader team — Identify bottlenecks and build systems that increase developer velocity and confidence across the organization.

Who You Are

Crosby is looking for someone obsessed with leverage — building systems that make other engineers dramatically more productive. You will thrive here if you bring the following:

• 3+ years of software engineering experience, ideally on developer tools, infrastructure, or platform teams.
• Strong proficiency in Python and hands-on experience building backend systems and internal tooling.
• Practical experience working with LLMs and modern AI workflows, including APIs such as OpenAI and Anthropic.
• Proven track record building systems for experimentation, evaluation, or data pipelines.
• Strong systems thinking with the ability to design tools that scale across teams.
• A pragmatic mindset focused on improving developer velocity and real-world outcomes.
• Excellent communication skills and a collaborative approach with engineers, researchers, and product teams.
• A proactive, resourceful attitude suited to a fast-moving startup environment.

Benefits & Perks

• Unlimited PTO
• Daily lunch and dinner provided in the office
• Medical insurance with multiple plan options through Anthem and Cigna
• HSA or FSA, depending on your plan choice
• Dental and vision insurance
• One Medical membership
• 401(k) match
• In-office desk setup stipend

This is a full-time position with a verified base salary range of USD 200,000 – 360,000 per year. If you are passionate about developer experience and excited by the challenge of building foundational systems in an industry where quality and speed both matter, Crosby is the place for you.