AI DevEx Engineer - Build Tools to Speed AI Innovation

Neolife Updates; New York (NY); Hybrid; USD 200,000 - 360,000; Full time

Benefits offered by this job

Unlimited PTO

Daily lunch and dinner provided in the

Medical insurance with multiple plan

HSA or FSA

Dental and vision insurance

One Medical membership

401(k) match

In-office desk setup stipend

Job summary

Crosby is seeking a Member of Technical Staff, AI Developer Experience in New York, NY to own internal systems that empower engineers to ship AI features faster. You will build developer tooling, evaluation frameworks, and data pipelines to accelerate production-ready AI capabilities.

You bring 3+ years of software engineering, strong Python, experience with LLMs and AI workflows, and a track record delivering scalable backend tooling. This is a full-time on-site role with competitive salary.

Qualifications

• 3+ years of software engineering experience, ideally on developer tools, infrastructure, or platform teams.
• Strong proficiency in Python and hands-on experience building backend systems and internal tooling.
• Practical experience working with LLMs and modern AI workflows, including APIs such as OpenAI and Anthropic.
• Proven track record building systems for experimentation, evaluation, or data pipelines.
• Strong systems thinking with the ability to design tools that scale across teams.

Responsibilities

• Build AI developer tooling — Design and implement internal tools for prompt management, evaluation, experimentation, and model iteration.
• Accelerate iteration speed — Create systems that enable rapid testing, debugging, and comparison of model outputs across different approaches.
• Own evaluation infrastructure — Develop frameworks for benchmarking, regression testing, and monitoring AI system performance in production.
• Streamline data workflows — Build pipelines and tooling to support data collection, labeling, and feedback loops for continuous model improvement.
• Support production systems — Partner with engineering teams to ensure AI features are reliable, observable, and maintainable at scale.
• Enable the broader team — Identify bottlenecks and build systems that increase developer velocity and confidence across the organization.

Skills

Python

LLMs

Backend systems

Data pipelines

Developer tools

Job description

Crosby is seeking a Member of Technical Staff, AI Developer Experience in New York, NY to own internal systems that empower engineers to ship AI features faster. You will build developer tooling, evaluation frameworks, and data pipelines to accelerate production-ready AI capabilities.

You bring 3+ years of software engineering, strong Python, experience with LLMs and AI workflows, and a track record delivering scalable backend tooling. This is a full-time on-site role with competitive salary.