AI DevEx Engineer — Accelerate ML Tooling & Pipelines

Crosby; New York (NY); On-site; USD 180,000 - 250,000; Full time

Job summary

Crosby in New York is seeking an AI Developer Experience Engineer to build systems enhancing developer experiences for AI features. You'll focus on tooling for iteration and quality assurance while collaborating closely with engineers and data scientists.

The ideal candidate has 3-7 years of software engineering experience and is proficient in Python, with hands-on knowledge of LLMs. The position offers a competitive salary and strives to improve developer velocity within a fast-paced environment.

Qualifications

• 3–7+ years of software engineering experience, ideally working on developer tools, infrastructure, or platform teams.
• Strong proficiency in Python and experience building backend systems and internal tooling.
• Hands-on experience working with LLMs, APIs (e.g., OpenAI, Anthropic), and modern AI workflows.

Responsibilities

• Design and implement internal tools for prompt management, evaluation, experimentation, and model iteration.
• Create systems that enable rapid testing, debugging, and comparison of model outputs.
• Develop frameworks for benchmarking, regression testing, and monitoring AI system performance.

Skills

Software engineering experience

Proficiency in Python

Knowledge of LLMs

Experience with APIs

Systems thinking

Strong communication skills

Job description

Crosby in New York is seeking an AI Developer Experience Engineer to build systems enhancing developer experiences for AI features. You'll focus on tooling for iteration and quality assurance while collaborating closely with engineers and data scientists.

The ideal candidate has 3-7 years of software engineering experience and is proficient in Python, with hands-on knowledge of LLMs. The position offers a competitive salary and strives to improve developer velocity within a fast-paced environment.