Forward Deployed Engineer, Applied AI

Jobtailor; California (MO); Hybrid; USD 180,000 - 270,000; Full time

Job summary

Jobtailor is seeking an Architect to build enterprise-grade AI solutions, including advanced AI agents, and to own workstreams from prototype to production. You will design systems, write robust Python/SQL pipelines, and collaborate with customers and internal teams.

You will define evaluation frameworks, build golden datasets, and run iterative evaluation loops to improve agent accuracy, safety, and faithfulness while maintaining security in large-scale environments.

Qualifications

• Bachelor's degree in Computer Science, Engineering, a related technical field, or equivalent practical experience.
• 3+ years of professional software engineering experience
• Willingness to travel
• Proven experience building applications using LLMs, especially RAG and agentic workflows
• Hands-on experience defining quality metrics and running evaluations for LLM or agent systems
• Experience using evaluations to systematically improve quality
• Excellent problem-solving and communication skills
• Ability to articulate complex technical concepts to diverse stakeholders
• Comfort with ambiguity and fast-paced, changing Generative AI environments

Responsibilities

• Architect, build, and deploy enterprise-grade AI solutions, including sophisticated AI agents
• Own workstreams end-to-end from prototype through production
• Translate customer goals into quality metrics, evaluation frameworks, and golden datasets
• Run systematic evaluation loops to improve agent accuracy, faithfulness, and safety
• Design, iterate, and ship robust, scalable code and pipelines using Python and SQL
• Implement, deploy, monitor, and optimize AI solutions in secure, large-scale production environments
• Build safety guardrails, observability, and human-review workflows
• Use production traces and user feedback to improve evaluations
• Partner directly with customer data science and engineering teams as a technical advisor
• Collaborate cross-functionally with Product and Engineering teams and share customer feedback
• Spend at least 25% of time onsite with strategic customers

Skills

Python

SQL

LLMs

A/B Testing

Data Analysis

Quality Metrics

Agentic Workflows

Production Traces

Education

Bachelor's degree in Computer Science, Engineering, a related technical field, or equivalent practical experience

Tools

AWS

Azure

GCP

Pandas

NumPy

Snowpark

Braintrust

LangSmith

Arize

Promptfoo

Job description

• Architect, build, and deploy enterprise-grade AI solutions, including sophisticated AI agents
• Own workstreams end-to-end from prototype through production
• Translate customer goals into quality metrics, evaluation frameworks, and golden datasets
• Run systematic evaluation loops to improve agent accuracy, faithfulness, and safety
• Design, iterate, and ship robust, scalable, performant code and pipelines using Python and SQL
• Implement, deploy, monitor, and optimize AI solutions in secure, large-scale production environments
• Build safety guardrails, observability, and human-review workflows
• Use production traces and user feedback to improve evaluations
• Partner directly with customer data science and engineering teams as a technical advisor
• Collaborate cross-functionally with Product and Engineering teams and share customer feedback
• Spend at least 25% of time onsite with strategic customers

Requirements

• Bachelor's degree in Computer Science, Engineering, a related technical field, or equivalent practical experience
• 3+ years of professional software engineering experience
• Willingness to travel
• Proven experience building applications using LLMs, especially RAG and agentic workflows
• Hands-on experience defining quality metrics and running evaluations for LLM or agent systems
• Experience using evaluations to systematically improve quality
• Excellent problem-solving and communication skills
• Ability to articulate complex technical concepts to diverse stakeholders
• Comfort with ambiguity and fast-paced, changing Generative AI environments
• Preferred: experience building evaluation sets from production traces and synthetic data
• Preferred: structured experimentation, including A/B tests, ablations, and offline evaluations
• Preferred: familiarity with Braintrust, LangSmith, Arize, Weave, Promptfoo, or custom evaluation harnesses
• Preferred: failure-mode analysis for agent or RAG systems
• Preferred: MLOps lifecycle experience in AWS, Azure, or GCP
• Preferred: familiarity with pandas, NumPy, or Snowpark
• Preferred: customer-facing technical experience
• Preferred: startup experience
• Must follow Snowflake confidentiality and security standards and data security plan

Demonstrates expertise in architecting and deploying enterprise-grade AI solutions, with a strong focus on building applications using LLMs and implementing evaluation frameworks. Proven ability to collaborate with cross-functional teams and articulate complex technical concepts to diverse stakeholders.

Highest-signal resume keywords

• AI Solution Architecture
• LLM Application Development
• Quality Metrics Definition
• Systematic Evaluation Improvement
• MLOps Lifecycle Experience

ATS Optimization Keywords

Hard Skills

• Python
• SQL
• LLMs
• Evaluation Frameworks
• Quality Metrics
• A/B Testing
• Data Analysis
• Agentic Workflows
• Failure-Mode Analysis
• Production Traces

Soft Skills

• Problem-Solving
• Communication
• Stakeholder Engagement
• Adaptability
• Collaboration

Industry Keywords

• Generative AI
• MLOps
• Data Security
• Enterprise AI
• Customer-Facing Experience

Tools & Technologies

• AWS
• Azure
• GCP
• Pandas
• NumPy
• Snowpark
• Braintrust
• LangSmith
• Arize
• Promptfoo