Senior AI Engineer — LLM Agents & Production Systems (Remote)

Smartsheet; United States; Remote; USD 175,000 - 245,000; Full time

Benefits offered by this job

Medical/Dental/Vision coverage

401k match

Parental leave (up to 24 weeks)

Volunteer day

Udemy courses

Teleworking options

Job summary

Smartsheet is seeking an experienced AI/ML engineer to design and operationalize LLM-powered agents at enterprise scale. You will work with Engineering and AI Platform teams to develop prompts, RAG pipelines, and evaluation frameworks, while delivering end-to-end AI systems on cloud infrastructure and mentoring peers.

You will shape how we design, build, and govern AI solutions, collaborate across functions, and stay current with the evolving AI landscape to deliver measurable impact across

Qualifications

• 8+ years of software engineering, with at least 2 years working directly with LLMs in production.
• Deep, hands-on experience with prompt engineering and context engineering; understand how model behavior changes with framing, structure, and input design.
• Strong working knowledge of RAG architectures: chunking strategies, embedding models, retrieval evaluation, and failure diagnosis.
• Experience building or extending LLM evaluation frameworks and designing scorers with golden datasets.
• Fluency in agent system design and ability to discuss architectural tradeoffs that affect quality.
• Strong Python skills and comfort with data-heavy environments such as Databricks or Delta tables.

Responsibilities

• Design and build production-grade LLM-powered agents and workflows, including architecture decisions on system design and data pipelines.
• Develop prompts, RAG pipelines, and agent reasoning patterns; build evaluation frameworks to measure accuracy and performance across model versions.
• Implement and manage end-to-end AI systems on cloud infrastructure (AWS, GCP, Azure), including monitoring and incident response.
• Collaborate with Engineering, Product, and cross-functional teams to translate business requirements into technical AI solutions.
• Mentor engineers on applied ML best practices and production considerations.
• Establish best practices for AI development, model governance, and responsible AI.
• Stay current with AI/LLM landscape and evaluate new tools and frameworks.
• Contribute to a full stack application, legacy and greenfield.

Skills

LLM in production

Prompt engineering

RAG architectures

Python

Data pipelines

Model evaluation

Stakeholder communication

Education

CS degree

Equivalent experience

Tools

Databricks

Delta tables

MLflow

CI pipelines

Multi-agent frameworks

Job description

Smartsheet is seeking an experienced AI/ML engineer to design and operationalize LLM-powered agents at enterprise scale. You will work with Engineering and AI Platform teams to develop prompts, RAG pipelines, and evaluation frameworks, while delivering end-to-end AI systems on cloud infrastructure and mentoring peers.

You will shape how we design, build, and govern AI solutions, collaborate across functions, and stay current with the evolving AI landscape to deliver measurable impact across