AI SME

DataDrone Inc.

United States

Remote

USD 140,000 - 230,000

Full time

3 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

DataDrone Inc. is seeking an experienced AI platform expert to own the production LLM workflow design.

You will shape prompts, agent usage, and retrieval strategies to ensure accurate, bounded, and measurable outcomes in client deployments. You will lead the creation of guardrails and evals, diagnose difficult LLM issues, and collaborate with Integration Engineers to ship production-ready solutions that stand up to real-world use.

Qualifications

  • Deep experience in prompt engineering for production systems.
  • Hands-on with agents and tool-use patterns.
  • Proven ability to design guardrails and evals.
  • Strong production LLM workflow design experience.
  • Ability to set a concrete acceptance bar for validation.

Responsibilities

  • Own the technical design of the production LLM workflow end to end.
  • Engineer prompts, agent/tool-use patterns, and retrieval for accuracy and reliability.
  • Design and configure guardrails so the workflow stays bounded and safe in production.
  • Build eval suites that make accuracy and quality measurable, not anecdotal.
  • Partner with the Integration Engineer to get the design live in the client's stack.
  • Set the acceptance bar with QA; define what "good enough to ship" means concretely.
  • Diagnose and solve hard LLM behavior problems; document design decisions.

Skills

Prompt engineering
Agents & tools
LLM APIs
Guardrails design
Eval frameworks
Production workflows

Job description

DataDrone runs partner implementation practices that ship one outcome in 45 days. The AI practice gets one production AI workflow live in 45 days on frontier LLMs - past the pilot graveyard, into real use. Typical outcome: roughly 80% of a manual process automated and running.


We sell outcomes, never licenses. No shelfware, no drift. We prove the work with screenshots, not poetry - and that means workflows that hold up in production, not demos that fall over. The work is the conversation. No surprises.


What You'll Do

You're the AI platform domain expert. You own the technical design of the production AI workflow: how we prompt, where we use agents and tool-use, how we wire LLM APIs, what guardrails we set, and how we prove it with evals. You're the person who knows what the model does well, where it breaks, and how to design around both.


You translate a locked use case into a design that survives production - accurate, bounded, and measurable. You set the eval bar so QA has something concrete to validate against. When the team hits a hard LLM problem, you're the one who solves it.


Daily Rhythm (How You Operate)

You start the day on the active design problems: the prompt that needs tightening, the agent loop that needs bounding, the guardrail that needs to hold. You spend deep blocks designing and testing - prompts, tool-use schemas, retrieval strategy, eval suites - and pairing with the Integration Engineer to get the design into the real stack. You set the accuracy and guardrail bar with QA so "done" is measurable, not a vibe. When the model behaves in a way the client didn't expect, you diagnose it and design the fix. You close the day with design decisions documented and the eval picture honest. No drift, no hand-waving.


Key Responsibilities


  • Own the technical design of the production LLM workflow end to end.

  • Engineer prompts, agent/tool-use patterns, and retrieval for accuracy and reliability.

  • Design and configure guardrails so the workflow stays bounded and safe in production.

  • Build eval suites that make accuracy and quality measurable, not anecdotal.

  • Partner with the Integration Engineer to get the design live in the client's stack.

  • Set the acceptance bar with QA; define what "good enough to ship" means concretely.

  • Diagnose and solve hard LLM behavior problems; document design decisions.


What We're Looking For


  • Real depth in prompt engineering - getting reliable, bounded behavior out of frontier LLMs.

  • Hands-on command of agents and tool-use patterns and LLM APIs.

  • Proven skill designing guardrails and building evals to measure quality.

  • Strong sense for where LLMs break and how to design workflows that hold in production.

  • The ability to set a concrete acceptance bar others can validate against.


Bonus Points For


  • Direct experience shipping LLM workflows to production, not just prototypes.

  • Familiarity with retrieval, context engineering, and structured output patterns.

  • Experience defining and running eval frameworks at scale.


The Mindset We Hire For

You design for production, not for the demo. You assume the model will surprise you and you build guardrails and evals so it can't surprise the client. You set a measurable bar because "it looks good" isn't a standard. You'd rather ship one workflow that holds than ten that impress in a sandbox and break in week two.


Success in This Role


  • 30 days - You own the technical design of active engagements and have a clear eval and guardrail approach for each.

  • 60 days - Your designs are live in real stacks with measurable accuracy, bounded behavior, and evals QA can run.

  • 90 days - You've shipped production LLM workflows that hold up, set a repeatable design-and-eval pattern for the practice, and become the person the team routes hard LLM problems to.


Tools You'll Use

LLM APIs and the model console; prompt and eval frameworks; agent/tool-use and retrieval tooling; the practice's AI Copilot; Notion (design docs, prompt and eval libraries); Trello / Jira / Linear for build tracking; CRM (read access for client context).


Why Join Us

You do the deep LLM work and you do it where it actually ships - to production, in 45 days, in front of real users. No prototype theatre, no shelfware. You set the standard for how the practice designs AI workflows, and you prove it with screenshots, not poetry. The hard problems land on your desk because you're the one who can solve them.


Self-Check Before You Apply


  • I can get reliable, bounded behavior out of frontier LLMs.

  • I'm fluent in agents, tool-use, and LLM APIs.

  • I design guardrails and build evals as a matter of course.

  • I know where LLMs break and design workflows that survive production.

  • I set a concrete, measurable acceptance bar others can validate.

  • I'd rather ship one workflow that holds than ten that only demo well.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Staff Engineer, Applied ML
Staff Engineer, Applied ML

Circuit • Jackson (TN)

On-site
USD 150,000 - 230,000
Staff Engineer, Applied ML
Staff Engineer, Applied ML

Circuit • Austin (TX)

On-site
USD 180,000 - 270,000
Customer Solutions Engineer
Customer Solutions Engineer

Wand AI • Palo Alto (CA)

On-site
USD 130,000 - 170,000
Forward Deployed AI Engineer
Forward Deployed AI Engineer

Malleable • United States

Remote
USD 150,000 - 180,000
QA AI Automation Engineer
QA AI Automation Engineer

Dynasty Financial Partners • Saint Petersburg (FL)

On-site
USD 120,000 - 150,000
AI Engineer
AI Engineer

Valsoft Corporation • United States

Remote
USD 150,000 - 210,000
Backend Engineer (AI-Native) — Risk Infrastructure & Integrations
Backend Engineer (AI-Native) — Risk Infrastructure & Integrations

Coris • Palo Alto (CA)

On-site
USD 150,000 - 210,000
Equity
Office in Palo Alto
Builder Product Manager I
Builder Product Manager I

Werkenbijecisolutions • United States

Remote
USD 120,000 - 180,000
Machine Learning Engineer, Applied AI
Machine Learning Engineer, Applied AI

Brain Co. • San Francisco (CA)

On-site
USD 140,000 - 210,000
Machine Learning Engineer, Applied AI
Machine Learning Engineer, Applied AI

Brain Co. • New York (NY)

On-site
USD 140,000 - 210,000