AI Engineer, Agents & Quality

Gold Rose AI; United States; Hybrid; USD 120,000 - 180,000; Full time

Job summary

Gold Rose AI seeks a software-focused product engineer to design and improve the agent tools, memory, and approval flows that control customer-facing AI actions. You will build evaluation datasets from anonymized owner conversations and validate every change through automated tests, ensuring accuracy and safe outputs.

You will reduce mistakes by anchoring actions to owner-approved prompts and memory rules, and track costs to keep the service affordable for small shops.

Qualifications

• 3+ years shipping software, with at least 1 year building on large language models in production.
• A habit of measuring quality instead of eyeballing it.
• Clear writing: prompts, specs and postmortems.

Responsibilities

• Build and improve agent tools, memory and approval flows.
• Create evaluation sets from real (anonymized) owner conversations and run them on every change.
• Reduce mistakes: wrong facts, made-up details, actions without permission.
• Track cost and speed so the product stays affordable for small shops.

Skills

Shipping software

Quality metrics

Clear writing

Job description

Our AI follows up with customers, drafts posts, reads documents and reports what it did. Every action is approved by the owner or pre-authorized. You'll design the agent's tools, prompts and memory, build the tests that catch mistakes before owners do, and measure whether the work it does is actually good.

What you'll do

• Build and improve agent tools, memory and approval flows
• Create evaluation sets from real (anonymized) owner conversations and run them on every change
• Reduce mistakes: wrong facts, made-up details, actions without permission
• Track cost and speed so the product stays affordable for small shops

What you bring

• 3+ years shipping software, with at least 1 year building on large language models in production
• A habit of measuring quality instead of eyeballing it
• Clear writing: prompts, specs and postmortems

Nice to have

• Tool use, structured outputs and evaluation frameworks
• Experience with AI governance or safety standards (e.g. ISO/IEC 42001)

What success looks like

In 90 days: an evaluation suite that runs on every release and a measurable drop in AI mistakes.

About Gold Rose

Gold Rose sends independent flower shops more orders through the Gold Rose Network, and offers Gold Rose AI: an AI employee that answers customers, follows up and handles the busywork, always with the owner's OK. We're a California company, launching with founding florists ahead of Valentine's Day 2027. Gold Rose AI is a trade name of Alterra Holdings LLC.

Talent network. This role is on our hiring plan. Pay range is our current expectation and is confirmed when the role opens.

More roles

Other ways to join

• Founding Account Executive, Florist Partnerships

  First hires Sales United States (hybrid) Full-time

• Florist Success Manager

  First hires Customer Success United States (hybrid) Full-time

• Senior Full-Stack Engineer

  First hires Engineering Remote (US) Full-time

• Product Designer, Mobile & Conversation

  Next wave Design Remote (US) Full-time

All roles