Enterprise AI Operations Engineer — Quality & ROI

Trissential, LLC; Eyota, Northern (MN, KY); Hybrid; USD 124,000 - 138,000; Full time

Benefits offered by this job

Competitive pay and benefits

Medical, dental, vision coverage

Paid holidays and PTO

Career development opportunities

Strong team culture

Job summary

Trissential, LLC is seeking a Customer AI Operations Engineer to own the operational success of enterprise AI agents, virtual assistants, and generative AI solutions in production.

You will collaborate with product, AI engineering, operations, security, governance, and business stakeholders to deliver measurable improvements in AI quality, adoption, and ROI, while contributing to responsible AI practices and long-term standards.

Qualifications

• 5+ years in technology operations, analytics, or product operations.
• 2+ years supporting AI agents, virtual assistants, or generative AI in production.
• Experience evaluating AI responses with transcripts and quality scorecards.

Responsibilities

• Own the operational performance of AI-powered solutions after implementation.
• Monitor AI quality, response accuracy, and business outcomes.
• Develop and maintain operational scorecards and governance artifacts.
• Investigate hallucinations, retrieval failures, and prompt issues.
• Coordinate incident investigations and root-cause analysis across teams.
• Translate findings into backlog items for Product Owners and AI Eng teams.
• Measure value realization including ROI and productivity gains.

Skills

Tech operations

AI agents support

Quality scorecards

Operational analytics

Responsible AI

Stakeholder comms

Documentation

Education

Bachelor's degree in IT/CS/Analytics

Job description

Trissential, LLC is seeking a Customer AI Operations Engineer to own the operational success of enterprise AI agents, virtual assistants, and generative AI solutions in production.

You will collaborate with product, AI engineering, operations, security, governance, and business stakeholders to deliver measurable improvements in AI quality, adoption, and ROI, while contributing to responsible AI practices and long-term standards.