AI Engineer – Conversational AI

Liberty Personnel Services, Inc.; Philadelphia (Philadelphia County); On-site; USD 124,000 - 179,000; Full time

Job summary

Liberty Personnel Services, Inc. is seeking an AI Engineer to own a production voice assistant for inbound calls related to disability and absence management. You will own the full stack from data ingress to live production, focusing on speed, accuracy, and safety in a regulated setting.

You will work with Amazon Connect, Lex V2, Bedrock, Python, and core AWS services to optimize latency, reliability, and guardrails while scaling the agentic conversational system for real customers.

Qualifications

• Hands-on production AI experience delivering at least one LLM-powered application to real users.

Responsibilities

• Triage and resolve production incidents and support tickets for the live pilot.
• Review transcripts, tune prompts, intents, and fallbacks from real conversations.
• Lead redesign from hybrid scripted/AI flows to fully agentic conversations.
• Drive latency reductions across voice interactions.
• Build test sets and evaluation suites to measure accuracy, hallucination, and latency.
• Run evaluation gates before releases to protect quality.
• Implement safety and compliance guardrails including PII/PHI redaction.
• Ensure robust audit logging and change-control alignment.
• Deliver via Azure DevOps with stories, acceptance criteria, PRs, and code reviews.
• Manage multi-stage pipelines moving bots through Dev to QA to Prod.
• Create monitoring dashboards, alerts, and incident playbooks for the AI assistant.
• Track and optimize AI token usage and AWS Connect consumption costs.

Skills

Production AI experience

Prompt engineering

Amazon Connect / Lex V2

Bedrock or similar LLM service

Python

AWS knowledge

Data privacy handling

Change control / audit trails

Tools

Amazon Connect

Amazon Lex V2

Amazon Bedrock

AWS Lambda

DynamoDB

S3

CloudWatch

KMS

Job description

AI Engineer – Conversational AI – Own a Production Voice Agent End-to-End

Philadelphia, PA

Take the mic on a live AI voice assistant

You’ll own a conversational AI voice assistant that’s actually talking to real customers — answering live inbound calls for disability and absence management.

Callers speak naturally. The assistant verifies who they are. Then it pulls from a knowledge base and gives them answers on the spot.

This isn’t R&D. This is build, ship, and keep it rock-solid in production.

You’ll support the live pilot, drive the shift to a fully agentic design with tighter latency, and harden it for scale and new features through a real engineering and release process.

If you’ve shipped LLM-powered apps into production and kept them running, we want to talk to you.

What You'll Work On

You’ll be the hands‑on engineer owning the full stack of this AI voice experience:

• Conversational AI for inbound customer calls
• Identity verification and secure Q&A flows
• Hybrid scripted + AI flows evolving to fully agentic conversations
• Latency optimization for real‑time voice interactions
• Automated evaluation of LLM quality and safety
• Production monitoring, incident response, and cost control
• Regulated data handling with auditability and guardrails
• Multi‑environment CI/CD and change management

Under the hood you’ll work with Amazon Connect, Amazon Lex V2, Amazon Bedrock, ElevenLabs, Python, and core AWS services.

Your mission: keep the voice assistant fast, accurate, and safe while you evolve it into a truly agentic system.

What You'll Be Doing

• Supporting the live pilot by triaging and resolving production incidents and support tickets
• Reviewing call transcripts and tuning prompts, intents, and fallback handling based on real conversations
• Leading the redesign from a hybrid scripted/AI flow to a fully agentic conversational experience
• Driving measurable latency reductions across the voice interaction
• Building test sets and automated evaluation suites to measure answer accuracy, hallucination, and response time
• Running evaluation gates before every release to protect quality
• Implementing safety and compliance guardrails, including PII/PHI protection, redaction, and prompt‑injection defenses
• Ensuring robust audit logging and alignment with formal change‑control processes
• Delivering through Azure DevOps with stories, acceptance criteria, pull requests, and code reviews
• Managing multi‑stage pipelines that promote bots and contact flows from Dev to QA to UAT to Prod
• Creating monitoring dashboards, alerts, and incident playbooks for the AI assistant
• Tracking and optimizing AI token usage and Amazon Connect consumption costs

What We're Looking For

• Hands‑on production AI experience delivering at least one LLM‑powered application to real users
• Comfort with prompt engineering, tool/function calling, structured output, and RAG
• Proven work with Amazon Connect and Amazon Lex V2, including contact flows, Lambda integrations, intents, slots, and fallbacks
• Experience with Amazon Bedrock or a comparable managed LLM service (knowledge bases, agents, guardrails)
• Background building evaluation suites for LLM output and tracking accuracy and latency over time
• Strong Python skills plus working knowledge of AWS Lambda, IAM, DynamoDB, S3, CloudWatch, and KMS
• Experience handling sensitive personal data in a regulated environment (insurance, healthcare, financial services, or similar)
• Comfort operating under formal change control, audit trails, and separation of duties

The Experience That Will Really Get Our Attention

You’ve been the go‑to engineer for a production conversational system, not just a hackathon bot. You understand where scripted flows end and agentic architectures shine, and you’ve cut real latency out of voice or chat experiences.

Ideally you’ve mixed AI, telephony, and DevOps: building with Connect and Lex, wiring Bedrock or similar LLM services, and shipping through pipelines with hard quality gates in a regulated setting.

Why This Opportunity?

This is not "help us explore AI." The system is built, the pilot is going live, and you will own it after launch.

You’ll see your work immediately reflected in real customer calls — quality, speed, safety, and scalability are all on you.

You’ll work across AI, telephony, and cloud infrastructure in a regulated environment, sharpening skills that are in rare demand and high value.

This is a contract role with an initial 6‑month engagement and the possibility of extension.

Work hours must align with US Eastern business hours, or have significant overlap with US Eastern mornings through 12 PM ET.