Senior Software Engineer - AI Innovation
Worth AI
Turn this role into an interview — a resume and cover letter built around what this employer wants.
Benefits offered by this job
Job summary
Worth AI is seeking a Senior Software Engineer - AI Innovation to design and ship production-grade agent systems for regulated financial data. You will own end-to-end lifecycle from architecting agent graphs to deploying against guardrails.
You will work with security, compliance, and legal to maintain SOC 2 / GDPR / CCPA posture, and mentor engineers while advancing production MLOps and eval discipline in a fast-paced, ownership-driven environment.
Qualifications
- 8+ years of professional software engineering experience, with at least 2 years building production LLM or agentic systems.
- Solid software engineering experience - front‑end, APIs, async patterns, queues, databases, and the failure modes of distributed systems.
- Demonstrated ownership of major features or subsystems in production.
- Mentoring junior engineers and raising team quality standards.
- Experience with event‑driven systems: enrichment, retries, dead‑lettering, backpressure.
- Experience managing containerized applications in Kubernetes, EKS, ArgoCD, operators, Kustomize.
- Hands‑on experience with at least one modern agent framework (LangGraph strongly preferred).
- Track record of shipping agents that run, fail gracefully, and recover.
- Experience with evals: golden sets, offline/online evaluations, and ship/no‑ship decisions.
- Production MLOps fluency: deploying LLM workloads with latency, cost and reliability constraints; instrument what you ship.
- Strong proficiency in Python; comfortable with TypeScript/Node.js for integration.
Responsibilities
- Design and ship multi-step agentic systems that automate KYB, underwriting, case review, and risk monitoring workflows.
- Architect agent graphs in LangGraph (or comparable frameworks) with explicit state, durable execution, retries, and safe fallbacks.
- Build and harden the retrieval layer powering our agents chunking strategies, hybrid search, reranking, and grounded citations across diverse data sources.
- Own the eval stack: golden sets, offline regression suites, LLM‑as‑judge, online A/B and shadow evals, and red‑teaming for jailbreaks and PII leakage.
- Wire agents into Worth’s production systems via well‑typed tools, MCP servers, and existing services; treat tool surface area as a product.
- Drive production MLOps for agents: deployment, versioning, traffic shaping, cost/latency budgets, observability, and on‑call playbooks.
- Partner with security, compliance, and legal to keep agents within SOC 2, GDPR, CCPA, and fair‑lending posture.
- Translate ambiguous product bets into concrete agent designs, prototypes, and shipped features.
- Mentor engineers on agent patterns, prompt hygiene, eval discipline, and failure modes of LLMs.
- Stay ahead of frontier models and patterns and bring back what works in production.
Skills
Tools
Job description
Worth AI, a leader in AI onboarding and underwriting, is looking for a talented and experienced Senior Software Engineer - AI Innovation to join our team. At Worth AI, we are on a mission to revolutionize decision-making with the power of artificial intelligence helping fintechs, lenders, payment processors, and financial institutions onboard small businesses faster, smarter, and more confidently. We’re building the infrastructure that powers real-time KYB, KYC/IDV, underwriting, and continuous risk monitoring at enterprise scale, and the Worth Score™ our unified credit score derived from 1,200+ data points across 700M+ SMBs.
As a Senior Software Engineer - AI Innovation, you won’t be wiring up demos you’ll be designing and shipping production agent systems that make consequential decisions on regulated financial data. Worth’s platform consolidates onboarding and underwriting into a single AI-powered system, and our agents read, reason over, and act on the messy, high‑stakes signals that come with that domain. You’ll own the end‑to‑end lifecycle: architecting agent graphs, building the retrieval and tool layers they rely on, instrumenting them with evals and observability, and getting them deployed against SOC 2 / GDPR / CCPA guardrails. You’ll partner closely with our Chief AI Officer, applied scientists, product, and platform teams to turn agentic patterns into customer outcomes.
- Design and ship multi‑step agentic systems (planner/executor, tool‑using, multi‑agent, human‑in‑the‑loop) that automate KYB, underwriting, case review, and risk monitoring workflows.
- Architect agent graphs in LangGraph (or comparable frameworks CrewAI, AutoGen, Claude Agent SDK) with explicit state, durable execution, retries, and safe fallbacks.
- Build and harden the retrieval layer powering our agents chunking strategies, hybrid search, reranking, and grounded citation across SoS filings, IRS records, bank data, and Worth’s 700M+ SMB graph.
- Own the eval stack: golden sets, offline regression suites, LLM‑as‑judge, online A/B and shadow evals, and red‑teaming for jailbreaks, prompt injection, and PII leakage.
- Wire agents into Worth’s production systems via well‑typed tools, MCP servers, and existing services (decisioning engine, case management, crosswalking). Treat tool surface area as a product.
- Drive production MLOps for agents: deployment, versioning, traffic shaping, cost/latency budgets, observability (traces, token spend, tool call success), and on‑call playbooks for agent incidents.
- Partner with security, compliance, and legal to keep agents inside Worth’s SOC 2, GDPR, CCPA, and fair‑lending posture — building from day one, not bolted on.
- Translate ambiguous product bets (“what if the underwriter had an AI co‑pilot for this?”) into concrete agent designs, prototypes, and shipped features.
- Mentor engineers across the org on agent patterns, prompt engineering hygiene, eval discipline, and the failure modes of LLM systems.
- Stay ahead of the frontier new models, frameworks, and patterns — and bring back what actually works in production.
Technology Stack
- Languages & Runtimes: Python, Node.js, TypeScript
- Agent / LLM frameworks: LangGraph, LangChain, Claude Agent SDK, MCP, OpenAI SDK
- Models: Anthropic Claude, OpenAI, open-weight (Llama, Mistral) where appropriate
- Retrieval & Data: PostgreSQL, pgvector / vector DBs, OpenSearch, Kafka, Redshift, Redis
- Infra & Orchestration: AWS, Kubernetes (EKS), ArgoCD, Terraform
- Evals & Observability: LangSmith / Langfuse / Braintrust‑style tooling, DataDog, custom eval harnesses
- 8+ years of professional software engineering experience, with at least 2 years building production LLM or agentic systems (not just notebooks or demos).
- Solid software engineering experience - front‑end, APIs, async patterns, queues, databases, and the failure modes of distributed systems.
- Demonstrated ownership of major features or subsystems in production.
- Demonstrated experience mentoring junior engineers and raising team quality standards.
- Demonstrated experience with event‑driven systems: enrichment, retries, dead‑lettering, backpressure.
- Experience managing containerized applications in Kubernetes, EKS, ArgoCD, operators, Kustomize.
- Deep, hands‑on experience with at least one modern agent framework (LangGraph strongly preferred) and a track record of shipping agents that actually run, fail gracefully, and recover.
- Real experience with evals you’ve built golden sets, run offline and online evaluations, and used them to make ship/no‑ship calls.
- Production MLOps fluency: you’ve deployed LLM workloads under real latency, cost, and reliability constraints, and you instrument what you ship.
- Strong proficiency in Python; comfortable in TypeScript / Node.js for integrating with Worth’s services.
- Clear, calibrated communicator - able to explain agent trade‑offs to product, security, and customers without hand‑waving.
- Operates with extreme ownership in ambiguous, fast‑moving environments. Excited to work alongside a team that values “One Team”, “Extreme Ownership”, and “Create Raving Fans.”
Success Metrics
- Agent Quality: Measurable improvements in task success rate, grounding accuracy, and hallucination rate on Worth’s eval suites, tied to customer‑visible outcomes.
- Production Reliability: Agents you own meet defined SLOs for latency (P90/P99), tool‑call success rate, and cost per task.
- Velocity: New agent capabilities go from prototype to production in weeks, not quarters, without skipping evals or guardrails.
- Risk Posture: Zero material incidents tied to prompt injection, PII leakage, or unsafe tool use on agents you own.
- Force Multiplier: Patterns, tools, and eval scaffolding you build were adopted by other engineers across Worth.
Bonus Points (nice to haves, not requirements)
- Prior experience in fintech, lending, payments, KYB/KYC, fraud, or AML — or any other regulated, high‑stakes data domain.
- Experience building MCP servers or other structured tool interfaces for LLMs.
- Background in classical ML (ranking, scoring, calibration) you can bring to bear alongside LLM systems.
- Experience designing explainable / auditable AI workflows for regulated environments (SOC 2, model risk management, fair lending).
- Open‑source contributions to agent frameworks, eval tooling, or retrieval libraries.
- Hands‑on AWS depth (EKS, MSK, RDS, S3, Lambda) and IaC with Terraform.
**All Remote Hires — will be required to travel to Orlando, Florida at least twice per year for Town Halls and team collaboration, in addition to orientation in Orlando, Florida.
- Health Care Plan (Medical, Dental & Vision)
- Retirement Plan (401k, IRA)
- Life Insurance
- Flexible Paid Time Off
- 9 paid Holidays
- Family Leave
- Remote
- Hybrid work (for Orlando Associates)
- Free Food & Snacks (Orlando)
- Wellness Resources
'/%3e%3cpath%20d='M101.744%2012.4726L102.894%2016.1947L99.0021%2013.2201L101.744%2012.4726ZM105.3%208.40937H100.488L99.0026%203.59473L97.5116%208.41047L92.7%208.40385L96.5965%2011.3834L95.1055%2016.1942L99.0021%2013.2196L101.408%2011.3834L105.3%208.40992V8.40937Z'%20fill='white'/%3e%3cdefs%3e%3clinearGradient%20id='paint0_linear_5593_1381'%20x1='90'%20y1='9.44471'%20x2='99'%20y2='9.44471'%20gradientUnits='userSpaceOnUse'%3e%3cstop%20stop-color='%2300AD70'/%3e%3cstop%20offset='0.998594'%20stop-color='%2300B67A'/%3e%3cstop%20offset='1'%20stop-color='%23DCDCE6'/%3e%3c/linearGradient%3e%3c/defs%3e%3c/svg%3e){kind=small}
