Software Engineering SMTS / LMTS - AI Core (REMOTE)

Enboarder; San Francisco (CA); Remote; USD 180,000 - 230,000; Full time

Job summary

Qualified is seeking an AI Core engineer to lead end-to-end retrieval and agent infrastructure. You will design, build, and operate retrieval systems using RAG and hybrid search, and craft tooling for production agents with emphasis on reliability and scalability.

You will own platform initiatives from design through adoption, mentor engineers, and help define data models for content ingestion and retrieval. Strong backend skills in Ruby, Python, JS/TS, and SQL are important.

Qualifications

• Shipping production AI or backend systems in high-ownership environments.
• Expertise building retrieval systems with RAG and hybrid search.
• Experience owning platform initiatives from design to adoption.
• Strong backend fundamentals with Ruby on Rails, Python, JavaScript, or Go.
• Proficiency with SQL and PostgreSQL.

Responsibilities

• Lead end‑to‑end retrieval initiatives using RAG and hybrid search.
• Build developer tooling for production agents and orchestration.
• Own answer quality by diagnosing retrieval vs. generation issues.
• Define content and data models for ingestion and retrieval.
• Add safe shipping practices with evaluation, observability, and rollout.

Skills

AI/ML systems

Retrieval systems

RAG search

Developer tooling

Architecture leadership

Backend engineering

Ruby on Rails

Python

JavaScript

TypeScript

SQL

PostgreSQL

Go

Vector indexes

Tools

pgvector

Vector indexes

Job description

Engineering at Qualified

Engineering at Qualified is a high-ownership, high-velocity environment where engineers ship customer-facing product to real enterprises at scale. We're building at the frontier of AI + product: developing agentic software and embedding AI directly into our platform, using modern toolchains and processes that help engineers move faster while staying focused on high-impact problem solving.

Following our acquisition by Salesforce, we continue to operate with the pace and ownership of a startup, backed by the distribution, resources, and long-term runway of a global enterprise. Engineers have true end-to-end ownership from idea to production: scoping, implementation, testing, deployment, monitoring, and iterative improvement based on real customer feedback.

You'll work alongside a high-talent-density team that cares deeply about craft, performance, reliability, and user experience, and that raises the bar through collaboration, code review, and strong engineering principles. You'll make architecture decisions, solve meaningful system design challenges, and see your work land quickly with customers, without layers of approvals slowing you down.

The Role

As an Engineer on the AI Core team, you'll operate as a technical leader who turns ambiguous, high-stakes AI problems into durable, high-quality systems, while raising the technical bar for the broader team. AI Core builds the retrieval systems and agent infrastructure that power how our AI agents find, trust, and use knowledge, and the developer tooling that lets engineers across the company ship production agents quickly and safely.

You'll lead significant initiatives end to end: faithful answering over a customer's own knowledge, retrieval that combines RAG and hybrid search, content lifecycle controls such as freshness, deduplication and source precedence, transcript processing, and the frameworks, registries, and evaluation tooling that make building a production agent a paved path. You'll partner closely with product, design, and engineering leadership to decide what we build, how we build it, and how we measure whether answers actually got better.

What You'll Do

Lead retrieval initiatives end to end: Design, build, and operate retrieval systems using RAG and hybrid search (lexical plus vector, reranking, filtering) so agents answer from the right content with the right amount of context. Own delivery from scope and design through launch, monitoring, and iteration.

Build developer tooling for production agents: Drive technical design for the frameworks and platform pieces our engineers use to build agents, including multi-agent orchestration, shared primitives for prompts, tools, tracing, and evaluation.

Own answer quality: Diagnose why answers are thin, wrong, or inconsistent by separating retrieval problems from generation problems (prompt, token budget, model choice). Set a quality bar we can test against, and lead the work to reach it.

Design content and data models: Define how content from documents, knowledge bases, and call transcripts is ingested, segmented, labeled, and retrieved, including the primitives behind freshness, dedupe, source precedence, and internal versus external visibility.

Make AI changes safe to ship: Build evaluation, observability, and rollout practices (offline evals, online monitoring, regression detection) so we can change prompts, models, and retrieval with confidence. Evaluate multiple solution paths and know when to trust, redirect, or override model and agent output.

Set performance and scaling benchmarks across the platform we own:

Define and track targets for retrieval and indexing, LLM provider usage, and agent orchestration.

Own operational health: Ensure the reliability and performance of the systems your team owns. Participate in on‑call, lead incident resolution for your area, and communicate clearly with the team and stakeholders during incidents.

Use agentic development to raise leverage: Use coding agents by default for implementation, testing, refactoring, and documentation while holding a high validation bar.

Create reusable workflows, patterns, and tooling that raise the effectiveness of your team beyond your own output.

Partner across functions: Work with product, design, and engineering leadership to shape roadmap, tradeoffs, and requirements, and to translate strategy into concrete execution plans.

Raise team effectiveness: Mentor engineers, give high-quality code review and technical guidance, and improve standards, tooling, and processes.

Who You Are

Proven experience shipping production AI or backend systems in a high ownership environment, with a recent focus on LLM‑powered applications.

Expertise building retrieval systems using RAG and hybrid search, including chunking and indexing strategy, embeddings, vector and lexical search, reranking, and measuring retrieval quality.

Expertise building developer tooling that lets developers build production agents, which can include frameworks for multi‑agent orchestration, an agent and tool registry, and shared infrastructure for tracing and evaluation.

Experience owning platform initiatives from design through adoption, across systems such as retrieval and indexing, LLM provider integrations, and agent orchestration, including driving adoption and evolving the platform as usage grows.

Demonstrated technical leadership: you lead significant projects, design solutions for non‑trivial problems by weighing tradeoffs, and influence architecture and execution beyond your immediate workstream.

Strong track record delivering complex projects with multiple stakeholders, dependencies, or phases.

Strong backend engineering fundamentals and proficiency in languages such as Ruby on Rails, Python, JavaScript, or Go, with SQL and RDBMS experience (PostgreSQL preferred).

Hands‑on experience with the AI application lifecycle: experiment setup, dataset curation, offline evaluation, error analysis, deployment, and online evaluation.

Comfortable operating in ambiguity and driving clarity: you can frame problems, propose solutions, and align others.

Strong communicator across functions, able to explain technical ideas to both technical and non‑technical audiences and to translate complex concepts into business impact.

Ownership mindset and bias for action. You raise the bar for the team through mentoring, code review, and your own work.

Curious and continuously learning, with an agent‑forward approach to how you build software.

Nice to Have

Experience with pgvector or other vector indexes, and with query performance at scale.

Experience with content ingestion from third‑party systems (document stores, knowledge bases, CRM knowledge objects) and the lifecycle problems that follow: sync state, dedupe, source conflict.

Experience processing conversation or call transcripts, including handling PII and consent constraints.

Experience designing evaluation suites that score qualities such as thoroughness and faithfulness, not only correctness.

Experience with voice or real‑time conversational agents.

Experience with enterprise SaaS (especially B2B marketing or sales tech) or Salesforce/CRM integrations.

Startup experience and rapid shipping.

Why Join Qualified

Foundational impact: Shape how our AI agents retrieve, reason, and answer, and the tooling every engineer here uses to build them.

Lead real initiatives: Own significant projects end to end, with scope and influence that grow as you deliver. You'll own the platform as it scales, from retrieval to model providers to agent orchestration, and set the performance and reliability bar that grows with the company.

AI as a core product surface: Turn new AI capabilities into dependable experiences that customers trust, with strong engineering around evaluation, observability, and iteration.

High bar, low bureaucracy: Work with a team that values craft and decisive execution, where strong ideas win and shipping is the default.

Room to grow: Take on larger technical and product scope over time: leading projects, influencing architecture, and leveling up quickly.

If you're excited to build retrieval systems and agent infrastructure at the intersection of AI and enterprise software, we'd love to hear from you.