Senior Software Engineer, AI Platform (Remote)

weedmaps77; United States; Remote; USD 197,000 - 222,000; Full time

Benefits offered by this job

Medical, dental and vision coverage

Job summary

weedmaps77 is seeking a senior software engineer to design, build, and operate production AI systems powering internal tooling and automation. You will lead an autonomous coding agent that handles issue trackers, pull requests, and review feedback, and you’ll shape a retrieval-and-context layer that unifies documentation and data.

You will own end-to-end delivery, measure impact with robust evals, and drive cost and latency improvements through caching and model selection, all while hardening

Qualifications

• 6+ years of professional software development with end-to-end ownership of production services.
• 1+ year of hands-on experience shipping LLM-powered features or agents to production, including tool/function calling and structured outputs.
• Experience building retrieval systems: embeddings, vector or hybrid search, chunking and ranking.
• Strong proficiency in at least one of Python, TypeScript, or Ruby, across a polyglot codebase.
• Experience with a major cloud platform (AWS preferred), CI/CD, and observability tooling.
• A track record of measuring what you build with evals, A/B tests, or clear before-and-after metrics.

Responsibilities

• Design, build, and operate production AI agents, including an autonomous coding agent that works from issue trackers and chat, opens pull requests, fixes CI failures, and applies review feedback.
• Build the retrieval and context layer behind the AI platform: embeddings, semantic search, knowledge-graph modeling, and integration with the existing semantic data layer and internal documentation.
• Extend the shared agent platform with reusable skills, tools, and processes for product and engineering teams.
• Define and run the evaluation program: offline test sets and online metrics, gate releases based on impact.
• Drive down cost and latency by routing work to the most economical capable model, using caching and structured outputs, and tracking spend per agent and per task.
• Harden agents against prompt injection, memory poisoning, and over-broad permissions through least-privilege tool access, auditable actions, and human approval where risk warrants it.
• Instrument agents with tracing and logging, participate in on-call for owned systems, and mentor other engineers on building with LLMs.

Skills

Python

TypeScript

Ruby

Education

Bachelor's degree or equivalent practical experience

Tools

AWS

CI/CD

Observability tooling

Job description

**Role overview**

A remote opening for a senior engineer to build the production AI systems an engineering organization runs on. The work spans an autonomous coding agent that picks up tickets, opens pull requests, and handles review feedback; a shared internal platform that gives every agent common skills and tools; and a retrieval-and-context layer that unifies documentation and a semantic data layer. Success means agents that are measurably useful, economical at scale, and safe to trust with production code.

**Responsibilities**

• Design, build, and operate production AI agents, including an autonomous coding agent that works from issue trackers and chat, opens pull requests, fixes CI failures, and applies review feedback.
• Build the retrieval and context layer behind the AI platform: embeddings, semantic search, knowledge-graph modeling, and integration with the existing semantic data layer and internal documentation.
• Extend the shared agent platform with reusable skills, tools, and processes that product and engineering teams can adopt in their own workflows.
• Define and run the evaluation program: build offline test sets and online metrics, and gate releases on whether a change made an agent better or worse.
• Drive down cost and latency by routing work to the most economical capable model, using caching and structured outputs, and tracking spend per agent and per task.
• Harden agents against prompt injection, memory poisoning, and over-broad permissions through least-privilege tool access, auditable actions, and human approval where risk warrants it.
• Instrument agents with tracing and logging, participate in on-call for owned systems, and mentor other engineers on building with large language models.

**Requirements**

• Bachelor's degree or equivalent practical experience.
• 6+ years of professional software development, including end-to-end ownership of production services.
• 1+ year of hands-on experience shipping LLM-powered features or agents to production, covering tool/function calling, prompt design, structured outputs, and graceful handling of model failures.
• Experience building retrieval systems: embeddings, vector or hybrid search, chunking and ranking, and measuring retrieval quality.
• Strong proficiency in at least one of Python, TypeScript, or Ruby, with comfort working across a polyglot codebase.
• Experience with a major cloud platform (AWS preferred), CI/CD, and observability tooling.
• A track record of measuring what you build, using evals, A/B tests, or clear before-and-after metrics.

**Nice to have**

• AWS Bedrock and managing model cost and latency at scale.
• Model Context Protocol servers or other agent tool frameworks.
• Knowledge graphs or semantic layers (e.g., semantic views, dbt metrics, graph databases).
• Agents acting on developer workflows: GitHub Actions, issue tracker and chat integrations, automated code review or code generation.
• LLM security: prompt-injection defenses, sandboxing, permission models, and red-teaming.
• Entity matching or classification at large scale using ML combined with LLM reasoning.
• Experience with Ruby on Rails, Elixir, or Node.js services, or at a marketplace, e-commerce, or regulated-industry company.

**Benefits and work setup**

Remote role. Base pay range of $196,984 to $222,000 per year plus a competitive bonus. Listed benefits include employer-paid premiums for employee medical, dental, and vision coverage (full U.S. benefits package details available in the original posting).