Senior AI Systems Engineer — Remote, Production-Ready

weedmaps77; United States; Remote; USD 197,000 - 222,000; Full time

Benefits offered by this job

Medical, dental and vision coverage

Job summary

weedmaps77 is seeking a senior software engineer to design, build, and operate production AI systems powering internal tooling and automation. You will lead an autonomous coding agent that handles issue trackers, pull requests, and review feedback, and you’ll shape a retrieval-and-context layer that unifies documentation and data.

You will own end-to-end delivery, measure impact with robust evals, and drive cost and latency improvements through caching and model selection, all while hardening

Qualifications

• 6+ years of professional software development with end-to-end ownership of production services.
• 1+ year of hands-on experience shipping LLM-powered features or agents to production, including tool/function calling and structured outputs.
• Experience building retrieval systems: embeddings, vector or hybrid search, chunking and ranking.
• Strong proficiency in at least one of Python, TypeScript, or Ruby, across a polyglot codebase.
• Experience with a major cloud platform (AWS preferred), CI/CD, and observability tooling.
• A track record of measuring what you build with evals, A/B tests, or clear before-and-after metrics.

Responsibilities

• Design, build, and operate production AI agents, including an autonomous coding agent that works from issue trackers and chat, opens pull requests, fixes CI failures, and applies review feedback.
• Build the retrieval and context layer behind the AI platform: embeddings, semantic search, knowledge-graph modeling, and integration with the existing semantic data layer and internal documentation.
• Extend the shared agent platform with reusable skills, tools, and processes for product and engineering teams.
• Define and run the evaluation program: offline test sets and online metrics, gate releases based on impact.
• Drive down cost and latency by routing work to the most economical capable model, using caching and structured outputs, and tracking spend per agent and per task.
• Harden agents against prompt injection, memory poisoning, and over-broad permissions through least-privilege tool access, auditable actions, and human approval where risk warrants it.
• Instrument agents with tracing and logging, participate in on-call for owned systems, and mentor other engineers on building with LLMs.

Skills

Python

TypeScript

Ruby

Education

Bachelor's degree or equivalent practical experience

Tools

AWS

CI/CD

Observability tooling

Job description

weedmaps77 is seeking a senior software engineer to design, build, and operate production AI systems powering internal tooling and automation. You will lead an autonomous coding agent that handles issue trackers, pull requests, and review feedback, and you’ll shape a retrieval-and-context layer that unifies documentation and data.

You will own end-to-end delivery, measure impact with robust evals, and drive cost and latency improvements through caching and model selection, all while hardening