Staff Engineer - GenAI Platform & LLMOps

vibehackers; United States; Hybrid; USD 133,000 - 182,000; Full time

Benefits offered by this job

Dental coverage

Health & dependent care

Disability insurance

Life insurance

EAP

Retirement plan matching

Job summary

Vibehackers seeks a Staff Engineer - GenAI to design, build, and operate an enterprise-scale agentic AI platform enabling autonomous, LLM-driven solutions across the enterprise. You will provide hands-on technical leadership, drive GenAI architecture, LLMOps practices, and mentor engineers to produce scalable, secure, and production-ready agentic systems.

The role focuses on architecting LLM-driven systems, enabling autonomous multi-step agent behaviors, ensuring scalability, availability,

Qualifications

• Extensive experience building enterprise software and GenAI platforms.
• Proven track record in scale architecture and mentoring engineers.
• Strong knowledge of prompt/context engineering and agent patterns.

Responsibilities

• Own end-to-end development of the Agentic AI platform: design, build, test, and deploy generative AI capabilities and agent frameworks to support autonomous task execution.
• Provide technical direction across the organization and collaborate with solution and enterprise architects to integrate LLMs, agent frameworks, and AI services into broader systems while meeting non-functional requirements (security, scalability, resilience, token economics, latency).
• Lead coding standards, prompt, and context engineering practices; conduct reviews and establish reproducible experiment/versioning.
• Build frameworks and orchestration pipelines to integrate LLMs and agents with enterprise data sources, leveraging GenAI tools and protocols and designing retrieval, memory, and tool-use patterns.
• Drive LLMOps/GenAI Ops practices: automated evaluation, prompt/agent/versioning, observability, guardrails, CI/CD for prompts/agents/models.
• Mentor engineers in GenAI and software engineering techniques; contribute to hiring and promotions.

Skills

Technical Leadership

System Design

Prompt Engineering

LLMOps

Observability

Security & Compliance

Education

Bachelor's degree in CS/DS

Tools

LangChain

LangGraph

Semantic Kernel

Bedrock

LangSmith

Job description

Deeply focused on GenAI/agent engineering—heavy on prompt/context engineering, agent frameworks, and LLMOps; building production AI platforms and tooling.

About the Role

Staff Engineer - GenAI responsible for designing, building, and operating a large-scale agentic AI platform that enables autonomous, LLM-driven solutions across the enterprise. Provide hands-on technical leadership, drive GenAI architecture, LLMOps practices, and mentor engineers to produce scalable, secure, and production-ready agentic systems.

Job Description

Role

The Staff Engineer - GenAI is a hands-on technical leader responsible for designing, building, and maintaining an enterprise-scale agentic AI platform. The role focuses on architecting LLM-driven systems, enabling autonomous multi-step agent behaviors, ensuring scalability, availability, security, and alignment with enterprise architecture and compliance standards.

Key Responsibilities

• Own end-to-end development of the Agentic AI platform: design, develop, test, and deploy generative AI capabilities and agent frameworks to support autonomous task execution.
• Provide technical direction across the organization and collaborate with solution and enterprise architects to integrate LLMs, agent frameworks, and AI services into broader systems while meeting non-functional requirements (security, scalability, resilience, token economics, latency).
• Lead coding standards, prompt engineering, and context engineering best practices; conduct code, prompt, and context-pipeline reviews and establish reproducible experiment/versioning practices (e.g., Git and LLMOps tools).
• Build frameworks and orchestration pipelines to integrate LLMs and agents with enterprise data sources, leveraging GenAI tools and protocols (MCP, A2A, function/tool calling) and designing retrieval, memory, and tool-use patterns for reliable inference-time context.
• Drive LLMOps/GenAI Ops practices: automated evaluation, prompt/agent/versioning, online/offline evals, observability (traces, token usage, hallucination/grounding metrics), guardrails, CI/CD for prompts/agents/models, and evaluate tools like LangSmith, LangFuse, Bedrock, or cloud AI services.
• Mentor and coach engineers in GenAI and software engineering techniques; contribute non-managerial feedback on hiring and promotions.

Requirements

• 7–10+ years building and scaling enterprise software systems, with substantial experience (10+ years overall and ideally 2+ years focused on GenAI/LLM or software architecture initiatives).
• Deep understanding of generative AI and transformer models (examples: Claude, GPT) and practical experience integrating LLMs into enterprise apps, including prompt engineering, context engineering, RAG/GraphRAG, and agentic patterns (ReAct, planner/executor, multi-agent orchestration).
• Proficiency with GenAI frameworks/libraries (e.g., LangChain, LangGraph, Semantic Kernel) and familiarity with model-serving runtimes and agent orchestration frameworks.
• Cloud and deployment experience (AWS Bedrock, kore.ai , GCP AI) and knowledge of containerization and serverless architectures for scalable agent deployments.
• Strong data engineering skills for AI-ready data: chunking, enrichment, governance, vector databases (pgvector, Elastic Search), hybrid search, reranking, knowledge graphs, embeddings, semantic layers, and access controls.
• Experience with LLMOps/AI DevOps practices: CI/CD for prompts and agents, eval-driven development, versioning, observability/tracing, cost/token monitoring, automated red-teaming, and guardrail enforcement.
• Demonstrated technical leadership, mentorship, communication skills, and a strong focus on AI ethics, safety, privacy, and compliance (prompt injection mitigation, PII handling, audit logging).
• Bachelor’s degree in Computer Science or Data Science preferred.

Compensation & Other Notes

• Pay range: $132,600 - $182,250 USD (DOE).
• Eligible for a target annual bonus of 20% of base salary.
• Position is not eligible for sponsorship.

Claude GPT LangChain LangGraph Semantic Kernel LangSmith LangFuse Bedrock AWS Bedrock kore.ai GCP AI pgvector Elastic Search Splunk CloudWatch Git MCP A2A function/tool calling GraphRAG ReAct

Skills

Technical Leadership System Design Prompt Engineering Context Engineering LLMOps / GenAI Ops Observability Architecture Cloud Architecture Data Engineering Retrieval & RAG Mentorship Code Quality & Testing Version Control Experimentation & Evaluation Security, Privacy & Compliance Communication Problem Solving

Experience Level

USD 132,600 - 182,250/year

Employment Type

Full Time

• • Dental coverage
• • Health care and dependent care spending accounts
• • Short- and long-term disability
• • Life insurance and accidental death & dismemberment insurance
• • Employee and Family Assistance Program (EAP)
• • Retirement plan with generous company match