AI Architect – AI Agent Platform

Stellar Consulting Solutions, LLC; United States; On-site; USD 150,000 - 190,000; Full time

Job summary

Stellar Consulting Solutions, LLC seeks a Lead Engineer to build AI agents and scalable enterprise platforms. You will lead the evolution of the platform powering AI at scale, focusing on security, observability, CI/CD, and rigorous validation of AI systems.

You will drive design for agent development, orchestration, and RAG pipelines, while enabling multiple model providers and ensuring robust testing and evaluation frameworks.

Qualifications

• 8+ years of software engineering experience.
• Strong experience with GCP (Vertex AI, GKE, BigQuery, APIs).
• Experience with distributed systems and scalable platform design.

Responsibilities

• Build AI agents and agentic workflows for real-world business use cases.
• Design and evolve core platform capabilities for agent development, orchestration, and runtime.
• Develop RAG pipelines, tool integrations, and structured reasoning systems.
• Enable usage across multiple LLM providers and model libraries.
• Implement secure, scalable, and observable architectures for AI systems.
• Establish robust CI/CD pipelines for agent lifecycle.
• Define and implement agent testing & evaluation strategies including automated evaluation pipelines, agentic testing models, and simulation-based validation.
• Build and enforce LLM observability and evaluation frameworks.
• Drive platform standards, reusable components, and engineering best practices.
• Partner across teams and external partners to scale platform adoption.

Skills

8+ years software engineering

GCP (Vertex AI, GKE, BigQuery)

Distributed systems

Tools

LangChain

LangGraph

Job description

We are building a next-generation enterprise AI platform that enables teams to design, deploy, and scale intelligent agents across digital, marketing, and operational workflows.

This platform provides standardized infrastructure, model access, orchestration, evaluation, and governance, enabling teams to rapidly build AI-powered capabilities in a secure, scalable, and production-ready way.

We're looking for Lead Engineers who can both:

• Build high-quality AI agents and workflows
• Lead the evolution of the platform powering them at scale

This role sits at the intersection of Agentic AI, platform engineering, and enterprise systems, with strong emphasis on security, CI/CD, observability, and rigorous validation of AI systems.

What You'll Do

• Build and deploy AI agents and agentic workflows for real-world business use cases.
• Design and evolve core platform capabilities for agent development, orchestration, and runtime.
• Develop RAG pipelines, tool integrations, and structured reasoning systems.
• Enable flexible usage across multiple LLM providers and model libraries.
• Implement secure, scalable, and observable architectures for AI systems.
• Establish robust CI/CD pipelines for agent lifecycle (build, test, deploy, rollback).
• Define and implement agent testing & evaluation strategies, including:
  • Automated evaluation pipelines
  • Agentic testing models and simulation-based validation
  • Continuous quality, accuracy, and regression testing
• Build and enforce LLM observability and evaluation frameworks (this is core to the role).
• Drive platform standards, reusable components, and engineering best practices.
• Partner across teams and external partners to scale adoption of the platform.

What We're Looking For

Core Engineering Skills

• 8+ years of software engineering experience.
• Strong experience with GCP (Vertex AI, GKE, BigQuery, APIs).
• Experience with distributed systems and scalable platform design.

AI / Agentic Systems

• Hands-on experience with:
  • LLMs, embeddings, and RAG architectures
  • Agentic workflows and AI-driven systems
• Experience working with multiple model providers / model ecosystems.
• Strong familiarity with agent frameworks such as LangChain, LangGraph, or similar ecosystems.

Platform, Security & DevOps

• Strong experience with:
  • CI/CD pipelines and automation for production systems
  • Secure system design (authentication, authorization, data access controls)
  • API-first and microservices architectures
• Experience designing highly observable systems (metrics, tracing, logging).
• Ability to manage performance, cost, and reliability trade-offs in AI workloads.

Evaluation, Observability & Testing (Must-Have)

• Hands-on experience with LLM evaluation frameworks and observability tooling.
• Model evaluation and benchmarking pipelines
• Continuous validation / regression testing for AI systems.
• Simulation or scenario-based testing for agent behavior

Nice to Have

• Experience with AI governance / Responsible AI practices.
• Familiarity with enterprise-scale AI platforms or shared developer platforms.
• Retail or domain experience (digital, marketing, operations).

Why This Role

• Work on a foundational AI platform initiative with enterprise-wide impact.
• Define how AI agents are built, governed, and scaled across the organization.
• Influence both hands-on engineering and long-term platform direction.