Staff AI Engineer | Canada | Remote

United States Digital Space LLC; United States; Remote; USD 140,000 - 210,000; Full time

Job summary

Grafana Labs is seeking a Staff Engineer (AI & Automation) to own the AI agent infrastructure and automation platform powering our Marketing Operations. You’ll build multi-agent architectures and backend services that connect AI models to internal and third-party data platforms, shipping reliable systems used daily.

This high-autonomy role lets you define the technical direction, establish governance, and drive self-service automation across Marketing, RevOps, and SDR teams, collaborating with

Qualifications

• 8+ years of software engineering experience in backend or data/analytics
• Hands-on experience applying LLMs/AI to production workflows
• Fluent in Python and JavaScript/Node.js with Git-based workflows
• Experience with AI frameworks and patterns including prompt engineering, RAG, function calls, and output parsing
• Proven ability to design and operate multi-agent systems at scale
• Familiarity with Google Cloud Platform and serverless/containerized services

Responsibilities

• Own end-to-end development of multi-agent AI systems from architecture to production
• Build modular agentic systems using orchestration frameworks
• Develop reusable agentic skills across interfaces (Slack, dashboards, internal apps)
• Implement observability, metrics, prompt iteration, and cost management
• Establish governance for AI workflows including access controls and audit trails
• Build MCP servers, APIs, CLIs, and microservices connecting AI models to business systems
• Architect data flows for retrieval-augmented generation (RAG)
• Design and deploy scalable serverless or containerized services
• Partner with cross-functional teams to scope automation problems and deliver measurable outcomes

Skills

Backend development

Systems integration

Data engineering

Python

JavaScript/Node.js

LLM frameworks

Prompt engineering

RAG

Cost management

Git workflows

Tools

LangChain

CrewAI

Anthropic MCP

BigQuery

GCP Cloud Functions

Cloud Run

Job description

Grafana Labs, the company behind the open observability cloud, is founded on the principles of open source, open standards, open ecosystems, and open culture. Grafana Cloud, our fully managed observability platform, is flexible and built for scale. With Grafana Cloud's actually useful AI, organizations can see, understand, and act on all their disparate data to move at the speed of their ambitions. Today, more than 35 million users and 7,000+ customers – including Anthropic, Bloomberg, NVIDIA, Microsoft, and Salesforce – trust Grafana Labs to ensure reliability of their applications and systems, resolve incidents quickly, and optimize their telemetry to reduce noise and cost. We are a 100% remote company with 1,600+ team members across 40+ countries, and we’re backed by leading investors including Lightspeed Venture Partners, Sequoia Capital, GIC, Coatue, J.P. Morgan, CapitalG, and Lead Edge Capital. Learn more at grafana.com and follow us on LinkedIn and X.

We’re scaling fast and staying true to what makes us different: an open-source legacy, a global collaborative culture, and a passion for meaningful work. Our team thrives in an innovation-driven environment where transparency, autonomy, and trust fuel everything we do.

You may not meet every requirement, and that’s okay. If this role excites you, we’d love you to raise your hand for what could be a truly career-defining opportunity.

This is a remote opportunity and we are looking for candidates from Canada. Residents of Quebec are not eligible for this role.

The Opportunity

Grafana Labs is seeking a Staff Engineer (AI & Automation) to own the AI agent infrastructure and automation platform that powers our Marketing Operations organization. You’ll build multi-agent architectures, LLM integrations, and backend services that connect AI models to internal and third-party data platforms. You’ll ship production systems that teams depend on daily.

This is a high-autonomy role where you own the technical direction. You’ll identify the highest-leverage problems across Marketing, RevOps, and SDR teams, design the solutions, and ship them. You’ll define the technical direction for the automation platform (data models, API contracts, shared libraries, reference architectures) and partner with Data Engineering, GTM Systems, and Field Operations to build scalable, self-service automation that eliminates manual work and drives operational efficiency.

What You'll Be Doing

Agentic Systems & AI Infrastructure

• Own end-to-end development of multi-agent AI systems, from architecture and implementation through testing, deployment, and ongoing operation

• Build modular, composable agentic systems using orchestration frameworks (LangChain, CrewAI, Anthropic MCP, or similar) that operate 24/7 across teams

• Develop reusable agentic skills that agents invoke across interfaces (Slack, dashboards, internal apps, CLIs)

• Implement observability and feedback loops including logging, performance metrics, prompt iteration, model evaluation, and cost management

• Establish governance and compliance standards for AI workflows including access controls, audit trails, PII handling, and human-in-the-loop escalation paths

Systems Integration & Backend Services

• Build MCP servers, APIs, CLIs, and microservices connecting AI models to business systems (BigQuery, Slack, CRMs, email, calendars, analytics tools)

• Architect data flows for retrieval-augmented generation (RAG), connecting LLMs to internal knowledge bases, customer data, and real-time business context

• Build serverless or containerized services (GCP Cloud Functions, Cloud Run) that scale with usage and integrate with Grafana's cloud infrastructure

Automation & Workflow Enablement

• Partner with RevOps, Demand Generation, Regional Marketing, and SDR teams to scope high-impact automation problems, identify bottlenecks, and build solutions with measurable business outcomes

• Design and deploy workflows using orchestration tools (n8n, Workato, or custom platforms) with CI/CD, testing, and production reliability standards

• Build systems designed for self-service with documentation, playbooks, and enablement materials that let partner teams operate independently

We invest heavily in developer productivity. You'll have access to AI coding assistants (Claude Code, Gemini CLI, OpenAI Codex, and others of your choice within security guidelines). We encourage pragmatic AI-assisted development paired with strong code review and quality standards.

What Makes You a Great Fit

• 8+ years of software engineering experience with depth in backend development, systems integration, or data/analytics engineering

• 2+ years hands-on experience applying LLMs/AI to production workflows, not just prototypes

• Strong proficiency in Python and JavaScript/Node.js with Git-based workflows, code review practices, and testing discipline

• Hands-on experience with LLM frameworks and patterns including prompt engineering, RAG, function calling/tool use, structured output parsing, and evaluation

• Experience building and operating multi-agent systems at scale including agent decomposition, orchestration patterns (sequential chains, router/dispatcher, parallel fan-out), state management, and production monitoring

• You diagnose business problems before writing code. You think in workflows and outcomes, not just functions.

• Deep familiarity with Google Cloud Platform, BigQuery, and serverless/containerized services (Cloud Functions, Cloud Run)

• Understanding of LLM failure modes and production mitigations including confidence thresholds, fallback logic, human escalation, and cost/latency management

• Proven ability to identify high-leverage problems, push back on low-impact requests, and deliver end-to-end with minimal direction

• Fluent with AI-assisted development tools (GitHub Copilot, Cursor, Claude Code). You use AI to build AI systems

• Clear technical communicator who can explain complex systems in simple terms to both engineers and business stakeholders

Bonus Points

• Experience with vector databases or retrieval pipelines (Pinecone, Weaviate, ChromaDB, Qdrant, pgvector)

• Familiarity with marketing or sales platforms (Salesforce, Customer.io, HubSpot, Marketo, Outreach)

• Experience with frontend frameworks (React, Slack Block Kit) for building user-facing AI tool interfaces

• Observability tooling for AI systems (LangSmith, Weights & Biases, custom evaluation frameworks)

• Experience with workflow orchestration platforms (n8n, Temporal, Prefect, Airflow)

• Familiarity with Model Context Protocol (MCP) or similar standards for connecting AI systems to data sources

• Prior work automating marketing, sales, or customer success workflows in a B2B SaaS environment

• Active in open-source communities. Grafana is built on OSS and we value engineers who share that DNA

In Canada, the base compensation range for this role is CAD 164,490- CAD 197,389. Actual compensation may vary based on level, experience, and skillset as assessed throughout the interview process. All of our roles include Restricted Stock Units (RSUs), giving every team member ownership in Grafana Labs' success. We believe in shared outcomes—RSUs