Lead AI Engineer

Mission.dev; San Francisco (CA); Hybrid; USD 150,000 - 200,000; Full time

Benefits offered by this job

Health insurance

Dental insurance

Vision insurance

Life insurance

401(k) retirement plan

Paid parental leave

14 paid holidays

Unlimited vacation

Flexible working hours

Job summary

Mission.dev is seeking an experienced Senior Data Engineer to build a comprehensive engineering intelligence platform. You will own the end-to-end data architecture, merging telemetry with commercial metrics to create a single source of truth and accelerate development velocity.

You will drive AI enablement across the engineering organization by designing and shipping LLM-powered agents and productivity tools, including code search assistants and runbook copilots, using RAG and modern lakehouse

Qualifications

• 8+ years of experience in application development and distributed systems architecture.
• 5+ years of production data engineering experience building ETL/ELT workflows and analytics platforms.
• Expert proficiency in Spark, PySpark, and SQL for large-scale data processing and performance tuning.
• Strong Python programming skills and extensive experience with dimensional modeling.
• Proven experience shipping LLM-powered applications, including production RAG systems or AI agents.
• Deep expertise in modern lakehouse architectures and medallion data structures.
• Professional background in eCommerce or similar transactional and operational data domains

Responsibilities

• Develop production ETL/ELT pipelines and dimensional models to transform raw eCommerce and observability data into an analytics-ready state.
• Construct summarized datasets and aggregation layers to support natural-language querying and executive-level reporting.
• Engineer a unified health dashboard that integrates system latency and error signals with commercial metrics like conversion and renewal rates.
• Standardize and surface engineering performance metrics, including delivery velocity, security posture, and incident trends.
• Build and deploy AI agents for automated anomaly detection, incident triage, and intelligent root-cause analysis.
• Design and implement LLM-powered productivity tools, including code search assistants and runbook copilots, using RAG and prompt engineering.
• Utilize modern lakehouse platforms to deliver internal tools, chatbots, and operational applications directly to stakeholders.
• Collaborate with cross-functional teams to identify automation opportunities and evangelize AI adoption across the engineering organization.

Skills

Distributed systems

Data engineering

Spark

PySpark

SQL

Python

LLM / AI agents

Lakehouse

Ecommerce domain

Tools

RAG

Job description

Employment type: Full-time employee (40 hours per week), hired directly by the client.

Work arrangement: Hybrid, based at the client’s office in Frisco, TX, the San Francisco Bay Area, or New York City. You'll be on-site on an as-needed basis and work from your home office otherwise.

Candidate location: Candidates must be based in or within commuting distance of one of these office locations. Relocation assistance is not available for this position.

Our company description

Mission.dev is the next-gen staffing platform for software talent.

We help you find, evaluate, and manage top software talent (contractors or direct hires) faster, smarter, and more efficiently.

Powered by AI. Backed by real humans.

About the client

This global consumer cybersecurity organization provides integrated digital safety products designed to protect individuals and families online.

Its operations are supported by a high-scale global eCommerce engine that manages millions of transactions, subscriptions, and renewals worldwide.

The company focuses on creating intuitive products that ensure secure digital experiences, leveraging a robust technical infrastructure to maintain system health and commercial performance across its global user base.

About the Role

This is a hands-on technical role focused on building a comprehensive engineering intelligence platform.

You will own the development of a unified operational system that merges technical telemetry with commercial performance data to create a single source of truth for the organization.

Additionally, you will drive AI enablement across the engineering department by designing and shipping agents and productivity tools.

You will be responsible for the full lifecycle of data architecture and AI features, directly impacting how the company monitors system health and accelerates development velocity.

What You'll Do

• Develop production ETL/ELT pipelines and dimensional models to transform raw eCommerce and observability data into an analytics-ready state.
• Construct summarized datasets and aggregation layers to support natural-language querying and executive-level reporting.
• Engineer a unified health dashboard that integrates system latency and error signals with commercial metrics like conversion and renewal rates.
• Standardize and surface engineering performance metrics, including delivery velocity, security posture, and incident trends.
• Build and deploy AI agents for automated anomaly detection, incident triage, and intelligent root-cause analysis.
• Design and implement LLM-powered productivity tools, including code search assistants and runbook copilots, using RAG and prompt engineering.
• Utilize modern lakehouse platforms to deliver internal tools, chatbots, and operational applications directly to stakeholders.
• Collaborate with cross-functional teams to identify automation opportunities and evangelize AI adoption across the engineering organization.

What You Bring

• 8+ years of experience in application development and distributed systems architecture.
• 5+ years of production data engineering experience building ETL/ELT workflows and analytics platforms.
• Expert proficiency in Spark, PySpark, and SQL for large-scale data processing and performance tuning.
• Strong Python programming skills and extensive experience with dimensional modeling.
• Proven experience shipping LLM-powered applications, including production RAG systems or AI agents.
• Deep expertise in modern lakehouse architectures and medallion data structures.
• Professional background in eCommerce or similar transactional and operational data domains

Nice to Haves

• Deep technical experience with specific lakehouse governance and workflow tools.
• Experience utilizing AI coding assistants to accelerate development cycles.
• Previous experience partnering with SRE, Security, and Architecture teams on operational tooling.
• Performance-based bonus program and 401(k) retirement plan.
• Comprehensive medical, dental, vision, and life insurance coverage.
• Short-term and long-term disability protection.
• Paid parental leave and 14 paid company holidays.
• Unlimited paid time off and flexible working hours.
• Community involvement and social programs.