Lead AI Engineer

Mission.dev; Frisco (TX); Hybrid; USD 160,000 - 230,000; Full time

Benefits offered by this job

Performance-based bonus

401(k) plan

Health insurance

Disability coverage

Parental leave

Holidays

Unlimited PTO

Job summary

Mission.dev is seeking a hands-on engineering leader to build a unified Engineering Intelligence Platform. You will craft scalable data pipelines, optimize Spark workloads, and deliver AI-powered tools that boost engineering productivity and incident response.

This role focuses on production data engineering, Databricks Apps, Genie, and Lakebase, with collaboration across Product, SRE, Security, and Architecture to automate workflows and adopt AI at scale.

Qualifications

• 8+ years of application development and distributed systems experience.
• 5+ years of hands-on production Data Engineering experience.
• Expert Spark/PySpark skills, including performance tuning at scale, Spark SQL, partitioning, and shuffle handling.
• Expert SQL, strong Python, and experience with dimensional modeling and large-scale data pipelines.
• Experience with modern lakehouse architectures, including Delta Lake and Medallion Architecture; Databricks expertise strongly preferred.
• Hands-on experience deploying production AI agents, with the ability to explain their architecture, tool use, evaluation, and failure handling.
• Experience shipping LLM-powered applications using techniques such as RAG, tool calling, guardrails, and evaluation.
• Hands-on Databricks application development experience; familiarity with Databricks Apps, Genie, and Lakebase is highly relevant.
• Experience in eCommerce or similar transactional domains, such as payments, subscriptions, or renewals.
• Regular use of AI coding assistants such as Claude or GitHub Copilot to accelerate development.

Responsibilities

• Build and maintain production ETL/ELT pipelines integrating eCommerce, observability, and operational data.
• Develop and optimize Spark/PySpark jobs at scale, including Spark SQL, partitioning, and shuffle performance.
• Design dimensional models and aggregated gold-layer datasets for analytics, dashboards, and natural‑language querying.
• Build a unified eCommerce Health dashboard combining system performance, incident trends, conversion, checkout, and subscription metrics.
• Develop and deploy production AI agents for anomaly detection, incident triage, and root‑cause analysis.
• Build LLM‑powered engineering tools, including intelligent search, RCA assistants, and runbook copilots.
• Develop internal applications and AI‑powered experiences using Databricks Apps, Genie, and Lakebase.
• Partner with Product, SRE, Security, and Architecture teams to automate workflows and drive AI adoption.

Job description

Employment type

Full-time employee (40 hours per week), hired directly by the client.

Work arrangement

Hybrid, based at the client’s office in Frisco, TX, the San Francisco Bay Area, or New York City. You'll be on‑site on an as‑needed basis and work from your home office otherwise.

Candidate location

Candidates must be based in or within commuting distance of one of these office locations. Relocation assistance is not available for this position.

Our company description

Mission.dev is the next‑gen staffing platform for software talent.

We help you find, evaluate, and manage top software talent (contractors or direct hires) faster, smarter, and more efficiently.

Powered by AI. Backed by real humans.

About the Client

This global consumer cybersecurity organization provides integrated digital safety products that protect individuals and families online. Its global eCommerce platform supports millions of transactions, subscriptions, and renewals worldwide.

About the Role

This is a hands‑on engineering role focused on building a unified Engineering Intelligence Platform that combines system observability with commercial performance data. You will build scalable data pipelines, optimize Spark workloads, develop production AI agents, and deliver AI‑powered applications that improve engineering productivity and accelerate incident response.

This is a builder role requiring strong Data Engineering, production AI, and application development skills, with a focus on using Databricks to deliver solutions that stakeholders actively use.

Work Arrangement

Hybrid role based in Frisco, TX, the San Francisco Bay Area, or NYC. Candidates must live within commuting distance and be available to work on‑site as needed. Relocation assistance is not available.

What You'll Do

• Build and maintain production ETL/ELT pipelines integrating eCommerce, observability, and operational data.
• Develop and optimize Spark/PySpark jobs at scale, including Spark SQL, partitioning, and shuffle performance.
• Design dimensional models and aggregated gold-layer datasets for analytics, dashboards, and natural‑language querying.
• Build a unified eCommerce Health dashboard combining system performance, incident trends, conversion, checkout, and subscription metrics.
• Develop and deploy production AI agents for anomaly detection, incident triage, and root‑cause analysis.
• Build LLM‑powered engineering tools, including intelligent search, RCA assistants, and runbook copilots.
• Develop internal applications and AI‑powered experiences using Databricks Apps, Genie, and Lakebase.
• Partner with Product, SRE, Security, and Architecture teams to automate workflows and drive AI adoption.

What You Bring

• 8+ years of application development and distributed systems experience.
• 5+ years of hands‑on production Data Engineering experience.
• Expert Spark/PySpark skills, including performance tuning at scale, Spark SQL, partitioning, and shuffle handling.
• Expert SQL, strong Python, and experience with dimensional modeling and large‑scale data pipelines.
• Experience with modern lakehouse architectures, including Delta Lake and Medallion Architecture; Databricks expertise strongly preferred.
• Hands‑on experience deploying production AI agents, with the ability to explain their architecture, tool use, evaluation, and failure handling.
• Experience shipping LLM‑powered applications using techniques such as RAG, tool calling, guardrails, and evaluation.
• Hands‑on Databricks application development experience; familiarity with Databricks Apps, Genie, and Lakebase is highly relevant.
• Experience in eCommerce or similar transactional domains, such as payments, subscriptions, or renewals.
• Regular use of AI coding assistants such as Claude or GitHub Copilot to accelerate development.

Nice to Haves

• Experience with real‑time data processing and event‑driven architectures.
• Experience with engineering metrics, observability platforms, and automated incident response.
• Experience with data governance, security, and cross‑functional AI adoption.

Compensation & Benefits

• Performance‑based bonus and 401(k) retirement plan.
• Medical, dental, vision, and life insurance.
• Short‑and long‑term disability coverage.
• Paid parental leave and 14 company holidays.
• Unlimited PTO for eligible exempt employees and flexible working hours.
• Family‑friendly benefits and community programs