Lead AI Data Engineer

Syndesus, Inc.; Frisco (TX); Hybrid; USD 180,000 - 240,000; Full time

Benefits offered by this job

Bonus program

401(k) retirement plan

Medical, dental, vision coverage

Job summary

Syndesus, Inc. is seeking a senior data engineering leader to own the build of two strategic initiatives: a unified operational intelligence platform and AI enablement across engineering.

You will design data models, ships AI-enabled tools, and deliver production-grade ETL/ELT pipelines at scale using Databricks and lakehouse architectures. You will partner with Product, SRE, Security, and Architecture, deliver dashboards and metrics, and own the full lifecycle of LLM features.

Qualifications

• 8+ years in application development and large-scale distributed systems.
• 5+ years hands-on production data engineering with ETL/ELT pipelines.
• Expert Spark/PySpark with billion-row scale and Spark SQL.
• Strong Python and SQL skills for analytics.
• Data modeling for analytics with star/snowflake schemas.
• Experience with lakehouse platforms (Databricks) and related tools.
• Hands-on AI engineering including LLM-powered apps.
• Domain expertise in eCommerce data and related metrics.
• Ability to partner with Product, SRE, Security, Architecture.

Responsibilities

• Build the data foundation by ingesting data from multiple sources and creating ETL/ELT pipelines with quality validation.
• Design dimensional models and summary datasets to power analytics and dashboards.
• Ship the eCommerce Health view combining system signals with business metrics.
• Develop AI agents and productivity tools to reduce toil and improve reliability.
• Own lifecycle of LLM features including prompts, grounding, guards, and evals.
• Use Databricks as a product platform to deliver internal tools and apps.
• Collaborate with Product, SRE, Security, and Architecture to drive adoption.

Skills

Expert Spark / PySpark

Expert SQL

Python programming

Data modeling

Databricks platform

AI agent development

AI tooling

Production data engineering

Domain knowledge: eCommerce data

Cross-functional collaboration

Tools

Databricks

Delta Lake

Unity Catalog

Medallion architecture

Databricks Workflows

Job description

The role

You will own the build of two strategic initiatives:

• A unified operational intelligence platform — an eCommerce Health view that combines system signals (latency, errors, integration and data-flow health) with commercial performance (conversion, checkout, auto-renewal, feature adoption) into one trusted source of truth.
• AI enablement across engineering — shipping agents and productivity tools that engineering teams actually adopt and use every day.

This is a builder's role, not a platform administration role. You will be writing Spark, designing schemas, tuning jobs, and shipping applications that stakeholders open and rely on.

What you'll do

Build the data foundation

• Bring data from eCommerce, MarTech, observability, and operational datastores into an analytics-ready and AI-ready state through production ETL/ELT pipelines, expert SQL, and deliberate data modeling — with automated quality validation built in.
• Design dimensional models (star/snowflake schemas, sensible partitioning and grain) purpose-built to power analytics, KPI reporting, and dashboards.
• Build gold-layer, summarized datasets — collapsing millions of raw events into one clean summary record per entity per day — the aggregation layer that makes natural-language querying and executive dashboards trustworthy rather than approximately correct.

Make performance visible

• Design and ship the eCommerce Health view: one place where system health and commercial outcomes sit side by side.
• Surface standardized engineering metrics across teams — delivery velocity (DORA-style), quality, security posture, incident trends, and service ownership.

Ship AI that reduces toil

• Build AI agents that detect anomalies, surface critical issues with actionable context, notify the right stakeholders, and drive intelligent triage that accelerates root‑cause analysis.
• Build AI productivity tools that help teams ship faster and operate more reliably — RCA assistance, intelligent search across telemetry and code, runbook copilots.
• Own the full lifecycle of LLM features: prompt design, retrieval and grounding, tool use, guardrails, evals, and operational reliability.

Ship applications, not just tables

• Use Databricks as a product platform, not only a pipeline runtime — Genie, Databricks Apps, and Lakebase to deliver internal tools, chatbots, and operational apps that stakeholders interact with directly.
• Partner closely with Product, SRE, Security, and Architecture to translate operational pain into solutions teams adopt.
• Identify and spread AI use cases across engineering teams — this role has an evangelism dimension as well as a build dimension.

What you'll bring

Experience

• 8+ years in application development, including designing and architecting large-scale production applications and distributed systems.
• 5+ years hands‑on production data engineering — building pipelines, ETL/ELT workflows, and analytics platforms that turn raw operational and business data into decisions.
• A track record as a genuine domain expert on business‑critical data work, and the versatility to carry those skills from one domain and one data platform to another.
• Expert Spark / PySpark. Performance tuning, Spark SQL, partitioning and shuffle handling at billion‑row scale. This is the foundation the rest of the role is built on.
• Expert SQL at scale and strong Python.
• Data modeling for analysis — dimensional modeling, star/snowflake schemas, thoughtful grain and partitioning decisions.
• Modern lakehouse platforms, with Databricks depth strongly preferred: Delta Lake, Workflows, Unity Catalog, Medallion architecture. Real‑time and event‑driven processing.

AI engineering (must‑have)

• Hands‑on experience shipping LLM‑powered applications: production RAG systems, AI agents, or LLM‑integrated internal tools, with sound judgment on tool use, guardrails, evals, and reliability.
• At least one AI agent you personally built and can walk through in detail — what it monitored, how it decided to act, what framework and model you used, and what happened when it was wrong.
• Daily use of AI coding assistants (Claude, Copilot, or similar) to ship real work faster — pipelines, dashboards, tooling.

Domain

• eCommerce or a comparable operational domain — funnel metrics, subscription flows, payment processing. You understand the shape of transactional and operational data before you model it.

Ways of working

• Efficiency‑driven impact: a demonstrable record of using AI to measurably reduce toil and accelerate incident response.
• Strong cross‑functional partnership across Product, SRE, Security, and Architecture.

Work arrangement

This is a hybrid role based in Frisco, TX. You'll be on‑site on an as‑needed basis and work from your home office otherwise. Candidates must be within commutable distance of Frisco, TX. Relocation assistance is not available for this position.

Benefits and perks

• Bonus program and 401(k) retirement plan
• Medical, dental, vision, basic life, short‑term and long‑term disability coverage
• Paid parental leave
• 14 paid company holidays
• Unlimited paid time off for exempt employees
• Flexible working hours and family‑friendly benefits
• Community involvement and social programs

Our client embraces diversity and inclusion and encourages everyone to bring their authentic selves to work. They prohibit discrimination based on race, colour, religion, gender, national origin, age, disability, veteran status, marital status, pregnancy, gender expression or identity, sexual orientation, or any other legally protected status.