Lead AI Data Engineer — Spark, AI Agents, Hybrid

Syndesus, Inc.; San Francisco (CA); Hybrid; USD 180,000 - 240,000; Full time

Benefits offered by this job

Bonus program and 401(k) retirement

Medical, dental, vision coverage

Parental leave

14 paid holidays

Unlimited PTO

Flexible working hours

Community involvement

Job summary

Syndesus, Inc. is seeking an experienced data/AI engineer to own the data foundation and AI enablement for our eCommerce analytics initiative. You will design scalable data pipelines, ship AI-powered tooling, and drive reliable analytics across product and business teams.

The role emphasizes hands-on development, collaboration with product and security teams, and delivering an integrated eCommerce Health view with actionable insights.

Qualifications

• 8+ years in application development and large-scale distributed systems.
• 5+ years hands-on production data engineering with ETL/ELT workflows.
• Expert Spark / PySpark with performance tuning at billion-row scale.
• Expert SQL and strong Python.
• Experience shipping LLM-powered applications and AI agents.
• Domain knowledge in eCommerce or related operational data.

Responsibilities

• Build the data foundation by bringing data from multiple datastores into analytics-ready and AI-ready state with production ETL/ELT pipelines.
• Design dimensional models (star/snowflake) for analytics, KPIs, and dashboards.
• Build gold-layer summarized datasets for reliable NL querying and executive dashboards.
• Make the eCommerce Health view combine system health with commercial outcomes.
• Surface standardized engineering metrics across teams (delivery velocity, quality, security, incidents).
• Ship AI agents and productivity tools to reduce toil and accelerate root-cause analysis.
• Own lifecycle of LLM features: prompts, grounding, tool use, guardrails, evals, reliability.
• Develop applications on Databricks platform and collaborate with Product, SRE, Security.

Skills

Spark / PySpark

SQL

Python

Data modeling

Databricks

LLM / AI

ETL/ELT

Analytics dashboards

Domain knowledge - eCommerce

Cross-functional collaboration

Tools

Databricks

Delta Lake

Unity Catalog

Medallion architecture

Databricks Apps

Job description

Syndesus, Inc. is seeking an experienced data/AI engineer to own the data foundation and AI enablement for our eCommerce analytics initiative. You will design scalable data pipelines, ship AI-powered tooling, and drive reliable analytics across product and business teams.

The role emphasizes hands-on development, collaboration with product and security teams, and delivering an integrated eCommerce Health view with actionable insights.