Senior Data Engineer for AI-Driven Data Platforms

Adobe Inc.; San Francisco (CA); On-site; USD 163,200 - 236,400; Full time

Job summary

Adobe is seeking a Senior Data Engineer to build scalable data architectures and ETL pipelines on Databricks for the Pro Design team. You will partner with Product, Engineering, and Data Science to instrument data logging for experiments and AI features, while establishing governance on data quality, privacy, and lineage.

You will also develop automated reporting foundations powering AI-driven insights and ensure pipelines are efficient and cost-optimized at scale.

Qualifications

• Bachelor's degree in Computer Science, Computer Engineering, or equivalent practical experience.
• Proficiency in SQL and Python, PySpark, Airflow, Databricks workflows.
• 5+ years in data engineering, including production pipelines at scale.
• Experience with high-volume event/telemetry data from a client application.
• Experience integrating AI tools to optimize workflows and latency.
• Experience building multi-agent AI systems or agentic workflows.
• Working knowledge of AI/ML data pipelines and modern BI tools.
• Proven ability to partner with Product, Engineering, and Data Science.

Responsibilities

• Build, scale and optimize the data architecture and ETL/pipeline on Databricks across Pro Design products, spanning shared platform data and in-app usage telemetry.
• Partner with Product, Engineering, and Data Science to define and implement data logging and instrumentation for product experimentation, AI/ML features, and analytics.
• Set and evolve governance standards regarding data quality, privacy, security, lineage, and SLAs in Unity Catalog.
• Build automated reporting and the data foundations that power AI-driven insights across our reporting stack.
• Keep production pipelines and queries healthy and performant at scale and drive optimization for cost and latency.

Skills

SQL

Python

PySpark

Airflow

Databricks workflows

Unity Catalog

Azure

Education

Bachelor's degree in CS/Engineering or related field

MS or PhD in CS/Engineering (preferred)

Tools

Databricks

Azure

Unity Catalog

Job description

Adobe is seeking a Senior Data Engineer to build scalable data architectures and ETL pipelines on Databricks for the Pro Design team. You will partner with Product, Engineering, and Data Science to instrument data logging for experiments and AI features, while establishing governance on data quality, privacy, and lineage.

You will also develop automated reporting foundations powering AI-driven insights and ensure pipelines are efficient and cost-optimized at scale.