Data Platform Engineer - Lakehouse & AI

Harley Ellis Devereaux (HED); Los Angeles (CA); Hybrid; USD 120,000 - 170,000; Full time

Job summary

Harley Ellis Devereaux is seeking a Data Platform Developer to build and optimize the data platform that powers our AI-enabled capabilities for the design practice. You will work on a modern lakehouse with Databricks, Delta Lake, and Unity Catalog to connect business systems to trusted analytics and emerging AI applications.

This individual contributor role offers ownership of pipelines, platform health, and data documentation, collaborating with analysts and developers to deliver governed data

Qualifications

• 3+ years building production data pipelines on cloud platforms.
• Strong SQL and Python skills; Spark or similar framework experience.
• Experience with monitoring, alerting, and data reliability practices.
• Ability to write clear technical docs and communicate with stakeholders.
• Familiarity with Delta Lake, Unity Catalog, or Databricks is a plus.

Responsibilities

• Develop, operate, and improve ingestion and transformation pipelines across Bronze, Silver, and Gold data layers, integrating sources such as Workday, Deltek, Dataverse, Microsoft Graph, and collaboration systems.
• Administer and continuously improve the Databricks environment, including workflows, compute standards, data storage, access controls, and Unity Catalog governance.
• Maintain version-controlled, reproducible platform configurations and deployment workflows; identify and resolve differences between source control and production environments.
• Implement automated data quality checks and schema contract tests to protect governed datasets from unexpected changes.
• Monitor pipeline freshness, availability, performance, and cost; maintain actionable alerts, dashboards, and operational runbooks.
• Investigate and resolve production incidents, coordinate data recovery and backfills, document root causes, and implement preventive improvements.
• Maintain technical lineage, source metadata, the data dictionary, and platform documentation; collaborate with business owners to confirm definitions and accountability.
• Assess downstream impacts of schema changes, coordinate communications with data consumers, and support controlled changes across dependent systems.
• Collaborate with analytics and AI development colleagues to deliver reliable, well-governed data products and evaluate AI-assisted operational workflows.
• Contribute to platform architecture, technical standards, and future data-source onboarding as experience and ownership expand.

Skills

SQL

Python

Spark

Data pipelines

Cloud experience

Documentation

Incident response

Monitoring

Tools

Databricks

Delta Lake

Unity Catalog

Snowflake

BigQuery

Job description

Harley Ellis Devereaux is seeking a Data Platform Developer to build and optimize the data platform that powers our AI-enabled capabilities for the design practice. You will work on a modern lakehouse with Databricks, Delta Lake, and Unity Catalog to connect business systems to trusted analytics and emerging AI applications.

This individual contributor role offers ownership of pipelines, platform health, and data documentation, collaborating with analysts and developers to deliver governed data