Data Platform Engineer – AI Lakehouse (Hybrid)

Harley Ellis Devereaux (HED)

Boston (MA)

Hybrid

USD 120,000 - 170,000

Full time

5 days ago

Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Harley Ellis Devereaux (HED) seeks a Data Platform Developer to build and enhance our data lakehouse infrastructure, enabling trusted analytics and AI-enabled workflows. You will own production pipelines, governance, and platform health while partnering with analysts and developers to deliver reliable data products.

You will work in a hybrid environment, collaborating across teams to improve ingestion, transformation, documentation, and platform automation.

Qualifications

3+ years building production data pipelines on cloud platforms.
Strong SQL and Python; Spark or similar distributed processing.
Experience with production incident response and data recovery.
Experience with monitoring, alerting, and data reliability practices.
Ability to document clearly for technical colleagues and business stakeholders.

Responsibilities

Develop, operate, and improve ingestion and transformation pipelines across Bronze, Silver, and Gold data layers.
Administer and improve the Databricks environment, including workflows and Unity Catalog governance.
Maintain version-controlled configurations and deployment workflows; resolve differences between source control and production.
Implement automated data quality checks and schema contract tests to protect data.
Monitor pipeline freshness, availability, performance, and cost; maintain alerts and runbooks.
Collaborate with analytics and AI teams to deliver governed data products and safe AI-assisted workflows.