Data Platform Engineer – Databricks, Spark & AWS Pipelines

Edmunds; United States; Hybrid; USD 125,000 - 159,000; Full time

Benefits offered by this job

Flexible time off

Comprehensive Health Benefits (medical

Dental, Vision, Life and Disability)

401K Plan with company matching 100%

Stock purchase program

Parental leave

Job summary

Edmunds seeks a data engineering professional to manage and evolve the Data Platform that powers BI, dashboards, and AI tools used across the company. You will maintain pipelines, Terraform/CDK IaC, and Databricks workloads in AWS, partnering with analysts, engineers, and executives to deliver reliable data assets.

The role emphasizes platform reliability, cross-functional collaboration, and the ability to translate business needs into scalable data solutions within a hybrid work setup.

Qualifications

• 4+ years of experience in Data Engineering or similar role.
• Proficiency in Spark (Python or Scala).
• Fluency in Python and SQL.
• Extensive experience using and administering Databricks.
• Familiarity with IaC concepts and hands on experience with Terraform and/or CDK.
• Excellent problem solving, troubleshooting, and communication skills in a hybrid environment.
• Bachelor's degree in Computer Science, Engineering, or equivalent experience.

Responsibilities

• Administer and support our Databricks platform and the underlying AWS infrastructure.
• Collaborate with cross-functional customers including Business, Executives, Engineering, Data Science, Analytics, Security & Infrastructure.
• Explore, POC, and adopt new features and technologies for the platform.
• Build and maintain ETL pipelines for operational data to monitor platform health.
• Create and maintain IaC artifacts for admin tooling (CDK, Terraform).
• Ensure teams can use the Data Platform to build AI solutions.
• Build, enhance, and maintain dashboards, alerts, and reports for operational, cost, and security insights.

Skills

Spark (Python/Scala)

Python

SQL

Databricks

IaC concepts

Terraform

CDK

Education

Bachelor's degree in Computer Science or Engineering

Tools

Databricks

Terraform

CDK

Job description

Edmunds seeks a data engineering professional to manage and evolve the Data Platform that powers BI, dashboards, and AI tools used across the company. You will maintain pipelines, Terraform/CDK IaC, and Databricks workloads in AWS, partnering with analysts, engineers, and executives to deliver reliable data assets.

The role emphasizes platform reliability, cross-functional collaboration, and the ability to translate business needs into scalable data solutions within a hybrid work setup.