Senior Data Engineer

Hearst Media Services; Troy (MI); Hybrid; USD 120,000 - 170,000; Full time

Job summary

Hearst Media Services (MOTOR Information Systems) seeks a Senior Data Engineer to architect, build, and evolve an AWS-based data platform focused on scalable, reliable data pipelines and products. You will own design decisions around performance, cost, and reliability while collaborating with a cross-functional team in a data-driven automotive domain.

The role requires expert Python/SQL, strong AWS experience, Databricks, Kubernetes, and DevOps practices, with a focus on handling large datasets

Qualifications

• Expert Python and SQL development.
• Strong AWS experience including Lambda, Glue, Step Functions, Fargate, CloudFormation, and Redshift.
• Strong experience with semi-structured data (XML, CSV, JSON, Parquet).
• Strong automation experience.
• Experience with Databricks, Delta Lake, notebooks, and SQL analytics.
• Experience with Kubernetes, including cluster orchestration and pod management.
• Experience with DevOps and CI/CD, including Octopus Deploy deployment orchestration.
• Experience with agentic AI: LangGraph, MCP servers, and agent frameworks.
• Experience with Pandas, NumPy, Apache Spark, and Ray.

Responsibilities

• Analyze large, complex data sets to support pipeline and platform decisions.
• Design, develop, and coordinate the generation of data pipelines.
• Collaborate with the team to plan and solve complex technical problems.
• Design for, or optimize existing pipelines, focusing on performance and cost efficiency.

Skills

Python and SQL development

AWS experience

Semi-structured data

Automation

Databricks

Kubernetes

DevOps and CI/CD

Agentic AI

Data handling (Pandas/NumPy, Spark,Ray

Education

Bachelor's Degree

Tools

Databricks

Delta Lake

Kubernetes

Octopus Deploy

LangGraph

MCP Servers

AWS Lambda

AWS Glue

AWS Step Functions

AWS Fargate

AWS CloudFormation

Amazon RedShift

Job description

The Senior Data Engineer role at Hearst Media Services (MOTOR Information Systems) focuses on architecting, designing, building, and evolving an AWS-based data platform. The position requires technical ownership across scalability, reliability, performance, and cost while delivering data pipelines and data products.

Responsibilities

• Analyze large and complex data sets to support pipeline and platform decisions.
• Design, develop, and coordinate the generation of data pipelines.
• Collaborate with the team to plan and solve complex technical problems.
• Design for, or optimize existing pipelines, with emphasis on performance and cost efficiency.

Requirements

• Expert Python and SQL development.
• Strong AWS experience, including: AWS Lambda, AWS Glue, AWS State Machines, AWS Fargate, AWS CloudFormation, and Amazon RedShift.
• Strong experience with semi-structured data such as XML, CSV, JSON, and Parquet.
• Strong automation experience.
• Experience with Databricks, including Delta Lake, notebooks, and SQL analytics.
• Experience with Kubernetes, including cluster orchestration, container networking, and pod management.
• Experience with DevOps and CI/CD, including deployment orchestration with Octopus Deploy.
• Experience with agentic AI, including LangGraph, MCP servers, and agent frameworks.
• Experience with data handling, including Pandas/NumPy, Apache Spark, and Ray.

Technologies

• Python, SQL
• AWS Lambda, AWS Glue, AWS Step Functions, AWS Fargate, AWS CloudFormation, Amazon RedShift
• XML, CSV, JSON, Parquet
• Databricks, Delta Lake, Databricks Notebooks, SQL Analytics
• Kubernetes
• Octopus Deploy
• LangGraph, MCP Servers
• Pandas, NumPy, Apache Spark, Ray
• Agent frameworks

Location & Schedule

• Location: Troy, MI (hybrid)
• Hybrid schedule: 3 days onsite in Troy, MI

Education

Bachelor's Degree.

Benefits

• No visa sponsorship available at this time.

Helpful Experience (Nice to Have)

• Experience with LLMs, machine learning & deep learning libraries, AI application frameworks, and vector databases.
• Experience with automotive data or the automotive industry.
• Experience with Agile or Scrum.

About MOTOR Information Systems (Hearst)

MOTOR is the industry leader in automotive data and insights. The organization powers the aftermarket ecosystem with comprehensive datasets, and is investing in modernizing its data platform to support the next generation of products and AI-driven capabilities.