Lead Data Engineer

Epsilon; Chicago (IL); On-site; USD 98,000 - 182,000; Full time

Benefits offered by this job

Flexible time off

Paid holidays and sick time

Family leave and assistance

Comprehensive health coverage

401(k)

Tuition assistance

Commuter benefits

Professional development

Job summary

Epsilon seeks a senior data platform engineer to build and maintain high-scale data pipelines and integrations with major advertising and CTV platforms. The role reports to the Engineering Director and collaborates with data engineers, product managers, and partner teams.

The position is based in Chicago and offers a comprehensive benefits package including health coverage, 401(k), and paid time off.

Qualifications

• Bachelor's degree in a related field.
• 5+ years of experience with big data pipelines.
• Expertise with Apache Spark.
• Scala and Python proficiency.
• SQL proficiency and cloud experience (AWS preferred).

Responsibilities

• Writing scalable code using Spark (Scala/Python) and SQL.
• Leveraging AI-assisted engineering tools.
• Optimizing data connectors.
• Resolving production issues.
• Mentoring engineers.
• Maintaining automated cloud processing routines.

Skills

Spark

Scala

Python

SQL

Big data pipelines

AWS

Education

Bachelor's degree in a related field

Tools

Airflow

Kafka

NoSQL databases

REST APIs

Job description

The Epsilon Activation Delivery Platform team seeks a senior engineer to build and maintain high-scale data pipelines and integrations with major advertising and CTV platforms. The role reports to the Engineering Director and collaborates with data engineers, product managers, and partner teams.

Responsibilities

• Writing scalable code using Spark (Scala/Python) and SQL
• Leveraging AI-assisted engineering tools
• Optimizing data connectors
• Resolving production issues
• Mentoring engineers
• Maintaining automated cloud processing routines

Requirements

• A bachelor's degree in a related field
• 5+ years experience with big data pipelines
• Expertise in Apache Spark
• Scala/Python
• SQL
• Cloud platforms (AWS preferred)
• Strong problem-solving and communication skills

Preferred Experience

• AdTech/MarTech
• Orchestration tools like Airflow
• High-concurrency REST API integrations
• Streaming technologies (Kafka)
• NoSQL databases

The position is full-time, based in Chicago, Illinois. Salary range is $98,000 to $182,000 annually.

Benefits

• Flexible time off
• Paid holidays and sick time
• Family leave and assistance
• Comprehensive health coverage
• 401(k)
• Tuition assistance
• Commuter benefits
• Professional development
• And more