Lead Data Platform Engineer - Remote, Flink/Kafka

Comcast Advertising; Reston, Northern (VA, KY); Hybrid; USD 175,000 - 200,000; Full time

Job summary

FreeWheel is seeking a Lead Software Engineer (Data) to provide technical leadership for data processing software built on Apache Flink and streaming technologies. You will design ETL pipelines, store structured data in HBase, and contribute to ad-tech platforms including ad serving and targeting.

Applicants should have a strong background in Flink, Kafka, Spark, AWS EMR, and Airflow, with experience mentoring engineers and driving software releases in a collaborative, fast-paced environment.

Qualifications

• 7+ years of experience developing data processing software using Apache Flink and related tech.
• Experience designing ETL data pipelines with Flink, Spark, Kafka, AWS EMR, Azkaban and Airflow.
• Experience storing structured data (e.g., HBase) and building streaming data pipelines.

Responsibilities

• Provide technical leadership to a team developing data processing software using Flink and ETL pipelines.
• Design, implement, and optimize data pipelines and streaming apps with Kafka, Flink, and Spark.
• Collaborate with stakeholders, mentor junior engineers, and guide software releases.

Skills

Apache Flink

Apache Kafka

ETL pipelines

HBase

Spark

AWS EMR

Azkaban

Airflow

Linux CLI

Bash scripting

Education

Bachelor's degree in Computer Science, Engineering, or related field

Tools

Airflow

Azkaban

Spark

AWS EMR

Linux CLI

Job description

FreeWheel is seeking a Lead Software Engineer (Data) to provide technical leadership for data processing software built on Apache Flink and streaming technologies. You will design ETL pipelines, store structured data in HBase, and contribute to ad-tech platforms including ad serving and targeting.

Applicants should have a strong background in Flink, Kafka, Spark, AWS EMR, and Airflow, with experience mentoring engineers and driving software releases in a collaborative, fast-paced environment.