Senior Real-Time Data Platform Engineer

Comcast Advertising; Reston, Northern (VA, KY); Hybrid; USD 175,000 - 195,000; Full time

Job summary

FreeWheel, a Comcast company, seeks a Lead Software Engineer to drive data infrastructure for the advertising platform. You will lead real-time data processing with Flink, develop ETL pipelines with Spark and Airflow, and oversee data movement via Kafka, all while ensuring system health with Prometheus and Grafana.

You will mentor junior engineers, document designs, and collaborate with stakeholders to define requirements. Remote/hybrid work options are available as per policy.

Qualifications

• Bachelor’s degree or foreign equivalent in Computer Science, Engineering, or related field.
• Seven years of experience processing continuous data streams using Apache Flink.
• Five years of experience with Flink, Spark, HBase, Azkaban, and Airflow for building data pipelines and ETL.
• Three years of experience with real-time data feeds using Apache Kafka.
• Experience storing data, running servers, monitoring logs, and managing permissions using AWS Data Stack (S3, EC2, CloudWatch, IAM).
• Experience automating tasks in Unix with Linux CLI and Bash.

Responsibilities

• Provide technical leadership in building and maintaining data infrastructure and products for the platform.
• Process continuous data streams in near real-time using Apache Flink.
• Use Prometheus and Grafana for monitoring health and alerts.
• Build data pipelines with Flink, Spark, HBase, Azkaban, and Airflow; move data via Apache Kafka.
• Manage data storage, servers, logs, and permissions with AWS Data Stack (S3, EC2, CloudWatch, IAM).
• Automate tasks in Unix environment using Linux CLI and Bash; assist with software updates and rollouts.
• Collaborate with stakeholders to identify requirements; document development activity and technical designs; mentor junior engineers.

Skills

Apache Flink

Kafka

Spark

HBase

Azkaban

Airflow

Prometheus

Grafana

S3

EC2

CloudWatch

IAM

Education

Bachelor’s degree in Computer Science, Engineering, or related field

Tools

AWS Data Stack

S3

EC2

CloudWatch

IAM

Job description

FreeWheel, a Comcast company, seeks a Lead Software Engineer to drive data infrastructure for the advertising platform. You will lead real-time data processing with Flink, develop ETL pipelines with Spark and Airflow, and oversee data movement via Kafka, all while ensuring system health with Prometheus and Grafana.

You will mentor junior engineers, document designs, and collaborate with stakeholders to define requirements. Remote/hybrid work options are available as per policy.