Lead Data Engineer: Pipelines & Platform Architecture

Comcast (CC) of Willow Grove; Reston (VA); On-site; USD 158,000 - 237,000; Full time

Job summary

FreeWheel, a Comcast company, leads the design and management of sophisticated data engineering solutions to enhance decision-making and operational efficiency. The role oversees robust data pipelines and storage systems with a focus on high data quality and accessibility.

The position involves strategizing data architecture across platforms, driving data-centric objectives, and collaborating across teams to refine sourcing and processing while maintaining QA standards.

Qualifications

• Bachelor's degree required.
• 10+ years of experience in data engineering.
• Experience designing data pipelines and data architecture.

Responsibilities

• Lead data structure and pipeline development to standardize, transform, and deliver data for insights and reporting.
• Ensure data quality throughout the lifecycle with data quality frameworks.
• Design standardized methods for data consumption (APIs, database views, data extracts).
• Design and deploy data solutions on Kubernetes, Teradata, AWS, and Databricks to meet business needs.
• Choose data storage solutions based on sensitivity and access needs (minIO, Teradata, AWS S3, Redshift).
• Map data lineage and apply transformation rules to streamline troubleshooting and impact analysis.
• Collaborate with cross-functional teams to refine data sourcing and processing and optimize operations.
• Set design standards and QA protocols for software/systems development to ensure interoperability.
• Maintain regular attendance and be able to work nights and weekends as needed.

Skills

Apache Spark

Databricks Platform

Data Engineering

Java

SQL

Education

Bachelor's Degree

Tools

Kubernetes

Teradata

AWS

Databricks

MinIO

S3

Redshift

Job description

FreeWheel, a Comcast company, leads the design and management of sophisticated data engineering solutions to enhance decision-making and operational efficiency. The role oversees robust data pipelines and storage systems with a focus on high data quality and accessibility.

The position involves strategizing data architecture across platforms, driving data-centric objectives, and collaborating across teams to refine sourcing and processing while maintaining QA standards.