Data Engineer 2

Comcast Corporation; Willow Grove (PA); Presencial; USD 80.000 - 133.000; Jornada completa

Descripción de la vacante

Comcast Advertising is building large-scale data products to power reliable analytics for many users. This onsite Data Engineering role in Willow Grove, PA focuses on designing high-performance data pipelines on AWS and implementing data lake architectures using Delta Lake and Apache Iceberg.

You will lead data quality, observability, and best-practice governance, and collaborate with Product and Leadership to deliver semantic layers for self-service analytics.

Formación

• Bachelor's degree required.
• 2–5 years of data engineering experience preferred.
• Proficiency in Python, SQL, and Spark required.

Responsabilidades

• Architect scalable data pipelines for large-scale data processing.
• Develop complex Big Data ETL workflows using Databricks Delta Live Tables (DLT).
• Design and implement data lake architectures (Delta Lake, Apache Iceberg).
• Define and enforce data engineering best practices for quality, integrity, reliability, and performance.
• Lead observability and monitoring across pipelines, including metadata management.

Conocimientos

Python

PySpark

SQL

Databricks

Snowflake

Data Warehouse

Educación

Bachelor's Degree

Herramientas

DLT

Delta Lake

Apache Iceberg

Looker

Tab

Snowflake

AWS

Apache Spark

Descripción del empleo

Comcast Advertising is building large-scale data products that power reliable analytics for many users. In this onsite Data Engineering role in Willow Grove, PA, you will design and deliver high-performance data pipelines on AWS, using modern big data tooling and open table formats. You’ll help enable self-service analytics by focusing on data architecture, quality, and operational excellence.

What you’ll do

• Architect, design, and build scalable data pipelines for large-scale data processing needs.
• Develop complex Big Data ETL workflows using frameworks such as Databricks Delta Live Tables (DLT).
• Design and implement data lake architectures, including Delta Lake and Apache Iceberg open table formats.
• Define, promote, and enforce data engineering best practices to support data quality, integrity, reliability, and performance.
• Lead observability and monitoring across pipelines, including data cataloging, lineage, and metadata management.
• Support delivery through peer reviews, knowledge sharing, production support, incident resolution, and change management.
• Follow strong code management and version control practices across development workflows.
• Create high-quality documentation (data flows, lineage diagrams, and architecture details) aligned to engineering standards.
• Ensure comprehensive coverage by building unit and integration tests using TDD principles.
• Own critical components of large-scale batch ETL systems, articulating data models and design decisions to Product, Engineering, and Leadership stakeholders.
• Build advanced semantic layers and support customer-facing self-service analytics solutions.
• Use independent judgment and discretion on complex, high-impact technical decisions.
• Maintain regular, consistent attendance with flexibility to work nights, weekends, or variable schedules as required.
• Work with semantic layer technologies such as Looker, Tab, Snowflake, and Databricks.
• Perform additional duties and responsibilities as assigned.

Skills and experience

• Minimum 2 years of Data Engineering experience.
• Proficiency in Python and PySpark, with a good grasp of SQL.
• Hands-on experience with Databricks and/or Snowflake is preferred.
• Experience with Data Warehouse modeling paradigms.
• Understanding of cloud-based architectures (AWS).
• Strong problem-solving skills.
• Nice to have certifications:
  • AWS Certified Developer Associate
  • Databricks Certified Data Engineer Associate
  • SnowPro Advanced: Data Engineer
• A Bachelor’s Degree.
• Relevant work experience typically falls within 2 to 5 years.

Tools you’ll work with

SQL, Python, PySpark, Apache Spark, AWS, Databricks Delta Live Tables (DLT), Databricks, Delta Lake, Apache Iceberg, Looker, Tab, Snowflake

Compensation and timeline

Pay Range: USD 79,728 - 132,880 per year. Comcast intends to offer base pay within this range based on job-related, non-discriminatory factors such as experience.

Application window: 30 days from the date the job is posted, unless the number of applicants requires it to close sooner or later.