Data Engineer - Cloud Pipelines, PySpark & Streaming

Blueface Ltd; West Whiteland Township (Chester County); On-site; USD 120,000 - 180,000; Full time

Job summary

Comcast is seeking a Data Engineer to join the Data Product Engineering team responsible for managing and evolving the enterprise Data Lake that supports critical datasets across the GTO organization. This role focuses on building scalable, reliable data solutions using AWS, PySpark, Databricks, Kafka, Kubernetes, and Airflow to enable analytics and business decision-making.

The ideal candidate will design and optimize distributed data pipelines, collaborate with cross-functional teams, and

Qualifications

• Bachelor's degree in Computer Science, Engineering, Information Systems, or related field, or equivalent practical experience.
• 5+ years of experience in Data Engineering, Data Platform Engineering, or related disciplines.
• Strong hands-on experience with: AWS PySpark Databricks
• Experience building and maintaining large-scale ETL/ELT pipelines.
• Strong understanding of distributed systems and large-volume data processing.
• Experience with data warehousing concepts and modern data architectures.
• Experience orchestrating workflows using Apache Airflow/MWAA.
• Knowledge of Kubernetes fundamentals, including pod lifecycle, job orchestration, and workload configuration.
• Proficiency in Python development within data engineering environments.
• Experience supporting production data platforms and driving operational excellence.
• Strong communication and collaboration skills with the ability to work effectively across multiple teams.

Responsibilities

• Design, develop, maintain, and optimize scalable data pipelines supporting workforce, billing, interaction, and other enterprise datasets.
• Build and enhance cloud-native data solutions using AWS, Databricks, and PySpark.
• Develop and support batch and streaming data processing frameworks, integrating source systems and interfaces through modern data architectures.
• Leverage technologies such as Kafka and Databricks streaming solutions to ingest and process high-volume data in near real-time.
• Drive data pipeline performance tuning, automation initiatives, and operational improvements across the platform.
• Provide production support, troubleshooting, and root-cause analysis for critical data workflows.
• Work with large-scale distributed systems and high-concurrency environments processing tens of terabytes of data.
• Utilize MWAA (Managed Workflows for Apache Airflow) to orchestrate and manage data workflows.
• Collaborate closely with Product, Data Governance, Analytics, and Engineering teams across both onshore and offshore delivery models.
• Support data warehousing initiatives and help establish best practices for data quality, scalability, and reliability.
• Mentor junior engineers, provide technical guidance, and contribute to the growth and development of Engineering I team members.
• Participate in architectural discussions and contribute to the long-term evolution of the enterprise data platform.

Skills

AWS

PySpark

Databricks

Data Engineering

ETL Pipelines

Mentorship

Communication

Education

Bachelor's degree in Computer Science, Engineering, Information Systems, or related field, or equivalent practical experience

Tools

MWAA

Kubernetes

Airflow

Spark

Job description

Comcast is seeking a Data Engineer to join the Data Product Engineering team responsible for managing and evolving the enterprise Data Lake that supports critical datasets across the GTO organization. This role focuses on building scalable, reliable data solutions using AWS, PySpark, Databricks, Kafka, Kubernetes, and Airflow to enable analytics and business decision-making.

The ideal candidate will design and optimize distributed data pipelines, collaborate with cross-functional teams, and