Senior Platform Engineer, Spark & Kubernetes Platform

Comcast; West Chester (Chester County); On-site; USD 140,000 - 190,000; Full time

Job summary

Comcast in the United States is seeking a Sr. Platform Engineer to build, manage, and optimize the data processing infrastructure that powers large-scale workloads.

You will design systems for metrics collection with Prometheus and visualization with Grafana, ensuring scalable and cost-efficient Spark execution across Kubernetes and cloud services. You'll develop automation in Python/Scala, deploy Terraform/Ansible, and collaborate with data engineers and scientists to improve data catalogs like

Qualifications

• 7–10 years of relevant DevOps or Systems Engineering experience.
• Expertise in Apache Spark with deep understanding of its architecture.
• Proficiency in Python, PySpark, and Scala/Java.
• CI/CD experience with GitHub and automation tooling.
• Experience with Prometheus, Grafana for observability.
• Cloud and on-prem environments across AWS/Azure/GCP; Kubernetes/Docker.
• Terraform/Ansible for infrastructure as code.
• Networking knowledge and data lake governance.
• Familiar with Iceberg, Unity Catalog, Kafka, Hadoop, and storage like S3/HDFS.

Responsibilities

• Architect and manage Spark platforms on Kubernetes and AWS EKS.
• Package Spark workloads with Docker/Kubernetes and integrate with Flyte.
• Deploy infrastructure via Terraform/Ansible.
• Troubleshoot and optimize Spark configurations to reduce costs.
• Build automation in Python/Java/Scala and Bash.
• Write medium to complex SQL queries as needed.
• Implement monitoring/logging with Prometheus and Grafana.
• Develop and optimize data catalog platforms like Iceberg/Unity Catalog.
• Automate workflows and incident resolution.
• Collaborate with data engineers/scientists to address data needs.
• Engage with open-source communities and contribute improvements.
• Document Kubernetes infrastructure and provide team training.

Skills

Apache Spark

Python

Scala/Java

Linux scripting

Terraform/Ansible

Kubernetes

Docker

SQL

Prometheus

Grafana

CI/CD

Networking

Education

Bachelor’s degree in computer science

Tools

Databricks

Snowflake

Unity Catalog

Apache Iceberg

Hadoop

S3

AWS

Azure

GCP

Terraform

Ansible

GitHub

Docker

Job description

Comcast in the United States is seeking a Sr. Platform Engineer to build, manage, and optimize the data processing infrastructure that powers large-scale workloads.

You will design systems for metrics collection with Prometheus and visualization with Grafana, ensuring scalable and cost-efficient Spark execution across Kubernetes and cloud services. You'll develop automation in Python/Scala, deploy Terraform/Ansible, and collaborate with data engineers and scientists to improve data catalogs like