Data Engineer II-9682

Comcast; Philadelphia (Philadelphia County); Remote; USD 80,000 - 120,000; Full time

Job summary

Comcast is seeking a Data Engineer to transform large datasets into business applications for analytics. This role involves analysis using SQL and Spark SQL, and automating ETL jobs with AWS services. Join a leader in the advertising tech industry that promotes innovation and equality.

Qualifications

• At least 1 year of experience in data analysis using SQL and Spark SQL.
• Experience in developing and automating ETL jobs with AWS services.
• Proficient in performing data quality checks and exploratory analysis with PySpark.

Responsibilities

• Transform complex data into business databases and applications for analytics.
• Develop and automate ETL jobs using Databricks, Jupyter, and AWS services.
• Design system architecture and conduct data quality checks on large datasets.

Skills

Spark SQL

PySpark

AWS Lambda

Data Quality

Education

Bachelor’s degree in Computer Science, Engineering, or related field

Tools

Databricks

Jupyter

AWS services

Job description

Comcast Advertising is driving the TV advertising industry forward, from delivering ads to linear and digital audiences to pioneering the tech that makes it possible. We help brands connect with their audiences on every screen using advanced data, technology, and premium video content. Our media sales division helps local, regional, and national brands reach potential customers through multiscreen TV advertising. Our ad tech division FreeWheel provides comprehensive adtech that makes it easier to buy and sell premium video advertising across all screens, data types, and sales channels.

Job Summary

We are seeking a Data Engineer to transform large, complex data into consumable business databases and applications for self-service analytics and reporting. The role involves performing data analysis using SQL and Spark SQL, developing and automating ETL jobs, and ensuring data quality across large datasets. The engineer will design system architecture, collaborate with data modelers and analysts, and optimize data storage and processing. This position is eligible for 100% remote work.

Responsibilities

1. Transform large, complex data into business databases and applications for analytics and reporting.
2. Perform data analysis using SQL and Spark SQL queries.
3. Develop ETL jobs using Databricks, Jupyter, and Pyspark.
4. Automate ETL jobs using AWS services such as S3, Athena, Lambda, RDS, Glue, and EMR.
5. Conduct data quality checks and exploratory data analysis on large datasets using PySpark.
6. Design system architecture and specifications to solve development problems.
7. Collaborate with data modelers and analysts to understand business needs and create data assets.
8. Perform engineering tasks including database design, data manipulation, and data flow analysis.
9. Resolve technical issues and ensure quality standards for databases and applications.
10. Implement solutions within big data systems like Hadoop and SQL.
11. Work with business and technical teams to ensure data relevance and actionability.
12. Optimize data storage and organize data for efficient access.
13. Develop relational models and determine table relations.
14. Collaborate with platform management to enhance data sourcing and processing.

Requirements

Bachelor’s degree in Computer Science, Engineering, or related field, and at least 1 year of experience in:

• Data analysis using SQL and Spark SQL
• Developing ETL jobs with Databricks, Jupyter, Pyspark
• Automating ETL with AWS services including S3, Athena, Lambda, RDS, Glue, EMR
• Performing data quality checks and exploratory analysis with PySpark

Additional Information

Comcast is proud to be an equal opportunity workplace. We consider all qualified applicants without regard to race, color, religion, age, sex, sexual orientation, gender identity, national origin, disability, veteran status, genetic information, or any other legally protected status.

Skills

Spark SQL, PySpark, AWS Lambda, Data Quality