Senior Data Engineer - Entertainment / Media Domain

YO IT Consulting; Glendale (AZ); On-site; USD 120,000 - 150,000; Full time

Job summary

YO IT Consulting is seeking a Sr Data Engineer for a 12-month contract role based in Glendale, AZ. The role requires a minimum of 5 years of experience in data engineering, focusing on large-scale data pipelines. Candidates should be proficient in Python, SQL, and technologies like Spark and Airflow. Responsibilities include maintaining data pipelines, building APIs, and ensuring data quality and operational efficiency. The position is onsite 4 days a week and offers a collaborative environment with product managers and other engineers.

Qualifications

• 5+ years of data engineering experience developing large data pipelines.
• Proficiency in at least one major programming language.
• Hands-on production experience with distributed processing systems.

Responsibilities

• Contribute to maintaining existing Core Data platform data pipelines.
• Build and maintain APIs to expose data to downstream applications.
• Develop real-time streaming data pipelines.

Skills

Data engineering experience

Spark

Airflow

Databricks

SQL

Python

Tools

Snowflake

Delta Lake

Job description

Overview

Job Title: Sr Data Engineer (Contract Role for 12 Months)
Domain: Entertainment/Media/Publishing
Location: Glendale, CA - Onsite 4 days a week
Experience: 5-20 Years

Must-Haves

• 5+ years of data engineering experience specifically developing large-scale data pipelines.
• Spark, Airflow, Databricks or Snowflake, SQL, Python.

Responsibilities

• Contribute to maintaining, updating, and expanding existing Core Data platform data pipelines.
• Build and maintain APIs to expose data to downstream applications.
• Develop real-time streaming data pipelines.
• Tech stack includes Airflow, Spark, Databricks, Delta Lake, and Snowflake.
• Collaborate with product managers, architects, and other engineers to drive the success of the Core Data platform.
• Contribute to developing and documenting both internal and external standards and best practices for pipeline configurations, naming conventions, and more.
• Maintain high operational efficiency and data quality across the Core Data platform datasets, ensuring our solutions consistently meet SLAs while delivering reliability and accuracy to all stakeholders, including Engineering, Data Science, Operations, and Analytics teams.

Required Qualifications

• 5+ years of data engineering experience developing large data pipelines.
• Proficiency in at least one major programming language (e.g., Python, Java, Scala).
• Hands-on production experience with distributed processing systems such as Spark.
• Hands-on production experience with data pipeline orchestration systems such as Airflow for creating and maintaining data pipelines.
• Experience with at least one major Massively Parallel Processing (MPP) or cloud database technology (Snowflake, Databricks, BigQuery).
• Experience in developing APIs with GraphQL.
• Advanced understanding of OLTP vs OLAP environments.
• Strong background in at least one of the following: distributed data processing or software engineering of data services, or data modeling.