Senior Data Engineer: Real-Time Data & Lakehouse

FOX; Los Angeles (CA); Hybrid; USD 184,000 - 202,000; Full time

Benefits offered by this job

Medical/dental/vision

401(k) plan

Paid time off

Job summary

FOX Corporation is seeking a Senior Data Engineer within the Data Platforms team to build robust, scalable data engineering solutions using a modern tech stack in a fast-paced environment.

You will collaborate with Agile teams to design, develop and optimize data pipelines, focusing on data quality, security and real-time ingestion. Remote/onsite flexibility includes up to 25% telecommuting within the LA area.

Qualifications

• Bachelor's degree in Computer Science, Computer Engineering, Data Science, Mathematics, Statistics, Engineering or related field plus five years of progressive data engineering experience.
• Experience building data pipelines with Python and PySpark.
• Experience building Lakehouse or batch/streaming data pipelines in Databricks.

Responsibilities

• Collaborate with Agile teams to design and develop data engineering solutions.
• Build distributed, low-latency data pipelines with high availability and timely delivery of data.
• Design and develop optimized data engineering solutions for Big Data workloads to handle increasing data volume.
• Build real-time data ingestion solutions for streaming workloads.
• Adhere to best practices and design patterns across Data Engineering solutions.
• Ensure code is clean, performant and well-tuned for production.
• Focus on data quality and implement monitoring to ensure production data accuracy.
• Create design and test documentation (Data Flow Diagrams, Design Specs, Source-to-Target mappings).
• Perform data analysis to troubleshoot issues and support resolution.
• Automate end-to-end data pipelines and validations; enforce data security and privacy with access controls and encryption.
• Proactively learn new technologies and educate the team, collaborating with analytics to improve data models for BI.

Skills

Python

PySpark

Data modeling

SQL

Cloud data engineering

Education

Bachelor's degree in Computer Science/Engineering/Data Science/Math/Statistics or related field

Tools

Snowflake

Fivetran

dbt

Kafka

Kinesis

Spark

Hadoop

Flink

Airflow

Astronomer

AWS S3

AWS EC2

AWS EMR

AWS Lambda

AWS RDS

AWS Redshift

AWS Glue

Job description

FOX Corporation is seeking a Senior Data Engineer within the Data Platforms team to build robust, scalable data engineering solutions using a modern tech stack in a fast-paced environment.

You will collaborate with Agile teams to design, develop and optimize data pipelines, focusing on data quality, security and real-time ingestion. Remote/onsite flexibility includes up to 25% telecommuting within the LA area.