Senior Data Engineer: Scale Big Data Pipelines

Ziff Davis; United States; Remote; USD 81,000 - 90,000; Full time

Job summary

Ookla, an Accenture company, seeks a Senior Data Engineer to join the Data Engineering team and help process the world’s best data on internet performance. You will develop tools, libraries and operations to support data scientists and analysts as they discover new and exciting ways to apply our data.

The role emphasizes scalable pipelines using Spark, Redshift, Athena, Iceberg and Airflow, with a focus on accuracy and performance.

Qualifications

• Bachelor's degree in Computer Science or related field or equivalent experience.
• Strong understanding of programming concepts with experience in at least 1 OOP language (Python, Scala, Java).
• Strong background and 3+ years of direct experience working with Python.
• Experience with high volume distributed data processing in Spark.
• Strong working knowledge of SQL, databases (MySQL / Redshift / Athena / Trino), data modeling, and modern data warehouse (lakehouse) management.
• Excellent troubleshooting and analysis skills.
• Excited to learn and work with new technologies in a dynamic environment.
• Strong communication skills both in person and via virtual mediums.
• Strong time management skills and a self driven work ethic.

Responsibilities

• Develop high volume data processing jobs across platforms including Redshift, Athena, Iceberg, Spark and Airflow.
• Work closely with Data Scientists to turn theory into viable products.
• Identify bottlenecks in system performance and improve process efficiency.
• Support and collaborate with data scientists, analysts, and product managers.
• Mentor and guide junior team members on architectural principles and best practices.

Skills

Python programming

SQL proficiency

Strong communication

Time management

Self-driven

Education

Bachelor's degree in Computer Science or related field

Tools

Spark

Airflow

MySQL

Redshift

Athena

Trino

Python

Job description

Ookla, an Accenture company, seeks a Senior Data Engineer to join the Data Engineering team and help process the world’s best data on internet performance. You will develop tools, libraries and operations to support data scientists and analysts as they discover new and exciting ways to apply our data.

The role emphasizes scalable pipelines using Spark, Redshift, Athena, Iceberg and Airflow, with a focus on accuracy and performance.