Data Engineer: Build Scalable Data Pipelines

YouGov; EUA; Teletrabalho; USD 120 000 - 180 000; Tempo integral

Resumo da oferta

You will join YouGov’s Data Enablement Team to build and maintain foundational data infrastructure powering decision-making across the company. The role focuses on ingestion, transformation, and modeling of data to ensure clean, reliable datasets accessible through our centralized warehouse.

Utilizing Snowflake, dbt, Python, and Airflow, you’ll enable downstream teams like product, finance, and marketing to own their analytics.

Qualificações

• Bachelor’s degree in computer science or related field.
• Proven years of experience in a Data Engineering or Software Engineering role.
• Very good knowledge of Python and experience with SQL.
• Experience with Kubernetes and containerization.
• Experience with Airflow or similar process orchestration tool.
• Familiarity with modern data file formats (Parquet, ORC, JSON).
• Experience with data architecture, modelling, and pipelines in an enterprise data warehouse (Snowflake, Redshift, BigQuery).
• Experience with AWS products (S3, Lambda, Athena).
• Ability to work both independently and in a team-oriented environment.
• Strong analytical skills with a critical thinking attitude.
• Engage in a highly collaborative environment, working closely with data analysts and other team members.
• Experience firsthand the impact of your contributions on the overall success of the team and organization.
• Demonstrable experience as a software or data engineer, demonstrating a good understanding of data architecture and engineering principles.
• You have great communication skills both written and verbal.
• You embrace the YouGov values.

Responsabilidades

• Design, develop, and maintain robust, scalable data pipelines using Python and Airflow.
• Collaborate with Analytics Engineers and other team members to deliver clean, well-modeled data to downstream users.
• Implement monitoring, logging, and alerting to ensure data pipeline reliability and performance.
• Ensure data quality and integrity through validation frameworks and testing.
• Optimize performance and cost-efficiency of cloud-based data systems.
• Contribute to architectural decisions around data storage, workflow orchestration, and infrastructure as code.
• Document data pipelines and processes

Conhecimentos

Python

SQL

Kubernetes

Containerization

Airflow

Data modeling

Data architecture

Formação académica

Bachelor's degree in computer science or related field

Ferramentas

Snowflake

Redshift

BigQuery

S3

Lambda

Athena

Airflow

DBT

Descrição da oferta de emprego

You will join YouGov’s Data Enablement Team to build and maintain foundational data infrastructure powering decision-making across the company. The role focuses on ingestion, transformation, and modeling of data to ensure clean, reliable datasets accessible through our centralized warehouse.

Utilizing Snowflake, dbt, Python, and Airflow, you’ll enable downstream teams like product, finance, and marketing to own their analytics.