data engineer for the Net-Zero Data Public Utility

Bloomberg; Greater London; On-site; GBP 95,000 - 125,000; Full time

Job summary

HireHi seeks a senior data engineer to join the Climate Data Utility Tech Team in London. Design and develop scalable Python data pipelines to read from APIs and files, transforming data for downstream use.

You will mentor juniors, document pipelines, and deploy via CI/CD to the cloud, working with data scientists to enable data-driven decisions.

Qualifications

• 7+ years of experience in data engineering or a similar role.
• Proficiency in Python and building data pipelines.
• Familiarity with data warehousing and ETL processes.
• Excellent problem-solving and communication skills.

Responsibilities

• Design and develop scalable Python data pipelines reading from APIs and files.
• Translate domain- and source-specific business logic into efficient code that transforms source data into structured data for downstream applications.
• Implement data transformations and structuring with Pandas and Pydantic to ensure data quality, consistency, and compliance with business logic.
• Collaborate with data scientists and analysts to support data-driven decision-making.
• Document data engineering processes, including data lineage, data dictionaries, and system architectures.
• Maintain best practices through code reviews, version control, and adherence to industry standards.
• Deploy production-quality code through CI/CD pipelines into the cloud environment.
• Mentor and guide junior team members in data engineering practices.

Skills

Python

Data Pipelines

Pandas

Pydantic

CI/CD

Cloud

Data Warehousing

Job description

Описание:

The Climate Data Utility Tech Team develops scalable data pipelines for the Net-Zero Data Public Utility, which provides open and accessible public-good data through its website and APIs.

Задачи:

Design and develop scalable Python data pipelines to read from APIs and structured files such as Excel and Parquet Translate domain- and source-specific business logic into efficient code that transforms source data into structured data for downstream applications Implement data transformations and structuring with Pandas and Pydantic to ensure data quality, consistency, and compliance with business logic Collaborate with data scientists and analysts to support data-driven decision-making Document data engineering processes, including data lineage, data dictionaries, and system architectures Maintain best practices through code reviews, version control, and adherence to industry standards Deploy production-quality code through CI/CD pipelines into the cloud environment Mentor and guide junior team members in data engineering practices

Требования:

7+ Years of experience in data engineering or a similar role Proficiency in Python Experience building and managing data pipelines Knowledge of data warehousing and ETL processes Excellent problem-solving and communication skills

Условия:

Location: London, England, United Kingdom