Weather Data Engineer

Balyasny Asset Management LP; City Of London; On-site; GBP 80,000 - 100,000; Full time

Job summary

A leading asset management firm in London is seeking a highly skilled Weather Data Engineer to build robust weather analytics infrastructure. The role involves designing and maintaining cloud-native data pipelines on AWS, developing APIs for data access, and collaborating with experts to optimize weather datasets. Candidates should have over 5 years of experience in data systems, proficiency in Python, and a strong background in environmental data. This role offers an exciting opportunity at the intersection of data engineering and meteorology.

Qualifications

• 5+ years of hands-on development experience building and supporting production data systems.
• Strong understanding of system architecture and the full technology stack.
• Experience working with weather, climate, or environmental datasets.

Responsibilities

• Design, implement, and maintain scalable cloud-native data pipelines.
• Develop APIs and data services for efficient weather data access.
• Collaborate with meteorologists and data scientists to optimize weather datasets.

Skills

Python

Cloud-native data pipelines (AWS)

Database technologies (Postgres, Redshift, Snowflake)

Time-series data

Geospatial data concepts

Education

Degree in Computer Science, Atmospheric Science, Engineering

Tools

Docker

Kubernetes

CI/CD tools

Monitoring tools (CloudWatch, Prometheus)

Job description

Overview

BAM is seeking a highly skilled and experienced Weather Data Engineer to help build a proprietary, differentiated weather analytics & infrastructure. Weather data and AI-driven modeling are at the core of our strategy to deliver a forecasting edge and drive Sharpe improvement for our trading business.

As a key member of our team, you will design, implement, and maintain cloud-native (AWS) data pipelines and infrastructure for ingesting, processing, and serving real-time and historical weather data - including raw observations, climate simulation data, and AI model outputs. You will play a critical role in supporting our Commodity teams AI efforts, supporting the development and deployment of proprietary, decorrelated forecasting models, and enabling advanced analytics and research.

You will collaborate closely with meteorologists, data scientists, technologists and AI researchers to integrate new weather datasets, support signal postprocessing and debiasing, and drive innovation in weather analytics. You will also work with central BAM resources to ensure seamless integration and operational excellence.

Responsibilities

• Design, implement, and maintain scalable, cloud-native (AWS) data pipelines for ingesting, processing, and storing real-time and historical weather data, including raw observations (e.g., satellite, radar, sensor networks) and climate simulation data (e.g., CMIP6).
• Develop and maintain robust APIs and data services to enable efficient access to weather data and AI model outputs for analytics, modeling, and visualization.
• Support the centralization and optimization of AI model infrastructure, including model blending, debiasing, and finetuning workflows.
• Collaborate with meteorologists, data scientists, and AI researchers to onboard, profile, and optimize new weather datasets and support research projects (e.g., initial condition research, uncertainty quantification, dashboarding).
• Implement and automate data quality validation, monitoring, and alerting to ensure high reliability and availability of all weather data feeds.
• Continuously improve data infrastructure to accelerate analytics, reduce time to insight, and enhance operational scale and stability.
• Champion best practices in collaborative software development: version control, CI/CD, automated testing, code review, and refactoring.
• Maintain clear documentation and promote knowledge sharing within the team.

Requirements

Essential

• Degree in Computer Science, Atmospheric Science, Engineering, or a related field with a computational focus.
• 5+ years of hands-on development experience building and supporting production data systems.
• Highly skilled in Python, comfortable with different programming styles (e.g., OO, functional), and strong on design patterns.
• Strong understanding of system architecture and the full technology stack (software, OS, CPU/memory, local/network storage, networking, etc.).
• Experience with collaborative software development: version control, CI/CD, automated testing, code review, and refactoring.
• Strong knowledge of one or more relevant database technologies (e.g., Postgres, Redshift, Snowflake).
• Solid understanding of time-series data, temporal queries, and geospatial data concepts.
• Experience with Linux platforms and related scripting.
• Experience working with weather, climate, or environmental datasets (e.g., GRIB, NetCDF, HDF5, CSV, JSON).
• Familiarity with weather data sources and formats (e.g., NOAA, ECMWF, GFS, satellite, radar, sensor networks).

Beneficial

• Proficient in one or more OO programming languages (e.g., Java, C#).
• Experience with distributed computing frameworks (e.g., Spark, Dask, Slurm).
• Experience with event-driven, asynchronous architectures and messaging technologies (e.g., Kafka, RabbitMQ).
• Experience with cloud platforms (e.g., AWS, GCP, Azure).
• Experience with orchestration and container technologies (e.g., Airflow, Kubernetes, Docker).
• Experience with monitoring and alerting tools (e.g., CloudWatch, Prometheus, Grafana, Sentry/OTel).
• Familiarity with weather modeling, forecasting, or analytics workflows.
• Experience with dashboarding, uncertainty quantification, and supporting research analytics.

If you are passionate about building world-class data infrastructure for weather analytics and want to work at the intersection of data engineering, meteorology, and advanced AI-driven analytics, we would love to hear from you!