ML Evaluation & Metrics Engineer — Hybrid Role

WindBorne Systems; Atlanta (GA); Hybrid; USD 140,000 - 240,000; Full time

Benefits offered by this job

401(k)

Dental, health, and vision insurance

Unlimited PTO

Stock Option Plan

Office food and beverages

Job summary

WindBorne Systems is seeking a Machine Learning Research Engineer focused on model evaluation for our global weather forecasting platform. You will design evaluation strategies, run large-scale experiments, and build reusable tooling to quantify forecast quality across regions and lead times.

You will communicate results with clear scorecards and visualizations, collaborating with meteorology and research teams to ensure rigorous, credible assessments of AI and physics-based models.

Qualifications

• Excellent scientific judgment and healthy skepticism.
• Strong experimental taste to identify meaningful evaluation.
• Systems thinking to build reusable infrastructure.
• Experience evaluating ML systems with large, scientific, geospatial, multidimensional, or time-series data.
• Able to investigate ambiguous results independently, synthesize evidence, and communicate conclusions clearly.

Responsibilities

• Evaluation strategy — develop a rigorous, meteorologically valid strategy for comparing WeatherMesh with leading models.
• Fast feedback and durable systems — build quick evaluations and reusable infrastructure with reproducibility in mind.
• Agentic tooling for evals — improve evaluation infrastructure, including AI-based tools for forecasts analysis.
• Technical communication — produce clear scorecards, visualizations, and explanations for researchers and leadership.

Skills

Scientific judgment

Experimental taste

Systems thinking

ML system evaluation

Independent analysis & communication

Weather/Climate context helpful

Tools

PyTorch

NumPy

pandas

xarray

Job description

WindBorne Systems is seeking a Machine Learning Research Engineer focused on model evaluation for our global weather forecasting platform. You will design evaluation strategies, run large-scale experiments, and build reusable tooling to quantify forecast quality across regions and lead times.

You will communicate results with clear scorecards and visualizations, collaborating with meteorology and research teams to ensure rigorous, credible assessments of AI and physics-based models.