Senior Data Platform Engineer — Build Core Pipelines

EndGame Economics; Sydney; Híbrido; AUD 140.000 - 190.000; Jornada completa

Descripción de la vacante

Endgame Economics is seeking a Senior Data Engineer to help define and build its nascent data platform. You will own core data pipelines feeding optimisation models and establish standards for data quality, testing and monitoring.

Reporting to the Head of Engineering, you’ll work with consultants and modellers to deliver robust, reusable data solutions. You’ll leverage SQL, Python, dbt, and cloud data warehouses, with experience in ELT tools like Airbyte or Fivetran and orchestration such as

Formación

• Strong SQL and data modelling skills, with experience in dbt and testing.
• Proficient Python for building ingestion/transformation pipelines.
• Experience building ELT pipelines with an ingestion tool and a workflow orchestrator.
• Familiarity with cloud data warehouses and structuring reusable data assets.
• Solid software engineering basics: version control, testing, CI/CD, and documentation.
• Experience in data quality/validation tooling (dbt tests, Great Expectations).
• Comfort with ambiguous data and independent requirement gathering.
• Ability to translate data needs to non-specialists and engineers.

Responsabilidades

• Lead technical design and architectural direction for the new data platform.
• Build and maintain data pipelines sourcing and shaping data for optimisation models.
• Set data engineering standards for quality, testing, and monitoring.
• Collaborate with stakeholders to translate complex needs into data solutions.

Conocimientos

SQL

Data modelling

dbt

Python

ELT pipelines

Airbyte / Fivetran / dlt

Airflow / Dagster / Prefect

Cloud data warehouse

Snowflake / BigQuery / PostgreSQL

Git / CI-CD / testing

Data quality tooling

Communication with non-specialists

Energy market domain knowledge

Herramientas

DBT tests

Great Expectations

Descripción del empleo

Endgame is a boutique, technology-led consultancy providing expert mathematical and economic advice to clients across energy, aviation, road and rail, and water. We combine advanced mathematical optimisation and machine learning with deep sector expertise. As a Gurobi Premier Partner, we build models that sit at the core of how clients make high-stakes decisions.

We’re a small, flat team of consultants and software engineers. There’s no bureaucracy to navigate here; the people who build things are the people who decide how they get built.

THE ROLE

We’re looking for a Senior Data Engineer to join us at a genuinely early stage: our internal data platform is still nascent, and you’ll be one of the people setting its foundations rather than inheriting someone else’s decisions.

Reporting to the Head of Engineering, you’ll own two connected pieces of work: continuing to build out our data platform from the ground up and contributing to the pipelines that feed our on-demand optimisation models.

WHAT YOU’LL BE DOING

• Lead the technical design and architectural direction of our new data platform, and own its technical vision as it matures beyond the first version
• Build and maintain the pipelines that source, clean, and structure data feeding our optimisation and modelling work, including data pulled from external market and regulatory sources
• Set the standard for data engineering across the firm, including data quality, testing and monitoring practices
• Collaborate closely with stakeholders across the business to translate complex or ambiguous needs into effective data solutions

ABOUT YOU

• Strong SQL and hands-on data modelling, ideally with dbt (version-controlled models, testing and documentation).
• Strong Python for building ingestion and transformation pipelines.
• Experience building ELT pipelines with an ingestion tool (for example Airbyte, Fivetran, dlt or similar) and a workflow orchestrator (Airflow, Dagster, Prefect or similar).
• Experience with a cloud data warehouse or store (Snowflake, BigQuery, PostgreSQL or similar) and a feel for structuring data so it is reusable across engagements.
• Software engineering fundamentals: version control (Git), testing, CI/CD and documentation. You treat data pipelines as real, maintainable software.
• Experience with data quality and validation tooling (dbt tests, Great Expectations or similar).
• Comfort with unfamiliar, ambiguous data, and the confidence to chase up requirements independently.
• Clear communication with non-specialists, and the ability to translate between raw data and what engineers and modellers need.
• Previous experience in energy market domain, mathematical optimisation or ML highly regarded.

ABOUT US

We know that our people are our greatest asset. That’s why we invest in our team by fostering opportunities for growth, development, and recognition.

Learning & Development – You’ll work across energy, aviation, road and rail, and water, on problems that are genuinely hard rather than variations on the same build. Sitting alongside mathematicians and economists, you’ll pick up depth in optimisation, machine learning and the sectors we advise in, the kind of exposure that’s difficult to get anywhere but a small firm doing specialist work.

Reward & Recognition – Remuneration is competitive and reviewed regularly. In a team this size, the work you do is visible to everyone, and scope expands as quickly as you can take it on.

Flexibility & Autonomy – The two go hand in hand. Endgame advocates flexibility to balance work and personal life, as well as significant autonomy and ownership of the systems you build. You’ll be setting the direction of the data platform, not implementing someone else’s plan for it.

Collaboration – You’ll be the data engineering voice in a team of technical specialists, working closely with the consultants and modellers who depend on your pipelines. Expect code review, pairing and design discussions with engineers whose experience spans a range of industries and stacks.

APPLICATION PROCESS

To apply, please email [e-mail] with a copy of your CV.