Senior Software Engineer (Python) - Supply & Demand

Socket.dev; United States; On-site; USD 140,000 - 210,000; Full time

Job summary

Kpler, a global data platform, is seeking a Senior Backend Software Engineer to help generalise the S&D data model across commodities and to evolve the shared ETL framework. You will own the publication reliability end-to-end, drive non-functional improvements, and shape API contracts for external clients.

You will collaborate with Product and Research, lead design reviews, and mentor mid-level engineers to grow into senior roles.

Qualifications

• 5+ years of experience as a software engineer, with experience delivering production systems.
• Experience delivering technically ambiguous, business-critical projects end to end, owning the design, the trade-offs and the production outcome, not just the implementation.
• Deep experience building and running data-intensive backends in Python (ETL/orchestration, relational data modelling, SQL/Postgres).
• Owning systems in production: on-call, observability, incident response and the prevention work afterwards ("you build it, you run it").
• Experience setting technical direction for a team and being the person other engineers consult on hard design decisions.

Responsibilities

• Lead the technical direction of generalising the S&D data model from crude oil to all commodities, owning the trade-offs between debt, design, reliability and time-to-market as the crew's call.
• Evolve the shared ETL Framework (Reader → Transformer → Writer) into the reusable foundation every commodity vertical builds on.
• Own publication reliability end to end: the anomaly-detection guardrail, the release path, and the data-quality bar that decides whether a balance is fit to ship to customers.
• Set and raise the non-functional bar (reliability, latency, observability, security, cost) across the crew's Python engine, TypeScript data-distribution APIs and Airflow orchestration.
• Shape the external + webapp data-distribution API contract that external clients depend on: versioning, pagination, backward compatibility.
• Lead design reviews within the crew and steward the crew's ADRs; bring engineering alignment to ambiguous, business-critical work.
• Own the response to critical incidents in the crew's systems, and drive the preventative work that follows so the same class of failure does not recur.
• Partner directly with Product and the Research team to shape the crew's technical and feature roadmap, including the SuDeP pipeline that Research consumes.
• Mentor mid-level (P3) engineers toward senior scope, and act as a cross-crew technical consultant within the Commodities Tribe and the wider Engineering chapter.

Skills

Data-intensive backend design

Production systems ownership

Technical leadership

Cross-team collaboration

Tools

Python

Airflow

TypeScript/NestJS

Terraform

ArgoCD

Kubernetes

Job description

At Kpler, we are dedicated to helping our clients navigate complex markets with ease. By simplifying global trade information and providing valuable insights, we empower organisations to make informed decisions in commodities, energy, and maritime sectors.

Since our founding in 2014, we have focused on delivering top-tier intelligence through user-friendly platforms. Our team of over 700 experts from 35+ countries works tirelessly to transform intricate data into actionable strategies, ensuring our clients stay ahead in a dynamic market landscape. Join us to leverage cutting-edge innovation for impactful results and experience unparalleled support on your journey to success.

Join Kpler’s Commodities Tribe as our next Senior Software Engineer and help shape the future of our Supply & Demand products.

Our Supply & Demand platform transforms complex global commodity data-from production sites, trade flows, imports, exports, refinery activity, consumption, and stock changes-into trusted market intelligence used by our clients around the world.

Historically focused on crude oil, our team is now evolving the platform to support a much broader range of commodities, including refined products, biofuels, metals, and beyond. This is a major engineering transformation: building a scalable, reusable platform that can support new commodity markets without creating separate solutions for each product.

As a Senior Backend Software Engineer, you will influence technical direction, solve complex engineering challenges, and help shape the systems behind one of Kpler’s core products. You will work closely with experienced engineers, Product, and Research teams to build solutions that combine strong engineering practices with meaningful business impact.s.

This role is ideal for an engineer who enjoys looking beyond individual features-someone who thinks about architecture, product impact, and building systems that stand the test of time.

Your future position:

• Lead the technical direction of generalising the S&D data model from crude oil to all commodities, owning the trade-offs between debt, design, reliability and time-to-market as the crew's call.
• Evolve the shared ETL Framework (Reader → Transformer → Writer) in `supply-demand-core` into the reusable foundation every commodity vertical builds on, so LNG, grains and new products stop diverging into bespoke pipelines.
• Own publication reliability end to end: the anomaly-detection guardrail (`supply-demand-ml`), the release path, and the data-quality bar that decides whether a balance is fit to ship to customers.
• Set and raise the non-functional bar (reliability, latency, observability, security, cost) across the crew's Python engine, TypeScript data-distribution APIs and Airflow orchestration.
• Shape the external + webapp data-distribution API contract (`supply-demand-oil-api`, `supply-demand-grains-api`) that external clients depend on: versioning, pagination, backward compatibility.
• Lead design reviews within the crew and steward the crew's ADRs; bring engineering alignment to ambiguous, business-critical work.
• Own the response to critical incidents in the crew's systems, and drive the preventative work that follows so the same class of failure does not recur.
• Partner directly with Product and the Research team to shape the crew's technical and feature roadmap, including the SuDeP pipeline that Research consumes.
• Mentor mid-level (P3) engineers toward senior scope, and act as a cross-crew technical consultant within the Commodities Tribe and the wider Engineering chapter.

Experience & Background

Essential:

• 5+ years of experience as a software engineer, with experience delivering production systems.
• Experience delivering technically ambiguous, business-critical projects end to end, owning the design, the trade-offs and the production outcome, not just the implementation.
• Deep experience building and running data-intensive backends in Python (ETL/orchestration, relational data modelling, SQL/Postgres).
• Owning systems in production: on-call, observability, incident response and the prevention work afterwards ("you build it, you run it").
• Experience setting technical direction for a team and being the person other engineers consult on hard design decisions.

Desirable:

• Worked with Airflow (or comparable orchestration) and TypeScript/NestJS APIs, or can credibly ramp into a polyglot Python + TS estate.
• Exposure to a real, evolving domain data model: commodities, energy, finance, logistics or another domain where correctness of the model is the product.
• Familiarity with data-quality / anomaly-detection approaches, or the discipline of gating publication on quality.
• Experience with IaC and GitOps delivery (Terraform, ArgoCD, k8s) and with serving versioned external client APIs.

Behavioural Competencies:

• You take ownership of problems and outcomes, not just individual tasks.
• You think beyond the code and understand how engineering decisions impact the product and customers.
• You are comfortable making technical decisions, challenging ideas, and helping shape the direction of the team.
• You enjoy working on complex problems, learning new things, and adapting in a fast-moving environment.
• You collaborate openly with engineers, Product, and Research teams to find the best solutions.
• You share knowledge and help the people around you grow.

We make things happen

We act decisively and with purpose, going the extra mile.

We buildtogether

We foster relationships and develop creative solutions to address market challenges.

We are here to help

We are accessible and supportive to colleagues and clients with a friendly approach.

Our People Pledge

Don’t meet every single requirement? Research shows that women and people of color are less likely than others to apply if they feel like they don’t match 100% of the job requirements. Don’t let the confidence gap stand in your way, we’d love to hear from you! We understand that experience comes in many different forms and are dedicated to adding new perspectives to the team.

Kpler is committed to providing a fair, inclusive and diverse work-environment. We believe that different perspectives lead to better ideas, and better ideas allow us to better understand the needs and interests of our diverse, global community. We welcome people of different backgrounds, experiences, abilities and perspectives and are an equal opportunity employer.