Senior Backend Engineer: Real-Time Data Platform

Grid Dynamics; Greater London; On-site; GBP 90,000 - 120,000; Full time

Benefits offered by this job

Bleeding-edge projects

Motivated team

Competitive salary

Flexible schedule

Medical insurance

Sports

Corporate social events

Professional development

Well-equipped office

Job summary

Grid Dynamics is partnering with a leading commodities firm to build a cloud-native data platform for real-time and end-of-day market data. The project replaces an existing system with a scalable solution, ingesting data from numerous providers and exposing API-compatible query capabilities to keep client applications functioning without change.

You'll work with experienced engineers, plan migration, deploy new systems, analyse traffic and expand feeds and APIs for new use-cases while ensuring

Qualifications

• Proficient in Java, Python and C# for building production data services.
• Strong experience with data pipelines, event-driven ingestion and backfill.
• Solid networking fundamentals and cloud data platform experience (AWS).
• Strong testing discipline and ability to operate autonomously within a client team.
• Experience with scalable data storage/analysis technologies and SQL querying.
• Excellent communication and pragmatic decision-making under pressure.

Responsibilities

• Building the feed ingestion subsystem: event-driven connectors pulling data directly from external market data providers.
• Building query APIs using modern scalable data technologies like ClickHouse and S3 Tables/Iceberg with Athena. Experience with other massively scalable data tech is beneficial (QuestDB, Snowflake, DuckDB).
• Building an intercepting proxy to intelligently direct traffic towards new query services. Achieving faithful API compatibility so that clients can continue to operate correctly.
• Gathering and analysing metrics and request payloads to discover and survey existing data clients. Validating the design end-to-end against real client requests, and rolling out new systems in carefully staged steps.
• End-to-end observability across every component: logs, metrics and traces into ClickStack and Grafana, augmented by AWS observability services.
• Working carefully to guide the client, learning through discovery sessions, and thinking carefully about platform architecture (streaming vs request-response use cases, staged and low-risk roll-outs, tracking data licenses, high-frequency feed downsampling).

Skills

Java

Python

C#

Data pipelines

Event-driven ingestion

Cloud platforms AWS

Tools

ClickHouse

Athena

Iceberg

Snowflake

DuckDB

QuestDB

Envoy

HAProxy

Kong

Traefik

Job description

Grid Dynamics is partnering with a leading commodities firm to build a cloud-native data platform for real-time and end-of-day market data. The project replaces an existing system with a scalable solution, ingesting data from numerous providers and exposing API-compatible query capabilities to keep client applications functioning without change.

You'll work with experienced engineers, plan migration, deploy new systems, analyse traffic and expand feeds and APIs for new use-cases while ensuring