Senior Data Platform Engineer: AI-Driven Lakehouse

Sword; Portugal; Presencial; EUR 53 000 - 83 000; Tempo integral

Resumo da oferta

Sword is building a truly AI-native data platform to power healthcare innovation. This Data Platform Engineer role focuses on designing a streaming lakehouse, building robust pipelines, and delivering self-service APIs for data consumers across clinical, operations, AI, and product teams.

You will own durable workflows, shape the API surface, and collaborate with data engineers and analysts on governance and lineage, while ensuring compliance in a regulated domain.

Qualificações

• Proven experience designing and operating data platforms at scale in production.
• Hands-on experience with Iceberg; Delta Lake or Hudi welcome for lakehouse formats.
• Clear mental model of data catalogs (REST, Polaris, Glue, Unity, Hive).
• Exposure to vendor lakehouses (Snowflake, Starburst, Databricks).
• Strong experience with a distributed processing engine (Flink preferred; Spark acceptable).
• Familiarity with durable execution concepts (Temporal, Restate).
• Production experience building and operating REST or gRPC APIs at scale.
• Solid understanding of Kafka and event-driven architectures.
• Comfortable in regulated environments (healthcare/fintech/gov).
• Platform mindset: API-first, self-service for humans, systems, and agents.

Responsabilidades

• Design and evolve Sword's streaming lakehouse as the foundation for data consumers.
• Build and operate distributed streaming pipelines with low latency and high reliability.
• Own durable workflows coordinating complex data movement across systems.
• Shape the platform's API surface for producers and consumers without infra context.
• Drive evaluations and integrations with vendor data platforms within architectural trade-offs.
• Contribute to self-service and agentic layers for humans, systems, and AI agents.
• Partner with data engineers and analysts on contracts, governance, and lineage.
• Build and maintain AI-ready data infrastructure powering ML and AI-driven products.
• Leverage AI coding assistants to accelerate development, documentation, and code quality.
• Work in a regulated environment where audit and governance are integral to design.

Conhecimentos

Data platforms

Lakehouse formats

Data catalogs

Lakehouse platforms

Flink

Spark

APIs (REST/gRPC)

Kafka

Regulated environments

API-first/self-service

Ferramentas

Iceberg

Delta Lake

Hudi

Kafka

Flink

Spark

Descrição da oferta de emprego

Sword is building a truly AI-native data platform to power healthcare innovation. This Data Platform Engineer role focuses on designing a streaming lakehouse, building robust pipelines, and delivering self-service APIs for data consumers across clinical, operations, AI, and product teams.

You will own durable workflows, shape the API surface, and collaborate with data engineers and analysts on governance and lineage, while ensuring compliance in a regulated domain.