Remote Data Platform Engineer - Real-Time Analytics

Block Labs; Italia; In loco; EUR 90.000 - 130.000; Tempo pieno

Descrizione del lavoro

Block Labs is seeking a Data Platform Engineer to own the end-to-end real-time data pipeline and build a scalable, multi-tenant data platform that powers real-time decisioning and predictive modelling across products.

You will design and operate platform services, focusing on reliability, cost efficiency, and production-grade pipelines, collaborating with data scientists, AI engineers, BI developers, and infrastructure teams.

Competenze

• 3+ years building and operating production data pipelines at scale.
• Experience with streaming and batch data processing (Kafka / MSK, Spark).
• Strong SQL and warehouse engineering with ClickHouse or equivalents.
• Proficient in PySpark / Spark Streaming.
• Experience with CI/CD and IaC (CloudFormation, Terraform, CDK).
• Data modeling for multi-tenant schemas and data contracts.
• Observability: data quality checks and monitoring.

Mansioni

• Design, build, and maintain scalable data pipelines using AWS Glue (PySpark), or equivalent orchestration and transformation tools.
• Engineer and optimise the ClickHouse warehouse for sub-second query performance across all back-offices.
• Implement data contracts between back-office and the platform. Onboarding a new operator is a config change, not new tables, topics, or feature views.
• Build the feature-serving layer providing pre-computed features to AI agents at millisecond latency.
• Integrate with third-party databases, back-office APIs, and external systems (CRM, affiliates, acquisition platforms).
• Establish monitoring, alerting, and maintenance procedures including pipeline health checks, freshness monitoring, anomaly detection, and data contract SLA enforcement.
• Own CI/CD and infrastructure-as-code for data workloads.
• Collaborate with data scientists, agent engineers, BI developers, and infrastructure teams to translate data requirements into reliable, production-grade pipelines.

Conoscenze

Apache Kafka / MSK

SQL & Data Warehousing

PySpark / Spark Streaming

CI/CD & IaC

Data Modeling

Strumenti

AWS Glue

Airflow

Docker / Kubernetes

Descrizione del lavoro

Block Labs is seeking a Data Platform Engineer to own the end-to-end real-time data pipeline and build a scalable, multi-tenant data platform that powers real-time decisioning and predictive modelling across products.

You will design and operate platform services, focusing on reliability, cost efficiency, and production-grade pipelines, collaborating with data scientists, AI engineers, BI developers, and infrastructure teams.