data engineer in data platforms

Enfint; Greater London; On-site; GBP 110,000 - 160,000; Full time

Job summary

Simple Machines in the United Kingdom seeks an experienced data architecture lead to own end-to-end cloud-native data platform architectures. You will design scalable data ecosystems using data mesh, data products, and data contracts, and make decisions across ingestion, storage, processing, and access layers.

You will ensure platforms are secure, compliant, and production-grade by design, delivering using Databricks, Snowflake, AWS, and GCP.

Qualifications

• Strong Python and SQL skills.
• Deep experience with Spark and modern data platforms such as Databricks and Snowflake.
• Solid understanding of cloud data services in AWS or GCP.
• Demonstrated ownership of large-scale data platform architectures.
• Strong data modelling and architectural decision-making skills.
• Ability to balance performance, cost, and complexity trade-offs.
• Experience building and operating large-scale data pipelines in production.
• Experience with multiple storage technologies and formats.
• Infrastructure-as-code experience with Terraform or Pulumi.
• Experience with CI/CD pipelines using tools such as GitHub Actions or ArgoCD.
• Experience with data testing and quality frameworks such as dbt, Great Expectations, or Soda.
• Experience in consulting or professional services environments.
• Strong consulting instincts and ability to challenge assumptions and guide clients toward better outcomes.
• Ability to mentor senior engineers and influence technical culture.

Responsibilities

• Own the end-to-end architecture of modern, cloud-native data platforms.
• Design scalable data ecosystems using data mesh, data products, and data contracts.
• Make architectural decisions across ingestion, storage, processing, and access layers.
• Ensure platforms are secure, compliant, and production-grade by design.
• Design and deliver cloud-native data platforms using Databricks, Snowflake, AWS, and GCP.
• Integrate with client systems to enable scalable, consumer-oriented data access.
• Build and optimise batch and real-time pipelines.
• Work with streaming and event-driven technologies such as Kafka, Flink, Kinesis, and Pub/Sub.
• Orchestrate workflows using Airflow, Dataflow, and Glue.
• Process and transform large datasets using Spark and Flink.
• Design production-ready systems.
• Work across relational, NoSQL, and analytical data stores.
• Optimise storage formats and access patterns.
• Implement secure, compliant data solutions with security by design.
• Embed governance while maintaining developer velocity.
• Work directly with clients to understand problems and shape solutions.
• Translate business needs into pragmatic engineering decisions.
• Act as a trusted technical advisor.
• Set engineering standards, patterns, and best practices across teams.
• Review designs and code, providing technical direction and mentorship.
• Improve data quality, testing, observability, and operational excellence.

Skills

Python

SQL

Spark

Databricks

Snowflake

AWS

GCP

Data modeling

Data governance

Consulting

Tools

Terraform

Pulumi

GitHub Actions

ArgoCD

dbt

Great Expectations

Soda

Airflow

Kafka

Kinesis

Pub/Sub

Glue

Job description

Описание: Simple Machines is a global, independent technology consultancy that designs and builds modern data platforms, intelligent systems, and bespoke software across data engineering, software engineering, and AI. It helps enterprises, scale-ups, and government turn complex data into products, platforms, and actionable decisions.

Задачи

• Own the end-to-end architecture of modern, cloud-native data platforms
• Design scalable data ecosystems using data mesh, data products, and data contracts
• Make architectural decisions across ingestion, storage, processing, and access layers
• Ensure platforms are secure, compliant, and production-grade by design
• Design and deliver cloud-native data platforms using Databricks, Snowflake, AWS, and GCP
• Integrate with client systems to enable scalable, consumer-oriented data access
• Build and optimise batch and real-time pipelines
• Work with streaming and event-driven technologies such as Kafka, Flink, Kinesis, and Pub/Sub
• Orchestrate workflows using Airflow, Dataflow, and Glue
• Process and transform large datasets using Spark and Flink
• Design production-ready systems
• Work across relational, NoSQL, and analytical data stores
• Optimise storage formats and access patterns
• Implement secure, compliant data solutions with security by design
• Embed governance while maintaining developer velocity
• Work directly with clients to understand problems and shape solutions
• Translate business needs into pragmatic engineering decisions
• Act as a trusted technical advisor
• Set engineering standards, patterns, and best practices across teams
• Review designs and code, providing technical direction and mentorship
• Improve data quality, testing, observability, and operational excellence

Требования

• Strong Python and SQL skills
• Deep experience with Spark and modern data platforms such as Databricks and Snowflake
• Solid understanding of cloud data services in AWS or GCP
• Demonstrated ownership of large-scale data platform architectures
• Strong data modelling and architectural decision-making skills
• Ability to balance performance, cost, and complexity trade-offs
• Experience building and operating large-scale data pipelines in production
• Experience with multiple storage technologies and formats
• Infrastructure-as-code experience with Terraform or Pulumi
• Experience with CI/CD pipelines using tools such as GitHub Actions or ArgoCD
• Experience with data testing and quality frameworks such as dbt, Great Expectations, or Soda
• Experience in consulting or professional services environments
• Strong consulting instincts and ability to challenge assumptions and guide clients toward better outcomes
• Ability to mentor senior engineers and influence technical culture

Условия

No conditions specified