Data Engineer, Data Cloud

The BDPN; Greater London; Hybrid; GBP 68,000 - 113,000; Full time

Job summary

Chainalysis is seeking a Data Platform Engineer to design, build, and optimize scalable data platforms with a focus on streaming data, cloud infrastructure, and automation. The role involves creating CI/CD pipelines, integrating data clouds with databases, and enforcing observability and governance across data pipelines.

The ideal candidate has 3+ years in data platform or infrastructure roles, strong AWS and Terraform skills, and experience with Kafka, Flink, Spark, and related lakehouse

Qualifications

• 3+ years of experience as a Data Platform Engineer or Software Engineer or Infrastructure Engineer, with hands-on expertise in building and maintaining cloud-based data platforms.
• Solid experience with AWS services, a good understanding of cloud architecture, and proficiency with Terraform for provisioning and managing cloud infrastructure
• Experience building streaming data pipelines using Apache Flink, Apache Kafka, Spark Structured Streaming, or similar technologies
• Knowledge of modern data lakehouse architectures and ecosystem such as Kafka, Flink, Spark, Databricks, Snowflake, DBT, Airflow, Debezium, Delta/Iceberg, and comfort with Python/Java and SQL
• Experience with networking and security concepts within AWS, including VPCs, subnets, routing, security groups, IAM, etc.

Responsibilities

• Design, develop, and optimize high-performance, scalable data platforms with a strong focus on streaming data processing, large data volumes, and cloud infrastructure
• Implement and maintain infrastructure automation and self-service using Terraform, ensuring reliable environments for development and production
• Build seamless integrations between Data Cloud and various relational and noSQL OLTP databases
• Design and build frameworks and abstractions to accelerate development of data pipelines while embedding data engineering best practices
• Establish and maintain observability, security, and data governance solutions to ensure high quality and reliability of data pipelines

Skills

AWS

Terraform

Streaming data

Python/Java

SQL

Tools

Apache Kafka

Apache Flink

Spark

Databricks

DBT

Job description

The engineering team at Chainalysis is inspired by solving the hardest technical challenges and creating products that build trust in cryptocurrencies. We’re a global organization with teams in the UK, Denmark, Canada, and the USA who thrive on the challenging work we do and doing it with other exceptionally talented teammates. Our industry changes every day and our job is to build a flexible platform that will allow us to adapt to those rapid changes.

We’re looking for Data Engineers to join our Data Cloud team. This group accelerates innovation & helps unlock new business lines, by empowering anyone at Chainalysis to quickly and reliably discover, access, analyze, and build on top of any and all data. You’ll be a key player in creating and optimizing our data platform and data processing systems. If you’re passionate about infrastructure automation, cloud services, and building scalable platforms to enable next-gen real-time streaming data applications, we want you to join our talented and growing team!

In this role, you’ll:

Design, develop, and optimize high-performance, scalable data platforms, with a strong focus on streaming data processing, large data volumes, and cloud infrastructure

Implement and maintain infrastructure automation and self-service using Terraform, ensuring reliable and repeatable environments for both development and production, and creating robust CI/CD pipelines

Build seamless integrations between Data Cloud and various relational and noSQL OLTP databases

Design and build frameworks and abstractions to accelerate the development of data pipelines while embedding data engineering best practices

Establish and maintain observability, security, and data governance solutions to ensure high quality and reliability of data pipelines

We’re looking for candidates who have:

3+ years of experience as a Data Platform Engineer or Software Engineer or Infrastructure Engineer, with hands-on expertise in building and maintaining cloud-based data platforms

Solid experience with AWS services, a good understanding of cloud architecture, and proficiency with Terraform for provisioning and managing cloud infrastructure

Experience building streaming data pipelines using Apache Flink, Apache Kafka, Spark Structured Streaming, or similar technologies

Knowledge of modern data lakehouse architectures and ecosystem such as Kafka, Flink, Spark, Databricks, Snowflake, DBT, Airflow, Debezium, Delta/Iceberg, and comfort with Python/Java and SQL

Experience with networking and security concepts within AWS, including VPCs, subnets, routing, security groups, IAM, etc.

Nice to have Experience:

Exposure to or interest in the cryptocurrency technology ecosystem

Experience working with different blockchain technologies is a plus

Experience with a SaaS or FinTech company

Technologies we use:

AWS Services: MSK, EC2, VPC, IAM, S3, SQS, Managed Flink, EKS

Data Lakehouse: Kafka, Flink, Spark, Databricks, DBT, Debezium