Azure Data Engineer

JATO Dynamics Limited; Uxbridge; Hybrid; GBP 65,000 - 90,000; Full time

Job summary

JATO Dynamics Limited is hiring an Azure Data Engineer to join agile teams. The role focuses on designing, building, and operating data pipelines in Azure to deliver data products for internal and external customers.

You will collaborate across Operations, Architecture, Delivery, Product, and Software Engineering to release platform increments. The role requires hands-on experience with Azure Data Factory and Azure Synapse, plus a solid background in ETL/ELT, data warehousing, and data quality

Qualifications

• 2+ years designing and developing complex ETL/ELT pipelines in the cloud.
• Experience with Azure Data Factory and Azure Synapse.
• Experience in Agile development and data warehousing concepts.

Responsibilities

• Ingest, warehouse, and store data in Azure cloud as data assets.
• Build data products from assets to satisfy requirements.
• Collaborate with Operations, Architecture, Delivery, Product, and Software Engineering.
• Contribute to solution-level architecture and design.
• Own releases from development to production following DevOps.
• Improve ADF and Synapse pipelines and data quality.

Skills

ETL/ELT pipelines

Cloud data engineering

Agile development

Data warehousing concepts

SQL

Data analysis

Automation & scripting

Tools

Azure Data Factory

Azure Synapse

PySpark

NoSQL

SQL Server / MySQL

Job description

JATO Dynamics is a global company and the leading provider of automotive market intelligence. With an insight into over 50 overseas markets, we deliver the world's most complete, accurate and up-to-date automotive data and insights, creating significant competitive advantage to our customers.

Find out more about what we do here: JATO

Our vision

Our vision is to be the world’s most exciting leader in automotive business intelligence solutions. We aim to generate excitement through implementing pioneering ideas, problem solving and going beyond our customer expectations. Our JATO colleagues are partners in our future and stakeholders in our desires. Our strategic intent is to help customers create significant competitive advantage by constantly leading in connected data, information and knowledge provision, ultimately improving our customers’ work processes, informed decision-making and business results.

Role Overview

Do you want to be part of building out next generation SaaS products that power the automotive industry?

JATO is currently looking to hire an Azure Data Engineer to join our agile teams.

As a Data Engineer, you will work in a product delivery team using your data analysis, ETL and cloud technology skills to discover, model, provision and store data at scale in the cloud. This data will be then converted into data products based on the commercial requirements and provisioned to internal and external customers

Please note this is a hybrid role and you will be expected to work in our Uxbridge office 2 days per week.

Key Responsibilities

• Ingest, warehouse, and store data at scale in the Azure cloud platform as JATO Data Assets
• Build Data Products from Data Assets and provision to customers to satisfy commercial requirement
• Work closely with colleagues in Operations, Architecture, Delivery, Product, and Software Engineering to collaboratively release platform increments
• Contribute toward solution level architecture and design
• Own your release from development to production following DevOps method
• Contribute towards ADF and Synapse pipeline design improvement and best practices and following design principles
• Understand data issues and able to analyse and work with Architects in addressing design challenge
• Migrate data from legacy databases and storage to Azure cloud-based lake and warehouse

Key Requirements

• Minimum 2+ years’ hands-on experience designing and developing complex ETL/ELT pipelines in the cloud (Azure Data Factory and Azure Synapse)
• Experience of preparing data mappings based on high level requirements and taking those to a solution level details
• Firm understanding of data warehousing concepts in the lake and traditional database with experience of working end-to-end in a ETL lifecycle from data acquisition to the consumption layer
• Knowledge and experience in value-added areas of automation, scripting
• Relational databases (SQL Server, MySQL etc.), NoSQL and writing querie
• Hands-on analytical and data exploration skills
• Experience of working in Agile methodology
• Good knowledge and hands-on experience of developing Stored Procedures
• Experience of working with PySpark Notebooks for data ingestion and provision
• Practical experience of improving data quality and efficiency in data pipelines