Data Engineer

Insight Global; United Kingdom; Remote; GBP 32,000 - 36,000; Full time

Job summary

An established industry player is looking for multiple Data Engineers to join a prestigious energy client in London. In this exciting role, you will design and manage live data streaming pipelines, ensuring seamless data flow for the Energy Trading team. You will also evaluate processes, conduct code reviews, and mentor junior engineers, all while collaborating remotely with teams across the UK and the US. If you have a strong background in Azure Databricks and data warehousing, this is a fantastic opportunity to make a significant impact in a dynamic environment.

Qualifications

• Deep expertise in Azure Databricks and data streaming.
• Experience in data warehousing and SQL proficiency.

Responsibilities

• Design and manage live data streaming pipelines using Azure Databricks.
• Conduct code reviews and provide mentorship to junior engineers.

Skills

Azure Databricks

Apache Spark

PySpark

Qlik Replicate

SQL

Job description

Direct message the job poster from Insight Global

Professional Recruiter at Insight Global UK

Insight Global is seeking multiple Data Engineers to join a prestigious energy client based in London. The successful candidates will be responsible for designing, implementing, and managing live data streaming pipelines for the client’s Energy Trading team.

Key Responsibilities:

• Pipeline Management: Design, implement, and manage live data streaming pipelines using Azure Databricks to ensure seamless data flow and real-time processing.
• Process Evaluation: Assess and optimize on-premise to cloud data exchange processes for accuracy, efficiency, and scalability.
• Code Review and Debugging: Conduct thorough code reviews and debugging sessions, providing guidance and mentorship to junior data engineers to ensure high-quality code and best practices.
• Problem Solving: Develop innovative solutions to address computing and cost challenges, leveraging advanced technologies and methodologies.
• Remote Collaboration: Work fully remotely within the UK, maintaining effective communication and collaboration with US teams during overlapping working hours.

Must Haves:

• Deep expertise with Azure Databricks (DLT, Data Streaming , Unity Catalogue etc.).
• Expert with Apache Spark and PySpark (ability to review quality of code and debug issues).
• Experience with Qlik Replicate to move data from on-prem to the cloud.
• Background in Data warehousing (SAP Hana, BI/BW, Oracle etc.).
• Proficient with SQL.

Seniority level

• Seniority level

  Mid-Senior level

Employment type

• Employment type

  Contract

Job function

• Job function

  Information Technology, Design, and Engineering

• Industries

  Oil and Gas and Oil, Gas, and Mining

Referrals increase your chances of interviewing at Insight Global by 2x

Get notified about new Data Engineer jobs in United Kingdom.

Edinburgh, Scotland, United Kingdom 1 month ago

London, England, United Kingdom 6 days ago

Edinburgh, Scotland, United Kingdom 5 months ago

London, England, United Kingdom 2 months ago

London, England, United Kingdom 1 month ago

United Kingdom
$32,400.00
-
$36,000.00
3 months ago

Manchester, England, United Kingdom 2 months ago

London, England, United Kingdom 1 month ago

Glasgow, Scotland, United Kingdom 2 months ago

Manchester, England, United Kingdom 2 weeks ago

Edinburgh, Scotland, United Kingdom 2 months ago

We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.