Data Engineer 12 month FTC

SmartestEnergy; Greater London; Hybrid; GBP 70,000 - 110,000; Full time

Job summary

SmartestEnergy in London is seeking a Data Engineer to build cloud-based data solutions on Microsoft's Azure Data Platform and Fabric, developing scalable pipelines and supporting Data Lakehouse and Data Warehouse solutions.

You'll work under experienced engineers to implement modern data architectures, CI/CD practices, and data governance, with hybrid working and opportunities to influence global data initiatives.

Qualifications

• Experience in a Data Engineering, Data Analytics, or Software Development role with ETL/ELT concepts.
• Strong PySpark for scalable data pipelines and SQL (T-SQL or Spark SQL) and Python knowledge.
• Hands-on experience with Microsoft Fabric and/or Azure Data Platform technologies including Data Factory and Synapse.
• Understanding of modern data engineering practices (Medallion Architecture, CI/CD, Azure DevOps, Agile/Scrum).
• Knowledge of data warehousing, semantic models, data quality, security, and governance, plus willingness to learn Fabric features.

Responsibilities

• Build enterprise-scale data pipelines and support Data Lakehouse/Data Warehouse solutions.
• Work with modern data architectures under guidance from senior engineers and architects.
• Collaborate to apply CI/CD, testing, and governance in data projects.

Skills

ETL/ELT concepts

PySpark

SQL

Python

Azure Data Platform

CI/CD

Agile/Scrum

Data governance

Data quality

Tools

Data Factory

Synapse

Lakehouse

Data Pipelines

Dataflows Gen2

Notebooks

Power BI

Job description

Join our growing team as a Data Engineer and play a key role in building modern, cloud-based data solutions that enable better business decision-making. This is an excellent opportunity for someone experienced and who is passionate about data, eager to learn, and excited to work with Microsoft's Azure Data Platform and Microsoft Fabric. You'll work on developing enterprise-scale data pipelines, supporting Data Lakehouse and Data Warehouse solutions, and working with modern data architectures under the guidance of experienced engineers and architects.

What skills/experience do I need to be successful?

• Experienced in a Data Engineering, Data Analytics, or Software Development role, with a good understanding of ETL/ELT concepts and data integration principles;
• Very strong PySpark for developing scalable data pipelines and data transformations and SQL skills (T-SQL and/or Spark SQL) and working knowledge of Python;
• Hands-on experience with Microsoft Fabric and/or Azure Data Platform technologies, including Data Factory, Synapse, Lakehouse, Data Pipelines, Dataflows Gen2, Notebooks, and Power BI integration;
• Understanding of modern data engineering practices, including Medallion Architecture (Bronze/Silver/Gold), CI/CD, Azure DevOps, Agile/Scrum methodologies, testing, and clean code principles;
• Knowledge of enterprise data warehousing, semantic models, data quality, security, and governance, with a willingness to learn and apply emerging Microsoft Fabric capabilities such as Real-Time Intelligence.

What sets us apart?

• Global Impact: With offices in the UK, US, and Australia, and plans for further expansion, you'll be part of a dynamic, globally-minded team, with opportunities to explore new markets and make a difference on a global scale.
• Flexible Working: Embrace the freedom to work from anywhere in the world for up to 30 days a year. We prioritize work-life balance, recognizing that your well-being matters.
• Commitment to Diversity and Inclusion: We celebrate our diverse culture and value individuals irrespective of background, disability, religion, gender identity, sexuality, or ethnicity. Join a team where diversity is not just welcomed but celebrated as a key driver of growth and innovation.

What does hybrid working mean to us?

Hybrid working typically means 2 days in the office location listed on this advert and 3 days working at home each week. Some occasional travel to our other offices may be required.

What happens next?

Once we receive your application, it will be reviewed by a human - no bots here! The average process typically takes around 2-3 weeks, with 2 stages of video interviews using Teams. However, this can vary depending on the role. We may invite you for a face-to-face meeting or require only 1 video interview. If you have any questions or need support, our Recruitment Team is here to assist you.

Ready to join us on our journey to digitise, decarbonise, and localize the future of energy?

We're committed to making the application process easy and comfortable. Let us know how we can help you with any reasonable adjustments that can be tailored to your needs.

At the bottom of each of our adverts you can find one of our recruitment teams' contact details. Please reach out so we can discuss with you further.