Data Engineer

SmartestEnergy Limited; Greater London; Hybrid; GBP 42,000 - 66,000; Full time

Benefits offered by this job

Hybrid work model

Flexible remote work

Job summary

SmartestEnergy Limited is looking for a data engineer to support the development and maintenance of enterprise data solutions, including the Data Lakehouse, Data Warehouse and Medallion Architecture. You will work with Azure Data Platform tools, Fabric, and Power BI to deliver scalable data pipelines in a mentorship-driven environment.

The role is hands-on, suitable for early-career professionals, with exposure to Azure Data Factory, Synapse pipelines, and CI/CD practices.

Qualifications

• Experience delivering sprints to stakeholders
• Data pipelines with reliable ingestion and validation
• Familiarity with Fabric AI and automation features
• Ability to implement unit tests for pipeline logic
• Understand data lineage and cataloguing concepts

Responsibilities

• Data Integration Support: build and maintain ingestion pipelines from multiple sources
• ETL Development: reusable pipelines across Medallion layers focusing on reliability
• Notebook Automation Support: metadata-driven notebooks in Fabric with Spark
• Data Quality and Testing: implement validation rules and lineage checks
• Documentation and Data Catalogue Support: document flows and assist catalog repos
• BI & Reporting Integration: connect data models to Power BI and semantic layers
• Environment & DevOps Support: contribute to CI/CD using Azure DevOps and Git
• Learning and Development: participate in code reviews and design sessions
• Testing and Data Quality Assurance: develop unit tests and improve coverage

Skills

Data integration

ETL development

Data quality

Notebook automation

BI integration

Azure DevOps

Tools

Microsoft Fabric

Azure Data Factory

Azure Synapse

Power BI

Git

Azure DevOps

Spark

Job description

The purpose of this role is to support the development and maintenance of enterprise data solutions including the Data Lakehouse, Data Warehouse, and Medallion Architecture. This role will work alongside senior engineers and architects to build, test, and deploy scalable data pipelines, with a focus on tasks such as ingestion, data validation, and structured transformations.

The role is a hands‑on opportunity for someone early in their data engineering career to gain deep experience with Microsoft’s Azure Data Platform and Microsoft Fabric, working on real‑world solutions under guidance and mentorship.

How will I spend my time in this role?

• Data Integration Support Assist with building and maintaining data ingestion pipelines from various sources using tools like Microsoft Fabric, Azure Data Factory and Synapse pipelines.
• ETL Development Develop and maintain reusable ETL processes, supporting the all layers of the Medallion Architecture, with a focus on reliability, reusability, and simplicity.
• Notebook Automation Support Help implement and schedule metadata-driven notebooks in Fabric working with Spark under the supervision of senior engineers.
• Data Quality and Testing Assist in implementing and monitoring data quality checks, validation rules, and basic lineage tracking.
• Documentation and Data Catalogue Support Document data flows, transformations, and support updates to data cataloguing tools or metadata repositories.
• BI & Reporting Integration Assist with connecting data models to Power BI, and help develop certified datasets and semantic layers for self-service users.
• Environment & DevOps Support Learn and contribute to CI/CD processes using Azure DevOps and Git. Support code reviews and testing cycles.
• Learning and Development Be an active participant in code reviews, design sessions, and platform learning. Take ownership of tasks and work through feedback constructively.
• Testing and Data Quality Assurance Develop and maintain unit tests to validate pipeline logic and data transformations. Collaborate with engineering peers to continuously refine test coverage and uphold data integrity standards.

What skills/experience do I need to be successful?

• Successful delivery of sprints to the satisfaction of stakeholders.
• Data Pipeline Reliability-Pipelines run with minimal failures and auto-recovery
• Data Latency - Freshness meets defined SLAs for Bronze, Silver, Gold layers
• Valuable insight into the business is delivered through the implementation of new solutions.
• Innovation -Regular adoption of new Fabric AI and automation features
• Implement Robust solutions with minimal time spent fixing bugs.

What sets us apart?

Global Impact: With offices in the UK, US, and Australia, and plans for further expansion, you'll be part of a dynamic, globally-minded team, with opportunities to explore new markets and make a difference on a global scale.

Flexible Working: Embrace the freedom to work from anywhere in the world for up to 30 days a year. We prioritize work-life balance, recognizing that your well-being matters. Find out more here.

Commitment to Diversity and Inclusion: We celebrate our diverse culture and value individuals irrespective of background, disability, religion, gender identity, sexuality, or ethnicity. Join a team where diversity is not just welcomed but celebrated as a key driver of growth and innovation.

What does hybrid working mean to us?

Hybrid working typically means 2 days in the office location listed on this advert and 3 days working at home each week. Some occasional travel to our other offices may be required.

What happens next?

Once we receive your application, it will be reviewed by a human – no bots here! The average process typically takes around 2-3 weeks, with 2 stages of video interviews using Teams. However, this can vary depending on the role. We may invite you for a face-to-face meeting or require only 1 video interview. If you have any questions or need support, our Recruitment Team is here to assist you.

Ready to join us on our journey to digitise, decarbonise, and localize the future of energy?

We're committed to making the application process easy and comfortable. Let us know how we can help you with any reasonable adjustments that can be tailored to your needs. At the bottom of each of our adverts you can find one of our recruitment teams' contact details. Please reach out so we can discuss with you further.

If you're a generator or a commercial and industial (C&I) business, you're in the right place.

If you're a micro or smaller business you should visit our SmartestEnergy Business website.