Data Engineer

Methods; Bristol; On-site; GBP 50,000 - 70,000; Full time

Job summary

A leading technology consultancy in the UK is seeking a Data Engineer to design and implement modern data solutions that meet business objectives. The ideal candidate will have strong skills in SQL, Python, and cloud technologies, as well as experience with Apache Airflow and data modelling. Key responsibilities include architecting data solutions and collaborating with stakeholders to provide data insights. This role offers competitive benefits and a chance to contribute to impactful projects.

Benefits

25 days of annual leave

Pension Salary Exchange Scheme

Private Medical Insurance

Worldwide Travel Insurance

Discretionary Company Bonus

Wellness 24/7 confidential employee assistance programme

Development access to Pluralsight and LinkedIn Learning

Qualifications

• Experience in SQL Server Integration Services (SSIS).
• Strong proficiency in SQL and Python for handling complex data problems.
• Hands-on experience with Apache Spark (PySpark or Spark SQL).
• Experience with the Azure data stack.
• Knowledge of workflow orchestration tools like Apache Airflow.

Responsibilities

• Design and architect modern data solutions.
• Implement advanced ETL / ELT pipelines using Python, SQL, and Apache Airflow.
• Build scalable data solutions leveraging cloud technologies.
• Make critical technical decisions on data challenges.
• Collaborate with stakeholders on business requirements.

Skills

SQL Server Integration Services (SSIS)

ETL - SSIS, SSRS, T-SQL

SQL

Python

Apache Spark (PySpark or Spark SQL)

Azure data stack

Apache Airflow

Docker

dimensional modelling techniques

CI / CD pipelines

test-driven development

strong communication skills

technical leadership

Tools

Power BI

Apache Superset

Kubernetes

Databricks

Microsoft Fabric

Job description

Responsibilities

• Design and architect modern data solutions that align with business objectives and technical requirements
• Design and implement advanced ETL / ELT pipelines using Python, SQL, and Apache Airflow
• Build highly scalable and performant data solutions leveraging cloud platforms and technologies
• Develop complex data models to handle enterprise-level analytical needs
• Make critical technical decisions on tools, frameworks, and approaches for complex data challenges
• Optimise large-scale data processing systems for performance and cost-efficiency
• Implement robust data quality frameworks and monitoring solutions
• Evaluate new technologies to enhance our data engineering capabilities
• Collaborate with stakeholders to translate business requirements into technical specifications
• Present technical solutions to leadership and non-technical stakeholders
• Contribute to the development of the Methods Analytics Engineering Practice by participating in our internal community of practice

Your Impact

• Enable business leaders to make informed decisions with confidence through timely, accurate data insights
• Drive adoption of modern data architectures and platforms
• Deliver seamless data solutions that enhance user experience
• Elevate the technical capabilities of the entire data engineering team
• Help cultivate a data-driven culture within the organisation
• Establish technical standards and patterns that ensure quality and maintainability

Experience & Qualifications

• Experience in SQL Server Integration Services (SSIS)
• Good experience with ETL - SSIS, SSRS, T-SQL (On-prem / Cloud)
• Strong proficiency in SQL and Python for handling complex data problems
• Hands-on experience with Apache Spark (PySpark or Spark SQL)
• Experience with the Azure data stack
• Knowledge of workflow orchestration tools like Apache Airflow
• Experience with containerisation technologies like Docker
• Proficiency in dimensional modelling techniques
• Experience with CI / CD pipelines for data solutions
• Experience implementing and advocating for test-driven development methodologies in data pipeline workflows, including unit testing, integration testing, and data quality validation frameworks
• Strong communication skills for translating complex technical concepts
• Track record of successful project delivery in a technical leadership capacity
• You may also have some of the desirable skills and experience :

Desirable skills and experience

• Experience designing and implementing data mesh or data fabric architectures
• Knowledge of cost optimisation strategies for cloud data platforms
• Experience with data quality frameworks and implementation
• Understanding of data lineage and metadata management
• Experience with technical project management
• Experience with data visualisation tools like Power BI or Apache Superset
• Experience with other cloud data platforms like AWS, GCP or Oracle
• Experience with modern unified data platforms like Databricks or Microsoft Fabric
• Experience with Kubernetes for container orchestration
• Understanding of streaming technologies (Apache Kafka, event-based architectures)
• Software engineering background with SOLID principles understanding
• Experience with high-performance, large-scale data systems
• Knowledge of recent innovations in AI / ML and GenAI
• Defence or Public Sector experience
• Consultant experience

Clearance

UKSV (United Kingdom Security Vetting) clearance is required for this role, with Security Check (SC) as the minimum standard.

Our Hiring Process

• Internal Application Review
• Initial Phone Screen
• Technical Interview
• Pair Programming Exercise
• Final Interview
• Offer

Benefits

• Autonomy to develop and grow your skills and experience
• Be part of exciting project work that is making a difference in society
• Strong, inspiring, and thought-provoking leadership
• A supportive and collaborative environment
• Development access to Pluralsight and LinkedIn Learning
• Wellness 24 / 7 Confidential employee assistance programme
• Social - office parties, pizza Friday and commitment to charitable causes
• Time off - 25 days of annual leave a year, plus bank holidays, with the option to buy 5 extra days each year
• Volunteering - 2 paid days per year to volunteer in our local communities or within a charity organisation
• Pension Salary Exchange Scheme with 4% employer contribution and 5% employee contribution
• Discretionary Company Bonus based on company and individual performance
• Life Assurance of 4 times base salary
• Private Medical Insurance which is non-contributory (spouse and dependants included)
• Worldwide Travel Insurance which is non-contributory (spouse and dependants included)