Senior Data Engineer

Nigel Wright Recruitment; Remote; GBP 70,000 - 90,000; Full time

Job summary

A prominent recruitment agency is seeking a Senior Data Engineer to join a leading organization in the UK. This role, primarily remote, involves designing and maintaining a data platform crucial for business growth. Candidates should have robust experience in data warehousing and governance, proficiency in Microsoft Stack technologies, and strong communication skills. The ideal applicant will lead efforts in integrating various data sources while ensuring data quality and governance across systems. Occasional travel to headquarters in London is expected.

Qualifications

• Strong experience in data engineering, with a focus on data warehousing and lake technologies.
• Track record of data governance, security, quality, and lineage.
• Ability to engage and educate senior stakeholders on data.

Responsibilities

• Develop and maintain the data warehouse in Fabric.
• Integrate data from multiple sources, ensuring quality and governance.
• Monitor, troubleshoot and optimize data pipelines for performance.
• Liaise with business and IT stakeholders on technical requirements.

Skills

Data warehousing

Data lake technologies

Data governance best practices

Microsoft Stack (Fabric, Azure, Power BI)

ETL processes and tools (DBT)

Python for data manipulation and automation

Strong communication skills

Job description

The Company

Nigel Wright are delighted to be partnering with the Institute of Directors in their search for a Senior Data Engineer.

The Institute of Directors (IoD) is a thriving membership community for Directors in the UK and beyond, where you can connect with other leaders, develop your skills and be heard. The IoD offers professional development, valuable connections and influence. With over 20,000 members across the UK and globally, the IoD has exciting plans for growth and the success of their strategy is underpinned by technology. Working in a newly created role, the Senior Data Engineer will be instrumental in designing, developing and maintaining their next generation data platform, ensuring business leaders have timely and accurate information to drive growth and efficiency.

You will be employed on a remote contract with occasional travel to their HQ in London. Unfortunately the IoD is unable to sponsor visas so candidates must have a permanent right to work in the UK.

The Role

Acting as a Technical Lead for data engineering within a small IT team you will :

• Develop and maintain the data warehouse (currently in Fabric)
• Integrate data from multiple sources (internal systems, external datasets)
• Ensure data quality, cleansing and governance processes are implemented
• Monitor, troubleshoot and optimise data pipeline and data warehouse performance to ensure high availability and efficiency
• Liaise closely with business and IT stakeholders, including directors and senior leadership
• Assist in defining technical requirements for new systems and integrations

The Requirements

You will have strong experience in data engineering, with a particular focus on data warehousing and data lake technologies. Additionally you will have a track record of establishing data governance best practises, including data security, data quality and data lineage. Other requirements include :

• Comfortable working in Microsoft Stack (Fabric, Azure, Power BI)
• Knowledge of ETL processes, semantic layers and tools like DBT
• Proficiency in Python for data manipulation and automation
• Ability to challenge assumptions and educate senior staff on data possibilities
• Strong communication skills to engage with non-technical stakeholders