Data Engineer

Job Purpose:

To develop, enhance and maintain the data pipelines and data infrastructure across NWEH's software products. To provide expertise and guidance for all data related work across the business.

Key accountabilities:

1. The development, performance, management, and troubleshooting of the ETL processes, data pipelines and data infrastructure supporting NWEH's software and service products
2. The effective and reliable operation of NWEH's data pipelines and ETL processes
3. Appropriate adoption of new tools, technologies and practices to ensure the team stay up to date and follow industry best practice
4. Supporting data work across the team and wider business

1. Designing, developing, maintaining, and optimising data pipelines, ETL processes, and databases
2. Driving continuous improvement by refining processes, products and identifying new tools, standards, and practices
3. Working with teams across the business to define solutions, requirements, and testing approaches
4. Assisting with process definition, always ensuring compliance with organisation processes and regulatory standards
5. Ensuring compliance with regulatory requirements and standards and audit readiness
6. Automation and ongoing monitoring of data and data processes, ensuring data quality and integrity
7. Share knowledge and provide guidance on databases and data
8. Maintain up to date, accurate and concise documentation of database configuration and processes
9. Work across the team to deliver best practice infrastructure and infrastructure deployment and management processes

1. A good degree in a relevant subject or equivalent professional experience in a data role
2. At least 3 years' professional experience developing data pipelines and ETLs using Microsoft products; at least 1 year working with cloud native technologies like Azure Data Factory
3. Demonstrable experience of delivering technical work within time and budget constraints
4. Good understanding of data security best practice
5. Experience of supporting ETLs or data pipelines crucial to a production system
6. Experience of working in a cross-functional team to deliver technical solutions

1. Experience with SQL Server, SSIS, Azure Data Factory and Azure SQL
2. Experience with Cloud and infrastructure as Code, particularly in an Azure setting using Bicep
3. Understanding of DevOps practices and the associated benefits
4. Skill in database testing; unit, performance, stress, security
5. Experience of working in an agile team
6. Experience of working in a highly regulated industry and with highly sensitive data
7. Exposure to large data solutions like Snowflake, Trino, Synapse, Azure Data Lake, and Databricks
8. Experience of data science using R, Stata or Python
9. Familiarity with Atlassian tools; JIRA, Confluence, BitBucket
10. Understanding of clinical trials, GCP and GxP

1. Collaboration - teamwork, listening and communication
2. Professionalism and commitment to delivery and improvement
3. Curiosity and continuous learning
4. Attention to detail but solution oriented
5. Adaptability