Senior Data Engineer

N Consulting Limited; Bath; Hybrid; GBP 55,000 - 75,000; Full time

Job summary

N Consulting Limited is looking for a Senior Data Engineer to work in Bath on a hybrid basis. This contract role lasts for 3 months, requiring expertise in AWS Cloud architecture and data services, along with skills in ETL development and Data Pipeline automation.

The ideal candidate will engage with both business and technical stakeholders, ensuring effective communication across varying levels of seniority. Experience with AWS Glue, Lambda, and Python is essential for success in this position.

Qualifications

• In-depth skills on AWS Cloud architecture.
• Clear communication with stakeholders at various seniority levels.
• Knowledge of data modeling techniques, including dimensional models.

Responsibilities

• Design, build, test, and performance tune ETL processes.
• Engage both business and technical stakeholders.

Skills

AWS Glue ETL

Data Pipeline automation

ETL development

AWS Cloud architecture

AWS data services

AWS Lambda

Python

Release management (CI/CD)

Athena SQL

Spark SQL

Tools

AWS Glue Data Quality

AWS Lake Formation

Azure DevOps

AWS CodeCommit

AWS CodePipeline

AWS Step functions

AWS CLI

Power BI

Job description

# Senior Data Engineer at N Consulting LtdWork mode: Hybrid Number of days a week worked in the office: 3 Contract duration: 3 Months Location: Bath --------------------------------------- JOB DETAILS Role Title: Senior Data Engineer The Senior Data Engineer must be an experience data professional with in-depth skills on AWS Cloud architecture, AWS data services, Data Pipeline automation (with AI integrations desirable) and ETL development (design, build, test and performance tune). Must have knowledge on data modelling techniques, including dimensional models and conformed datasets. As the role is to engage both business and technical stakeholders, the individual must have clear and adequate communication with different stakeholders at the various levels of seniority. Knowledge and Experience - essential - AWS Glue ETL - AWS Glue Data Quality - AWS Lambda - AWS Lake Formation - Medallion Architecture based ETL processes using S3, Iceberg and Athena - Azure DevOps/AWS CodeCommit/AWS CodePipeline - AWS Step functions - Athena SQL ; Spark SQL ; Python - AWS CLI - Release management (CI/CD) - Knowledge and Experience - desirable - AWS Databrew - AWS RDS/Redshift - AWS Macie - AWS CloudWatch - AWS Quick - AWS Sagemaker (Unified Studio and AI) - Power BI