Senior Data Engineer

Optimizely; Greater London; On-site; GBP 80,000 - 100,000; Full time

Job summary

A leading technology company in the UK is seeking a Senior Data Engineer to design and develop their Data platform. This includes building optimal data pipelines, managing large datasets, and collaborating with cross-functional teams to deliver actionable insights. Candidates should have 5+ years of experience, expertise in data modeling, ETL, and familiarity with AWS technologies. A Bachelor's or Master's degree in a relevant field is required. The role emphasizes problem-solving and effective communication skills.

Qualifications

• 5+ years of experience as a Data Engineer or similar role.
• Experience with building data pipelines and applications.
• Sound knowledge of distributed systems and data architecture.

Responsibilities

• Create and maintain optimal data pipeline architecture.
• Assemble, analyze, and organize large, complex data sets.
• Identify, design, and implement internal process improvements.
• Build analytics tools that provide actionable business insights.

Skills

Data modeling

ETL pipelines

SQL

Data warehousing

Big data technologies

Problem-solving

Communication

Communication skills

Education

Bachelor's or Master's degree in Computer Science

Tools

AWS Big Data Technologies

Snowflake

DBT

GCP

Job description

Introduction

As Senior Data Engineer, you should be an expert with data warehousing technical components (e.g. Data Modeling, ETL and Reporting), infrastructure (e.g. hardware and software) and their integration. You will be responsible for collecting data from multiple sources and building optimal pipelines to process & leverage the data to meet various business requirements. You will be responsible for the execution of our data strategy through design and development of the Data platform using, but not limited to, AWS technologies, Airflow and Snowflake to deliver Reporting, BI and Analytics solutions. You will be working closely with business and technical stakeholders to aggregate, analyze & transform data to report insights.

Job Responsibilities

• Create and maintain optimal data pipeline architecture
• Assemble, analyze and organize large, complex data sets that meet functional / non-functional business requirements
• Identify, design, and implement internal process improvements : automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using DBT, SQL, Snowflake and AWS / GCP ‘big data’ technologies.
• Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics
• Assist the data science team by preparing data for prescriptive and predictive modelling
• Collaborating with the data architects, analysts and scientists on the team

Knowledge and Experience

• 5+ years of experience as a Data Engineer or in a similar role
• Experience with data modeling, data warehousing, and building ETL pipelines
• Experience in SQL
• Experience with building data pipelines and applications to stream and process datasets
• Sound knowledge of distributed systems and data architecture (lambda) - design and implement batch and stream data processing pipelines, knows how to optimize the distribution, partitioning, and MPP of high-level data structures.
• Knowledge of Engineering and Operational Excellence using standard methodologies.
• Expertise in designing systems and workflows for handling Big data volumes
• Knowledge of data management fundamentals and data storage principles
• Strong problem-solving skills and ability to prioritize conflicting requirements.
• Excellent written and verbal communication skills and ability to succinctly summarize key findings.
• Experience working with AWS Big Data Technologies (EMR, Redshift, S3)

Education

Bachelor\'s or Master\'s degree in Computer Science, Information Systems, or equivalent

Competencies

Driving Continuous Improvement Driving for Results Driving Projects to Completion Interacting with People at Different Levels Using Computers and Technology