Senior Data Engineer (UK)

YLD; United Kingdom; Remote; GBP 50,000 - 80,000; Full time

Job summary

A leading software engineering consultancy is seeking a Data Engineer to build core infrastructure software and data pipelines. The role involves using technologies like Python, Kafka, and Spark while mentoring other engineers. The position is remote, allowing for a flexible work environment.

Benefits

Company Private Health care

Enhanced maternity and paternity leave

Pension Scheme

25 days annual holiday

£2000 annual allowance for Training/Conferences

£300 annual allowance for additional hardware

Mental Health support

Meditation/Sleep support

Bonus based on performance

Company laptop

Generous referral schemes

Qualifications

• Proven experience writing highly maintainable, performant Python and JavaScript code.
• Experience building modern data pipelines using technologies such as dbt, Kafka, and Spark.
• Understanding of Data Modelling patterns.
• Deep knowledge of complex SQL with performance emphasis.
• Experience with end-to-end monitoring & alerting.

Responsibilities

• Responsible for building core infrastructure software (pipelines, APIs, data modelling).
• Instrument systems for performance and enhancement.
• Coach & mentor other engineers.

Skills

Python

JavaScript

dbt

Kafka

Spark

AWS Kinesis

AWS Lambda

Apache Airflow

SQL

CloudWatch

Datadog

Job description

Location

UK (Remote)

Department

Engineering

Hours

Full-time

About YLD

Everything we do is to empower our clients to move forward. Our purpose is to help our clients develop the capabilities they need to outperform their competitors. Our success is a consequence of ensuring our clients are successful. Our company is built on extraordinary people; we aim to attract, inspire, develop and retain them. YLD is a software engineering and design consultancy; we create digital capabilities for our clients that last beyond our engagement. We have offices in London, Lisbon, Porto.

About the role

As a Data Engineer in this role, you will be responsible for building core infrastructure software (pipelines, APIs, data modelling) as part of our client\'s data platform team. Your work will include instrumenting systems for performance, and enhancement throughout. You will work on ensuring these data offerings are to various internal & external stakeholders using secure authentication patterns.

Your role will include choosing and implementing the appropriate technologies for scaling data access patterns, batch processing, and data streaming for soft real-time consumption while considering the unique domain knowledge of the client\'s business. As a senior collaborator on the team, you will coach & mentor other engineers to support the growth of their technical expertise.

Technical competencies

• Proven experience writing highly maintainable, performant Python and JavaScript code;
• Experience building modern data pipelines using dbt, Kafka, Spark, AWS Kinesis, AWS Lambda, and Apache Airflow (or similar);
• Understanding of Data Modelling patterns;
• Deep knowledge of complex SQL, with emphasis on Common Table Expressions, window functions, and their performance;
• Experience with end-to-end monitoring & alerting experience (CloudWatch, Datadog, etc.).

Non-Technical competencies

• Problem-solving skills that balance innovation with pragmatic technology choices to solve business needs;
• Comfortable working in a dynamic production environment and taking care of client expectations effectively;
• Distinct customer focus and quality mindset;
• Experience working closely with engineering leadership and architects to deliver high-quality solutions;
• Experience maintaining a high degree of ownership and transparency in deliverables;
• An exemplar of YLD\'s brand and safe-guarder of our reputation;
• Exceptional communication skills, able to communicate complex ideas simply.

Benefits

• As well as the chance to work with a forward-thinking, ethical, and fun group of people, you\'ll also get:
• Company Private Health care (currently provided by BUPA)
• Enhanced fully paid maternity and paternity leave for up to 6 months
• Company\'s Pension Scheme
• 25 days annual holiday (excluding Public Holidays)
• £2000 annual allowance for Training/Conferences
• £300 annual allowance for additional hardware
• Mental Health support: access to Spill app
• Meditation/Sleep support: Headspace or Calm app subscription
• Bonus (depending on Company performance and results)
• Company laptop
• Generous referral schemes

Recruitment process

• 1st Interview with someone from the Talent team (30/45 mins)
• Technical Interview with our Senior Developers (1h30)

Our values

• Growing every day
• Including everyone
• Relationships built on honesty and ethics
• Inspiring solutions
• Winning together

We\'re an equal-opportunity employer and value diversity of all kinds. We don\'t discriminate based on race, religion, colour, national origin, gender, sexual orientation, pregnancy or maternity, age, marital status, or disability status. Likewise, we also offer a remote-first working environment, meaning that flexible working and work-life balance come as standard for all employees.