Lead Data Engineer

Springer Nature Group; London; Hybrid; GBP 60,000 - 100,000; Full time

Job summary

Join a forward-thinking organization as a Lead Data Engineer, where you'll transform data capabilities and drive innovation. This role involves leading data engineering projects, collaborating with diverse teams, and mentoring junior engineers. You'll work in a supportive environment that values co-working and continuous learning, with opportunities to explore new technologies. With a focus on delivering quality solutions, you'll play a crucial role in shaping the future of data at a leading publisher. This position offers a unique chance to make a significant impact in a dynamic and inclusive culture.

Qualifications

• Several years of experience in Data/Software engineering on a cloud platform.
• High competence in SQL and deep experience of at least one programming language.

Responsibilities

• Lead streams of data engineering work and collaborate with stakeholders.
• Mentor less experienced team members and guide technology choices.

Skills

Data Engineering

Software Engineering

Cloud Platforms

SQL

Python

Java

Data Quality Concepts

Iterative Software Development

Automated Testing

User-Centric Design

Tools

DBT

Job description

We’re looking for a Lead Data Engineer to join SN Data – Data Competence Centre within Springer Nature Operations.

Springer Nature is a leading publisher of scientific books, journals and magazines with over 3000 journal titles and one of the world’s largest corpora of peer-reviewed scientific text data. You would be joining a new programme of work to transform how Springer Nature uses its data: building up data capabilities, creating a data platform and engineering capability (technology, people and process) to create a foundation for the future, adding value to cross-organisation initiatives and kick-starting data-driven innovation.

Across the programme, our teams are cross-functional, diverse and made up of different experience levels. All team members collaborate to deliver the best solutions that satisfy our customers’ needs.

We are committed to growing and nurturing our people for the long term. We attend conferences and make time for people to explore new technology that interests them and is relevant to our work, as well as host data engineering community-of-practice sessions to share knowledge.

We work in a supportive environment. We value face-to-face co-working and require two days per week in the office on average in a month.

The position is based in either London (UK) or Berlin (Germany), with some travel required, typically 3-4 times a year. This role is on a small, autonomous team, and you will be expected to impact what we do and how we work. We like to keep our processes light and bureaucracy slim.

About you:

Role responsibilities

1. Proactively lead streams of data engineering work, collaborating with stakeholders and other non-engineering roles to come up with creative solutions to the problems your team is presented with.
2. Work with both data producers and consumers to optimise existing data products and the data within them to meet evolving business needs.
3. Work collaboratively with other engineers, using techniques like pair and ensemble programming, to foster collective ownership and help upskill, reskill and learn from other team members.
4. Mentor and develop less experienced members of the team, providing them with a positive role model.
5. Help guide technology choices, adopting company-standard tech stacks by default while responsibly investigating alternative tools and services and looking for opportunities to innovate.

Skills & experience:

1. Several years of experience in Data / Software engineering on a cloud platform.
2. An understanding of data and distributed systems concepts.
3. Knowledge of data quality concepts and experience using tools to identify data quality problems and come up with strategies to tackle them.
4. Experience working with iterative software development principles.
5. Automated testing at different levels.
6. Experience owning software systems and/or data pipelines end-to-end, having full responsibility for operating them in production and responding to problems that arise.
7. The ability to work with stakeholders and other non-technical roles to translate business requirements into technical work, and conversely to articulate necessary technical work to the same people for prioritisation.
8. High competence in SQL and deep experience of at least one programming language e.g. Python, Java.
9. Experience with decentralised Data Mesh and Data Product architecture principles.
10. Experience building and testing data pipelines with DBT.
11. Understanding and experience of User-Centric Design principles.

What you will be doing:

1. 1 month

• Actively contributing to the codebase in collaboration with other engineers and deploying changes into production.
• Familiarising yourself with our tech stack and processes.
• Starting to understand the data landscape in and around your team.
• Getting to know the various stakeholders and their general requirements.
• Collaborating effectively with each discipline on the team.
• Participating in technical discussions and sharing ideas.

3 months

• Gaining an understanding of the team’s context within the wider organisation.
• Leading a stream of work, working with both engineers and stakeholders.
• Triaging support queries and diagnosing issues in live data pipelines.
• Setting the technical direction of the work done by the team.
• Holding discussions within the engineering team in order to improve product architecture and code quality.
• Ensuring that data is stored securely and in compliance with GDPR.

6 months

• Switching context between multiple streams of work, providing guidance to both developers and stakeholders.
• Taking part in the recruitment process, helping to assess applicants.
• Fostering a culture of continuous feedback, giving and receiving constructive feedback within your team, and proactively improving ways of working.
• Mentoring others in the principles of data engineering and best practices and looking for opportunities to help other engineers on the team grow.
• Arbitrating disagreements within the team and not avoiding difficult conversations.
• Gauging the complexity and scope of a piece of work, breaking it into smaller pieces when appropriate with a focus on iteratively delivering value to end users.

At Springer Nature, we value the diversity of our teams and work to build an inclusive culture where people are treated fairly and can bring their differences to work and thrive. We empower our colleagues and value their diverse perspectives as we strive to attract, nurture and develop the very best talent.

Time left to apply End Date: May 1, 2025 (14 days left to apply)

Why work here?

We are an ambitious and dynamic organisation, and home to some of the best-known names in research, educational and professional publishing.

Working at the heart of a changing industry, we are always looking for great people who care about delivering quality to our customers and the communities we work alongside. In return, you will find that we open the doors to discovery for all our employees – offering opportunities to learn from some of the best in the business, with a culture that encourages curiosity and empowers people to find solutions and act on their instincts. Whether you are at the beginning of your career or are an experienced professional, we invite you to find out more about the roles we offer and explore our current vacancies.

About Us

We are a global and progressive business, founded on a heritage of trusted and respected brands – including Springer, founded in 1842, Macmillan, founded in 1843 and Nature, first published in 1869.

Nearly two centuries of progress and advancement in science and education have helped shape the business we are today. Research and learning continues to be the cornerstone of progress, and we will continue to open doors to discovery through trusted brands and innovative products and services. Springer Nature Group was created in May 2015 through the combination of Nature Publishing Group, Macmillan Education and Springer Science+Business Media.