Principal Data Engineer - Contract (Manchester)

Insight Investment; Manchester; On-site; GBP 90,000 - 120,000; Full time

Job summary

Insight Investment is seeking an experienced Principal Data Engineer to drive the design, development, and governance of a scalable data platform. You will work within the Data Platform team, leveraging Snowflake and related tooling to deliver strategic data solutions across the business.

You will collaborate with multiple business areas, ensure alignment with privacy and security standards, and promote adoption of advanced analytics, AI, and cloud-native patterns.

Qualifications

• Snowflake-based data platform design and governance experience.

Responsibilities

• Contribute to the design and implementation of a scalable data platform using Snowflake and tooling.
• Help evolve the Insight Data Strategy and communicate outcomes from leadership to teams.
• Set firm-wide standards for ingested, governed, transformed, persisted data.
• Provide expertise in ELT pipeline performance optimization.
• Enable adoption of advanced analytics and AI via best-in-class tooling.
• Evolve infrastructure-as-code alongside cloud engineering.
• Align designs with data privacy, security, and compliance policies.
• Stay updated on Snowflake and big data trends and propose advancements.

Skills

Snowflake environment

End-to-end pipelines

Python programming

CI/CD processes

Agile teamwork

Docker

Event streaming

Pyspark

AI engineering awareness

Tools

Snowflake

Snowpark

Python

GitHub Actions

TeamCity

Artifactory

Docker

Kafka

Pulsar

Azure Event Hub

Collibra

Pyspark

Job description

We are looking for an experienced Principal Data Engineer to join our growing team building our strategic data

platform as we look to adopt best in class data governance and democratise data around the business. The

successful candidate will play a pivotal role in shaping and executing our data strategy, leveraging our data

assets to the full, elevating our decision-making, streamlining processes, and drive innovation within our

organisation.

You will be part of the long-standing Data Platform team using Agile methods to deliver solutions on top of our

cloud platform. Starting with Snowflake as a foundational component, we will look to integrate other

technologies and grow capability to meet the demands of the business. You will work closely with a variety of

business areas and be involved in all aspects of the Secure System Development Lifecycle (S-SDLC).

We operate in a collaborative and inclusive environment, where each team member's expertise is valued. Our

culture emphasizes open communication, mutual support, and a shared commitment to delivering exceptional

results. We foster a learning culture, providing opportunities for growth and innovation while maintaining high

standards of professionalism and integrity.

Role Responsibilities

• Contribute to the solution design and implementation of a robust and scalable data platform using
• Snowflake with complementary tooling to meet the organisation's needs
• Help evolve the Insight Data Strategy, communicate outcomes from a leadership through to team level
• Set the firm-wide standards and conventions for ingesting, governing, transforming, persisting, and democratising data
• Provide expertise in ELT pipeline performance optimisation
• Enable the adoption of advanced analytics & AI by configuring best-in-class tooling
• Evolve the usage of infrastructure-as-code in collaboration with cloud engineering
• Align technical designs with internal data, privacy, security and compliance policies
• Stay abreast of industry trends and emerging technologies related to Snowflake and big data processing, recommending relevant advancements for consideration

Experience Required

• Snowflake (Warehouses, Tasks/Streams, Snowpipe, Snowpark, Distributed UDF/Sproc, Time Travel, Cloning)
• Demonstratable experience of delivering end to end pipelines in Snowpark (transferable frameworks such as Pyspark, Polars or Dask are also welcome)
• Python including build tools, test automation frameworks and engineering best practices
• Experience with CI/CD pipelines (GitHub Actions / TeamCity / Artifactory)
• Demonstrate initiative and independence to solve problems. § Be able to communicate abstract or complex ideas or solutions effectively. § Stand by software engineering principles around quality and doing things the right way.
• Experience working in and with Agile teams performing iterative development. § Ability to collaborate with end users, stakeholders, teams, and team members. § Have a constant desire to improve knowledge and drive adoption of new tech or approaches within the teams. § Own mistakes and work with your colleagues to redress them, learn from them and make improvements for the long-term
• Docker
• Event streaming technologies such as Kafka, Pulsar, or Azure Event Hub
• Integration with Collibra
• Pyspark
• AI engineering (Snowflake AI, Langchain, Llamaindex, Pinecone, etc)

Insight is committed to being an inclusive employer and encourages applications from all suitably qualified applicants irrespective of background, circumstances, age, disability, gender identity, ethnicity, religion or belief, sexual orientation or other factors protected by federal, state and/or local laws. If you are a candidate with a disability, or are assisting a candidate with a disability, and require an accommodation to apply for one of our jobs, please email us at [e-mail]

About Insight Investment

Insight Investment is a leading asset manager focused on designing investment solutions to meet its clients' needs. Founded in 2002, Insight's collaborative approach has delivered both investment performance and growth in assets under management. Insight manages assets across its core liability-driven investment, risk management, full-spectrum fixed income, currency and absolute return capabilities. Insight has a global network of operations in the UK, Ireland, Germany, US, Japan and Australia. More information about Insight Investment can be found at:https://www.insightinvestment.com/corporate/