Data Engineer

Talent Smart; Sheffield; On-site; GBP 60,000 - 90,000; Full time

Job summary

Talent Smart is seeking an experienced Data Engineer to join a high-performing data team on a 6-month contract for a leading enterprise financial services organisation. The role is Sheffield-based with 3 days on-site per week.

You will design, develop and support enterprise-scale data pipelines, optimise Spark performance using PySpark and Databricks, and resolve runtime issues in a complex, large-scale environment. Inside IR35, initial six-month engagement.

Qualifications

• Hands-on PySpark and Databricks expertise.
• Extensive experience designing, developing, and supporting enterprise-scale data pipelines.
• Strong ability to optimise Spark performance in large-scale environments.

Responsibilities

• Design, develop and support enterprise-scale data pipelines.
• Troubleshoot and resolve Spark job performance issues.
• Contribute to a major data transformation programme within a financial services context.

Skills

PySpark

Databricks

Spark optimization

Troubleshooting

ETL pipelines

Data engineering

Big data

Job description

Data Engineer

We're supporting a leading enterprise financial services organisation in the search for an experienced Data Engineer to join a high-performing data team on an initial 6-month contract.

Contract Details:

• Inside IR35
• Initial 6-month engagement
• Sheffield-based, 3 days per week on-site

Key Requirements:

• Strong hands-on experience with PySpark and Databricks
• Deep understanding of Spark performance optimisation, including lazy evaluation, column pruning, groupBy operations, and window functions
• Proven ability to troubleshoot and resolve Spark job performance issues
• Extensive experience designing, developing, and supporting enterprise-scale data pipelines
• Strong technical depth across modern data engineering platforms, tools, and technologies
• Experience working within complex, large-scale enterprise environments

This is an excellent opportunity to contribute to a major data transformation programme within a leading financial services organisation.

Additional information available on successful application