Data Engineer

Creditsafe; Cardiff; Hybrid; GBP 60,000 - 90,000; Full time

Benefits offered by this job

Competitive salary

Company laptop

Bonus scheme

25 days annual leave

Hybrid working model

Healthcare & company pension

E-learning and career development

Job summary

Creditsafe is building a new Consumer Bureau in the UK. We seek a Data Engineer experienced with cloud-native systems to drive data pipelines, quality, and scalability. You will work with Python, Spark, Airflow, and AWS technologies to deliver real-time and batch solutions.

Join a collaborative team focused on high throughput, low latency data delivery and data quality, with opportunities for mentoring and leadership across engineering disciplines.

Qualifications

• You understand data engineering best practices.
• You write clean and efficient code and leverage the cloud for performance.
• 6+ years of development experience building production-grade data pipelines in Python.
• You enjoy solving problems and collaborating with teammates to deliver robust solutions.
• Excellent communication and ability to explain viewpoints clearly to the team.

Responsibilities

• Contribute actively to the codebase and participate in peer reviews.
• Design and build a metadata-driven, event-based distributed data processing platform using Python, Spark, Airflow, Iceberg, DynamoDB, AWS Glue, S3.
• Play an active role in design, development, and deployment of the business-critical system.
• Build and scale Creditsafe’s data pipelines with batch and streaming approaches and sub-500ms latency.
• Understand domain data to make recommendations to improve the product.
• Utilise AI tools to assist in all aspects of work.

Skills

Data engineering

Python

Cloud platforms

Problem solving

Team collaboration

Tools

Python

Apache Spark

Apache Airflow

Iceberg

DynamoDB

AWS

S3

Kinesis

Job description

Join the team, an exciting new project aiming to have us enter the Consumer Credit Bureau space in the United Kingdom and eventually become a fully-fledged Consumer Reporting Agency. We’re looking for a Data Engineer experienced in testing cloud-native software systems.

WHO WE ARE

Privately owned and independently minded, at Creditsafe, we are guided by values of customer-focus, trust, respect, and a willingness to try new things. For over 20 years, these values have shaped a culture of growth, innovation, and support, where our team members build successful, long-term careers.

With a diverse team spread across 25 offices in 14 countries, we value collaboration, learning, and problem-solving together. We have been featured twice in The Sunday Times' '100 Best Companies to Work For' and continue to prioritize well-being, career growth, and inclusion.

Our continued success has made us the most used business data provider in the world and one of the industry’s strongest growing business intelligence providers. Today Creditsafe has 23 offices across 3 continents serving110,000 customersfrom SMEs to enterprise businesses.

THE TEAM

Join a team of highly technically skilledengineerswhoaredesigningCreditsafe’snewConsumer Bureauwithdata quality,lineage, andauditabilityas primary goals whilstmaintaininghigh throughputand scalability.Thedata platformwill be builtusingAWSinfrastructure such as S3and S3-tables(Iceberg), AuroraServerless, DynamoDB, Lambda,Fargate, ECS, Airflow,Sparkand Kinesis.Theplatformisexpectedto manage overone hundred millionobjectswith a high volume of daily updateshandlingaddition, deletion,andcorrectionof our dataand indexesinanauditable manner.The platform will employ a sophisticated matching algorithm to enable us to assign incoming updates to existing objects.Our data processing application is entirely based onPythonand designed toefficientlytransformincomingrawdata volume intoAPIconsumableschema.The teamis alsobuildinghighlyavailableand low latency APIstoenableour clientswithfasterdata delivery.

WHAT YOU'LL OWN

Contributing activelyto the codebaseandparticipatingin peer reviews.This is not a passive "take a ticket and close it" role -- you will drive development from 3-Amigo's refinement sessions all the way to post-deployment observability.

Design and build metadata driven,eventbaseddistributeddata processing platformusing technologies such asPython,Spark,Airflow, Iceberg,DynamoDB,AWS Glue, S3.

Playan activerole in the design, development, and deployment ofour business-criticalsystem.

Building and scalingCreditsafe’sdata pipelinescombining batch and streaming approachesandtargetingsub-500ms latency serving.

Understandingdomaindatatomake recommendationsto improvetheexisting product.

Employing various AI tools such as Perplexity, Cursor, CoPilot, etc to aid in all aspects of your work.

WHAT YOU WIILL NEED TO BRING

You understand and can implement data engineering best practices.

You demonstrate the ability to write clean&efficient code and combine it with the cloud environment forbestperformance.

You have a minimum6years ofdevelopment experience within a commercial environment creating production gradedatapipelines in python.

You are looking to growyour skillsthrough daily technical challenges andenjoyproblem solving,whiteboardingin collaboration with team.

You have excellentcommunication skills,andability to explain your viewsclearlytotheteam andare open tounderstandingtheviews of others.

You have a proventrack recordto draw from a deep and broad technicalexpertiseto mentor engineers, complete hands-on technical work, and provide leadership on complex technology issues.

You share your ideascollaborativelyviawikis,discussions boards, etcandshareany decisionsmade,for the benefit of others.

You take ownership of end-to-end deliverables from planning all the way to production.

WHY CREDITSAFE

• IC ladder with teeth.Staff and Principal are a destination, not a waypoint - with scope, comp and influence that genuinely match the title.

• Written design culture.Anything non-trivial gets an RFC. Decisions are recorded. We disagreeindocuments, not in meetings.

• Real operability.We run what we build. No hand-offs to a separate ops team.

• Pragmatic agile.Teams own their own process. Weoptimisefor shipping, not ceremony.

• Customer-first. Our focus is on making work that adds value to our clients first. Stability, freshness, durability, trust – non-negotiable.

• Leadershipyou’ll want to work with.Engineering leadership is technical, present, and will back your calls.

Benefits

• Competitive Salary.

• Company Laptop.

• Bonus Scheme.

• 25 Days Annual Leave (plus bank holidays).

• Hybrid working model.

• Healthcare & Company Pension.

• E-learning and career development opportunities.

• Global company gatherings and events.

• Plusmore that can be foundonthe benefits section on the Careers page, https://careers.creditsafe.com/gb.

We are committed to providing a welcoming, supportive environment. Creditsafe is an equalopportunitiesemployer that values diversity. Please contact us if you need anyassistancewith your application.