Data Engineer - Databricks (Newcastle or Erskine)

DXC Technology; Newcastle upon Tyne; Hybrid; GBP 50,000 - 70,000; Full time

Job summary

A leading tech company is seeking an experienced Data Engineer specializing in Databricks to join their team in Newcastle or Erskine. You will build scalable data pipelines, develop ETL workflows, and collaborate with various teams on impactful projects. The ideal candidate has over 5 years of relevant experience, strong skills in PySpark, and is willing to achieve Databricks certification. This role offers hybrid working options and a supportive company culture.

Benefits

Continuous learning and development opportunities

Flexible working arrangements

Inclusive and supportive company culture

Qualifications

• 5+ years of relevant work experience in industry.
• Proven experience with Databricks and its ecosystem.
• Databricks Certified Data Engineer Associate or willingness to achieve one.

Responsibilities

• Build and maintain scalable data pipelines using Databricks.
• Develop modular ETL workflows using dbt and orchestrate them with Apache Airflow.
• Collaborate with data scientists and analysts to deliver data solutions.

Skills

Databricks

PySpark

dbt

Airflow

Data cleansing

Exploratory data analysis

Data visualization

Education

Bachelor's degree in a relevant field

Job description

Job Description:

Job Opportunity: Data Engineer (Databricks Specialist) - Newcastle or Erskine Area

Location: Newcastle, UK or Erskine, Scotland
Type: Full-Time
Salary: Competitive + Benefits
Remote Work: Hybrid options available

About the Role:

We're on a mission to harness the power of data to drive innovation and deliver transformative solutions. We're looking for a talented Data Engineer with deep expertise in Databricks to join our growing team and help shape the future of our data projects. This is a fantastic opportunity to work on cutting-edge AI and data engineering projects using the latest tools and technologies in a collaborative and innovative environment.

Please note that due to security clearance requirements, candidates should be sole UK nationals who are willing to undergo SC clearance. Applicants should also be willing to travel within the UK when required depending on project requirements.

Key Responsibilities:

• Build and maintain scalable data pipelines using Databricks.
• Manage and optimize Databricks Workspace, Clusters, Jobs, Repos, and Delta Live Tables.
• Develop modular ETL workflows using dbt and orchestrate them with Apache Airflow.
• Write advanced PySpark code, including UDFs and Pandas UDFs, for efficient data processing.
• Ensure data quality, reliability, and performance across all systems.
• Collaborate with data scientists, analysts, and engineers to deliver data solutions that support business goals.
• Contribute to the mentoring and development of junior team members.
• Support senior team members in identifying and addressing data science opportunities.

Required Skills & Experience:

• Proven experience with Databricks and its ecosystem
• Strong proficiency in PySpark, especially with UDFs and Pandas UDFs
• Hands-on experience with dbt and Airflow
• Databricks Certified Data Engineer Associate certification or willingness to achieve one
• Excellent problem-solving skills and a passion for data and AI
• Bachelor's degree in a relevant field or equivalent combination of education and experience
• Typically, 5+ years of relevant work experience in industry, with a minimum of 2+ years in a similar role.
• Proficiencies in data cleansing, exploratory data analysis, and data visualization
• Continuous learner that stays abreast with industry knowledge and technology

Why Join Us?

• Work with a forward-thinking team on impactful AI projects
• Access to continuous learning and development opportunities
• Flexible working arrangements
• Inclusive and supportive company culture

At DXC Technology, we believe strong connections and community are key to our success. Our work model prioritizes in-person collaboration while offering flexibility to support wellbeing, productivity, individual work styles, and life circumstances. We're committed to fostering an inclusive environment where everyone can thrive.