Data Engineer: AWS Cloud, Big Data Pipelines

PA Consulting; Manchester; Hybrid; GBP 52,000 - 92,000; Full time

Benefits offered by this job

Hybrid working two days in office

Private healthcare benefits

25 days annual leave + Christmas Eve

Pension scheme

Annual bonus opportunity

PA share ownership

Job summary

PA Consulting is seeking a data-focused professional to design and deploy production data pipelines on AWS. You will collaborate with product, design, and cross-disciplinary experts to solve client problems and advance internal projects. Hybrid work is offered with at least two days in the office or on client site.

The role emphasizes hands-on work with Java/Python/Scala/Spark/SQL in a big data environment, and opportunities to grow through certifications and a broad project mix across sectors.

Qualifications

• Strong problem-solving and analytical thinking.
• Collaborates with multiple stakeholders in fast-paced environments.
• Experience designing and deploying production data pipelines from ingestion to consumption in big data architectures using Java, Python, Scala, Spark and SQL.
• Experience writing scripts, extracting data via APIs, and writing SQL queries.
• Experience processing large amounts of structured and unstructured data, including integration from multiple sources via ingestion and data curation on AWS.
• Experience with AWS cloud technologies for ETL pipelines, data warehouses and data lakes, and data movement.
• Experience with AWS data and analytics services or open-source equivalents such as EMR, Glue, Redshift, Kinesis, Lambda, and DynamoDB.

Responsibilities

• Work within agile best practices and cross-functional teams.
• Apply technical skills to solve client problems and internal projects.
• Design and build for the AWS cloud.
• Bring ideas to life through innovative software with product and design teams.
• Participate in in-person whiteboarding and asynchronous problem-solving sessions.

Skills

Problem solving

Analytical thinking

Stakeholder collaboration

Big data experience

Python

Java

SQL

Scala

Spark

Data ingestion

Tools

AWS

Lambda

Redshift

EMR

Glue

Kinesis

DynamoDB

SQL

Java

Python

Spark

Big Data

Job description

PA Consulting is seeking a data-focused professional to design and deploy production data pipelines on AWS. You will collaborate with product, design, and cross-disciplinary experts to solve client problems and advance internal projects. Hybrid work is offered with at least two days in the office or on client site.

The role emphasizes hands-on work with Java/Python/Scala/Spark/SQL in a big data environment, and opportunities to grow through certifications and a broad project mix across sectors.