Data Engineer

Experis; London; Remote; GBP 100,000 - 125,000; Full time

Job summary

A leading technology consultancy is seeking a Data Engineer to work remotely for a 7-month contract. The ideal candidate will possess advanced skills in Python and PySpark, with strong knowledge of data management and processing. This role involves developing data pipelines and ensuring adherence to data security principles in a dynamic Agile environment.

Qualifications

• Strong understanding of data types, structures, and schemas.
• Experience with Data Pipelines using Python/PySpark frameworks.
• Familiarity with Agile methodologies and incident triage.

Responsibilities

• Manage data concepts and structures effectively.
• Develop data pipelines with custom frameworks.
• Ensure data security principles are upheld.

Skills

Data concepts

Complex JSON manipulation

Data Pipelines

Data Security principles

Problem solving

Analytical skills

Linux system administration

Agile environments

Tools

Python

PySpark

SQL

Azure Databricks

Apache Spark

PowerBI

Git

Job description

Social network you want to login/join with:

Location: City of London | Job Type: Contract | Industry: Cloud & Infrastructure | Job reference: BBBH386530_1742318777 | Posted: about 2 hours ago

Data Engineer

Remote | 7 months | Inside ir35 - Umbrella only

Required skills

• Strong understanding of data concepts - data types, data structures, schemas (both JSON and Spark), schema management etc.
• Strong understanding of complex JSON manipulation.
• Experience working with Data Pipelines using custom Python/PySpark frameworks.
• Understanding of the 4 core Data categories (Reference, Master, Transactional, Freeform) and managing Reference Data.
• Knowledge of Data Security principles - data owners, access controls, GDPR, handling sensitive datasets.
• Strong problem solving and analytical skills, able to demonstrate these intuitively.
• Experience in support roles, incident triage, and handling skills (SLAs).
• Fundamental Linux system administration knowledge - SSH keys, Bash CLI, scripting, environment variables.
• Experience with browser-based IDEs (Jupyter Notebooks, RStudio).
• Experience working in dynamic Agile environments (SAFE, Scrum, sprints, JIRA).

Languages / Frameworks

• JSON
• YAML
• Python (advanced, Pydantic experience a bonus)
• SQL
• PySpark
• Delta Lake
• Bash (CLI usage and scripting)
• Git
• Markdown
• Scala (bonus)
• Azure SQL Server as HIVE Metastore (bonus)

Technologies

• Azure Databricks
• Apache Spark
• Delta Tables
• Data processing with Python
• PowerBI (Data Ingestion / Integration)
• JIRA

If this role suits you, please submit your CV at your earliest convenience. If you do not receive a response within 2 weeks, consider your application unsuccessful.