DATA ENGINEER - SC CLEARED

TN United Kingdom; London; Remote; GBP 50,000 - 80,000; Full time

Job summary

An established industry player is seeking a Data Engineer with advanced Python and PySpark skills to design and maintain data pipelines. This fully remote position requires occasional travel to Central London and involves managing complex data manipulation and ensuring compliance with data security principles. Candidates with high-level security clearance are encouraged to apply. Join a dynamic Agile environment where your problem-solving skills will shine, and help shape the future of data management in a collaborative team setting.

Qualifications

• Advanced knowledge of Python and PySpark for data pipeline development.
• Experience with data security principles and GDPR compliance.

Responsibilities

• Design and maintain data pipelines using Python and PySpark frameworks.
• Implement data security principles and manage complex data types.

Skills

Python

PySpark

SQL

JSON

YAML

Bash scripting

Git

Markdown

Tools

Jupyter Notebooks

RStudio

JIRA

Job description

Social network you want to login/join with:

Position: Data Engineer - SC Cleared

Location: Fully remote with occasional travel to Central London

IR35 Status: Inside

Duration: 6-month initial contract

Job Responsibilities:

• Design, develop, and maintain data pipelines using custom Python/PySpark frameworks.
• Manage complex JSON data manipulation and schemas (JSON and Spark).
• Handle reference data, master data, transactional data, and freeform data, understanding their implications.
• Implement data security principles, including access controls and GDPR compliance, especially for sensitive datasets.
• Support incident triage and handling, demonstrating strong problem-solving skills.
• Perform basic Linux system administration tasks, including SSH, Bash scripting, and environment configuration.
• Utilize browser-based IDEs such as Jupyter Notebooks and RStudio.
• Work within a dynamic Agile environment using tools like JIRA, Scrum, and SAFE methodologies.

Required Skills and Technologies:

• Languages: JSON, YAML, Python (advanced), SQL, PySpark, Delta Lake, Bash scripting, Git, Markdown
• Optional/Bonus: Scala, Azure SQL Server, PowerBI, Azure Databricks, Apache Spark, Delta Tables

Additional Information:

Candidates with prior high-level security clearance are highly encouraged to apply. Successful applicants will require security clearance, which may take up to 10 weeks.