DATA ENGINEER - SC CLEARED

Onyx-Conseil; London; Remote; GBP 50,000 - 80,000; Full time

Job summary

An innovative consultancy is seeking a skilled Data Engineer to join their dynamic team. This role offers the opportunity to work fully remote while engaging in exciting projects that involve complex data manipulation and pipeline development. Ideal candidates will possess advanced proficiency in Python and PySpark, along with a solid understanding of data security principles. You'll be part of a forward-thinking environment that values diversity and inclusion, with a commitment to excellence recognized by prestigious awards. If you're passionate about data and eager to make an impact, this is the perfect opportunity for you.

Qualifications

• Strong understanding of data concepts and data structures.
• Experience with complex JSON manipulation and data pipelines.

Responsibilities

• Develop and maintain data pipelines using Python and PySpark.
• Ensure data security and compliance with GDPR regulations.

Skills

JSON Manipulation

Python/PySpark

Data Security Principles

Analytical Skills

Linux Administration

Agile Methodologies

Tools

Azure Databricks

Apache Spark

PowerBI

JIRA

Git

Job description

REQUIRED: DATA ENGINEER - SC CLEARED

Location:

Fully remote with occasional travel to Central London

IR35 Status:

Inside

Duration:

6-month initial contract

Job Description:

Seeking a Data Engineer with a strong understanding of data concepts, including data types, data structures, schemas (JSON and Spark), and schema management.

Key Skills and Experience:

1. Strong understanding of complex JSON manipulation
2. Experience with Data Pipelines using custom Python/PySpark frameworks
3. Knowledge of the 4 core Data categories (Reference, Master, Transactional, Freeform) and handling Reference Data
4. Understanding of Data Security principles, access controls, GDPR, and handling sensitive datasets
5. Strong problem-solving and analytical skills
6. Experience in support roles, incident triage, and handling (SLAs)
7. Linux system administration basics, Bash scripting, environment variables
8. Experience with browser-based IDEs like Jupyter Notebooks
9. Familiarity with Agile methodologies (SAFE, Scrum, JIRA)

Languages and Frameworks:

• JSON
• YAML
• Python (advanced proficiency, Pydantic bonus)
• SQL
• PySpark
• Delta Lake
• Bash
• Git
• Markdown
• Scala (bonus)
• Azure SQL Server (bonus)

Technologies:

• Azure Databricks
• Apache Spark
• Delta Tables
• Data processing with Python
• PowerBI (Data ingestion and integration)
• JIRA

Additional Notes:

Candidates with current or past high-level security clearance are encouraged to apply. Successful candidates will need to undergo security clearance, which may take up to 10 weeks.

LA International is a globally operating, HMG-approved ICT Recruitment and Project Solutions Consultancy, committed to diversity and inclusion.

Recognitions include multiple awards, including The Queen's Award for Enterprise: International Trade.