Data Platform Engineer - DV Cleared

CBSbutler Holdings Limited trading as CBSbutler; England; On-site; GBP 80,000 - 100,000; Full time

Job summary

A technology consultancy is seeking a Data Platform Engineer based in the UK to design and implement secure data pipelines using the Elastic Stack and Apache NiFi. The successful candidate will ensure compliance with security standards while managing large-scale data flows in real-time. Key responsibilities include developing data models, visualizing data in Kibana, and ensuring system performance. This role offers a daily rate between £500 and £600, outside IR35, and includes options for compressed hours.

Qualifications

• Strong expertise in building and managing data pipelines using the Elastic Stack and Apache NiFi.
• Experience in ensuring data quality and security compliance.
• Ability to document data architecture and workflows.

Responsibilities

• Design, develop, and maintain secure and scalable data pipelines.
• Implement data ingestion and transformation processes.
• Collaborate with data architects and security teams.
• Manage large-scale data flows for performance and reliability.
• Develop data models for analytics and reporting.
• Perform troubleshooting and performance tuning.
• Build dashboards and visualizations in Kibana.
• Document workflows and security protocols.

Skills

Building and managing data pipelines

Elastic Stack (Elasticsearch, Logstash, Kibana)

Apache NiFi

Data quality and security compliance

Troubleshooting and debugging

Data quality assurance

Job description

Data Platform Engineer DV Cleared

500 - 600 per day - Outside IR35

JOB DESCRIPTION

This role requires strong expertise in building and managing data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi. The successful candidate will design, implement, and maintain scalable, secure data solutions, ensuring compliance with strict security standards and regulations. This is a UK based onsite role with the option of compressed hours. The role will include:

• Design, develop, and maintain secure and scalable data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi.
• Implement data ingestion, transformation, and integration processes, ensuring data quality and security.
• Collaborate with data architects and security teams to ensure compliance with security policies and data governance standards.
• Manage and monitor large-scale data flows in real-time, ensuring system performance, reliability, and data integrity.
• Develop robust data models to support analytics and reporting within secure environments.
• Perform troubleshooting, debugging, and performance tuning of data pipelines and the Elastic Stack.
• Build dashboards and visualizations in Kibana to enable data-driven decision-making.
• Ensure high availability and disaster recovery for data systems, implementing appropriate backup and replication strategies.
• Document data architecture, workflows, and security protocols to ensure smooth operational handover and audit readiness.