Data Engineer

CFP Energy Ltd.; City Of London; On-site; GBP 40,000 - 60,000; Full time

Job summary

A reputable energy supplier in London is seeking a Data Engineer to join its data engineering team. The successful candidate will design and maintain data ingestion pipelines and work with open-source technologies. Ideal for someone with strong SQL and Python skills and an interest in the energy industry. This role offers a collaborative environment with opportunities for technical innovation.

Qualifications

• 2-5 years of experience in data engineering or related roles.
• Experience working in the Energy industry.

Responsibilities

• Design, develop, and maintain data ingestion pipelines.
• Build and optimize ETL/ELT processes.
• Develop data models and schemas for analytics.
• Monitor and optimize data pipeline performance.
• Collaborate with stakeholders to understand data requirements.

Skills

SQL skills

Python programming

Data ingestion pipeline maintenance

Data processing libraries (pandas, NumPy)

Open-source data processing frameworks (Apache Spark, Kafka, Airflow)

Distributed computing concepts

Version control systems (Git)

Containerization technologies (Docker, Kubernetes)

Data orchestration tools (Apache Airflow)

Cloud platforms with IaC

Education

Bachelor's degree in Computer Science, Engineering, Mathematics

Job description

The Role

Brook Green Supply is an independent B2B energy supply company. You will be joining a highly experienced team at the forefront of the UK energy market, providing a fantastic opportunity for a driven and commercially minded individual to make a meaningful contribution to the growth and evolution of our trading desk.

We are seeking a Data Engineer to join our growing data engineering team. This role is perfect for someone passionate about building robust, scalable data infrastructure using open-source technologies. You'll work in an environment that values innovation, technical excellence, and collaborative problem-solving.

Key Responsibilities

• Design, develop, and maintain data ingestion pipelines using open-source frameworks and tools
• Build and optimise ETL/ELT processes to handle small to large-scale data processing requirements
• Develop data models and schemas that support analytics, business intelligence and product needs
• Monitor, troubleshoot, and optimise data pipeline performance and reliability
• Collaborate with stakeholders, analysts and product team to understand data requirements
• Implement data quality checks and validation processes to ensure data integrity
• Participate in architecture decisions and contribute to technical roadmap planning

Analytical and Soft Skills

• Strong analytical mindset with attention to detail
• Excellent problem-solving and debugging skills
• Ability to work independently and manage priorities
• Strong communication skills and ability to explain technical concepts to non-technical stakeholders
• Experience working in agile development environments
• Passion for continuous learning and staying current with emerging technologies

Required Skills and Experience

Technical Skills

• Great SQL skills with experience in complex query optimization
• Strong Python programming skills with experience in data processing libraries (pandas, NumPy, Apache Spark)
• Hands-on experience building and maintaining data ingestion pipelines
• Proven track record of optimising queries, code, and system performance
• Experience with open-source data processing frameworks (Apache Spark, Apache Kafka, Apache Airflow)
• Knowledge of distributed computing concepts and big data technologies
• Experience with version control systems (Git) and CI/CD practices
• Experience with relational databases (PostgreSQL, MySQL or similar)
• Experience with containerization technologies (Docker, Kubernetes)
• Experience with data orchestration tools (Apache Airflow or Dagster)
• Understanding of data warehousing concepts and dimensional modelling
• Understanding of cloud platforms using infrastructure-as-code (IaC) approaches
• Knowledge of streaming data processing and real-time analytics
• Experience with data quality and monitoring tools

Preferred Qualifications

• Bachelor\'s degree in Computer Science, Engineering, Mathematics, or related field
• 2-5 years of experience in data engineering or related roles
• Experience working in the Energy industry

The CFP Energy Group is committed to ensuring equal opportunities, fairness of treatment, dignity and respect, and the elimination of all forms of discrimination in the workplace for all employees/contractors and job applicants.