Senior Data Engineer: Real-Time IoT for Energy

AgnesCole Consulting Services Pvt Ltd; Netherlands; On-site; EUR 70,000 - 95,000; Full time

Job summary

AgnesCole Consulting Services Pvt Ltd seeks a Senior Data Engineer to design, build and optimise scalable data pipelines and cloud-based data platforms for energy and renewables data. You will work with Python, SQL, Azure Databricks and Spark to process IoT, operational and real-time energy data and ensure data quality and security across pipelines.

Ideal candidates have hands-on Py/SQL, Delta Lake, and Azure experience, plus strong collaboration with architects, analysts and engineering teams

Qualifications

• Strong hands-on experience with Python and SQL.
• Advanced expertise in Azure Databricks and Apache Spark.
• Proven experience developing real-time and batch data pipelines.
• Strong knowledge of Delta Lake and modern data platform architecture.

Responsibilities

• Design, build and maintain scalable data pipelines using Python, SQL, Azure Databricks and Apache Spark.
• Develop real-time and batch data processing solutions for energy and industrial IoT data.
• Implement and optimise Delta Lake architecture for efficient data storage and processing.
• Design and maintain cloud-based data platforms within Microsoft Azure.
• Integrate data from IoT sensors, operational systems and external data sources.
• Ensure data quality, reliability, performance and security across data pipelines.
• Collaborate with data architects, analysts and engineering teams to deliver robust data solutions.
• Support performance optimisation, troubleshooting and continuous improvement of data infrastructure.

Skills

Python

SQL

Azure Databricks

Apache Spark

Real-time processing

Data modelling

Tools

Delta Lake

Job description

AgnesCole Consulting Services Pvt Ltd seeks a Senior Data Engineer to design, build and optimise scalable data pipelines and cloud-based data platforms for energy and renewables data. You will work with Python, SQL, Azure Databricks and Spark to process IoT, operational and real-time energy data and ensure data quality and security across pipelines.

Ideal candidates have hands-on Py/SQL, Delta Lake, and Azure experience, plus strong collaboration with architects, analysts and engineering teams