Data Engineer

Energetech; Dubai; On-site; AED 60,000 - 100,000; Full time

Job summary

An innovative firm is seeking a Data Engineer to enhance their real-time trading systems. In this pivotal role, you will design and maintain robust data pipelines, manage Data Lakes, and support the deployment of ML/AI models. Your expertise in programming languages and modern data technologies will be crucial in driving technical excellence. Collaborating with a dynamic team, you will automate processes to improve efficiency and ensure data integrity. If you're passionate about technology and sustainability, this is an exciting opportunity to make a significant impact in a forward-thinking environment.

Qualifications

• Deep knowledge of programming languages such as Java, C#, C++, and Python.
• Significant experience with modern data technologies like Postgres and MongoDB.

Responsibilities

• Design and maintain robust data pipelines for real-time trading applications.
• Manage and optimize Data Lakes and data warehousing solutions.

Skills

Java

C#

C++

Python

Postgres

MongoDB

Kafka/Event Hub

MLOps

Data Integrity

Environmental Sustainability

Education

Bachelor of Technology/Engineering(Computers)

Tools

Docker

Kubernetes

Azure

AWS

GCP

Databricks

Dagster

Airflow

Job description

Bachelor of Technology/Engineering(Computers)

Nationality

Any Nationality

Vacancy

1 Vacancy

Job Description

About the role

As a Data Engineer, you will help us drive technical excellence and innovation. In this role, you will play a crucial role in developing and maintaining the infrastructure that powers our real-time trading systems. With a focus on strategic value, you will be responsible for managing multiple projects while supporting the implementation of new business and technology changes.

Your responsibilities

• Design, build, and maintain robust data pipelines for real-time trading applications.
• Manage and optimise Data Lakes and data warehousing solutions.
• Support development and deployment of ML/AI Model, using MLOps best practices
• Oversee data collection processes and ensure high standards of data integrity.
• Employ automation to enhance data processes and system efficiency.
• Develop and maintain scalable data pipelines to support real-time decision-making.
• Manage the storage and accessibility of data in Data Lakes and warehousing systems.
• Implement automation strategies to improve data integrity and processing.
• Collaborate with Software Engineers and other Data Engineers on diverse projects to enhance trading systems.

Qualifications

• Deep knowledge of programming languages such as Java, C# and/or C++ and developing application in Python
• Significant experience working with modern data technologies, like Postgres, Timescale dB, Quest dB, MongoDB, Kafka/Event Hub
• Acquaintance in deploying and managing microservices with docker, Kubernetes and bash scripting
• Expert knowledge of Azure, AWS or GCP and management of large-scale data pipeline system
• Desired capabilities with MLOps Frameworks like Kubeflow, MLFlow, Databrick ML tools, etc
• Experience with Databricks, Data Lakes, and workflow orchestration tools like Dagster or Airflow.
• Problem solver oriented with a keen interest in environmental sustainability and technological innovation.
• Familiarity with automated trading systems and real-time data streaming for high-frequency trading, is beneficial but not mandatory.

Disclaimer: Naukrigulf.com is only a platform to bring jobseekers & employers together. Applicants are advised to research the bonafides of the prospective employer independently. We do NOT endorse any requests for money payments and strictly advice against sharing personal or bank related information. We also recommend you visit Security Advice for more information. If you suspect any fraud or malpractice, email us at [e-mail]