Data Engineer - Capital Markets or Stock Exchange

micro1; Dubai; On-site; AED 200,000 - 300,000; Full time

Job summary

A leading financial services institution in Dubai is seeking a Senior Data Engineer specializing in capital markets. Join the team to shape the data infrastructure, design and optimize data solutions, and support business intelligence initiatives. The ideal candidate has significant experience in data engineering, particularly within stock exchanges, and strong command of SQL and big data technologies.

Qualifications

• 6-9 years of data engineering experience, with 2-5 years at a stock exchange.
• Proficiency in developing end-to-end ETL pipelines and integrating data sources.
• Hands-on experience with big data tools and event streaming technologies.

Responsibilities

• Design and maintain scalable data architectures and ETL pipelines.
• Implement real-time event streaming frameworks using Kafka.
• Collaborate with data scientists and stakeholders on data requirements.

Skills

SQL

Kafka

Hadoop

Spark

MySQL

Event Streaming

Metadata Management

Database Optimization

Capital Markets Domain Knowledge

Problem-Solving

Documentation

Education

Bachelor's degree in Computer Science or related discipline

Tools

PostgreSQL

MongoDB

Job description

Data Engineer - Capital Markets or Stock Exchange

Required Skills

• sql
• kafka
• hadoop
• spark
• mysql
• event streaming
• metadata management
• database optimization
• capital markets domain knowledge
• problem-solving
• documentation

Job Description

Job Type: Full-time

Job Summary:

Join our team as a Senior Data Engineer - Capital Markets and play a vital role in shaping the data infrastructure for a dynamic stock exchange environment. You will design, implement, and optimize robust data solutions that power advanced business intelligence and real-time insights for our leading financial services institution. Our async culture values written communication, enabling deeper focus and effective collaboration across the business.

Key Responsibilities

• Design, develop, and maintain scalable, secure data architectures and ETL pipelines to support business intelligence initiatives.
• Implement and optimize real-time event streaming frameworks and services using technologies such as Kafka.
• Manage and optimize relational and non-relational databases to ensure high performance and availability.
• Leverage big data technologies (Hadoop, Spark, Kafka) to process, analyze, and deliver insights from large volumes of capital markets data.
• Establish and maintain robust metadata management and data lineage practices, ensuring data quality and compliance.
• Collaborate closely with data scientists, business analysts, and stakeholders to understand and deliver on evolving data requirements.
• Lead cross-functional problem solving for data integrity, reliability, and performance issues.

Required Skills and Qualifications:

• Bachelor's degree in Computer Science, Information Technology, or a related discipline.
• 6-9 years of data engineering experience, including 2-5 years specifically within a leading stock exchange (e.g., Dubai, London, etc.).
• Strong command of SQL and expertise with database management systems (MySQL, PostgreSQL, MongoDB).
• Proficiency in developing end-to-end ETL pipelines and integrating multiple data sources.
• Hands-on experience with big data tools (Hadoop, Spark) and event streaming technologies (Kafka, Event Hub, etc.).
• Solid grasp of DevOps practices relevant to data engineering workflows.
• Exceptional problem-solving abilities and a meticulous approach to data quality and documentation.

Preferred Qualifications

• Prior experience working with or supporting data initiatives for capital markets or stock exchanges.
• Demonstrated expertise in creating and managing metadata repositories.
• Advanced knowledge of real-time data streaming and analytics platforms.