Senior Data Engineer

INCEPTION; Abu Dhabi; Hybrid; AED 70,000 - 110,000; Full time

Job summary

An innovative firm is seeking a Senior Data Engineer to enhance its data infrastructure. This role involves developing scalable data pipelines, ensuring data quality, and collaborating with cross-functional teams to drive data-driven decision-making. The ideal candidate will have extensive experience in data engineering, proficiency in big data technologies, and a passion for working with data. Join a dynamic team that values collaboration and innovation in the AI space, and contribute to meaningful projects that shape the future of technology in the region.

Benefits

Healthcare

Education support

Leave benefits

Hybrid work policy

Career development opportunities

Qualifications

• 7+ years of experience in a Data Engineering role.
• Strong understanding of data warehousing concepts and ETL processes.
• Hands-on experience with big data technologies.

Responsibilities

• Design, construct, and maintain scalable data pipelines.
• Collaborate with ML engineers and stakeholders for data integration.
• Ensure compliance with data security and privacy policies.

Skills

Python

Java

Scala

Hadoop

Spark

Flink

API Development

Azure

Docker

Kubernetes

Education

Bachelor's degree in Computer Science

Equivalent work experience

Tools

Hadoop

Spark

Flink

MySQL

PostgreSQL

MongoDB

Cassandra

HBase

Redis

Kafka

Job description

Overview

As a Senior Data Engineer at Inception, you will be responsible for building and maintaining the infrastructure that supports data collection, processing, and storage. Reporting to the Lead Data Engineer or Principal Data Engineering, you will work closely with data scientists, analysts, and other stakeholders to ensure that data systems are reliable, scalable, and secure. Your work will be crucial in enabling data-driven decision-making across the organization.

The opportunity

The Data Engineer is a key technical role focused on developing and optimizing the company's data infrastructure. This role involves designing and implementing data pipelines, ensuring data quality, and collaborating with cross-functional teams to support various data initiatives. The ideal candidate will have strong technical skills in data engineering, experience with data architectures, and a passion for working with data.

Inception is the UAE's national-scale enabler in AI Research and Development. Partnering with Microsoft's AI SaaS, we offer domain-specific Agentic AI Orchestrator platforms utilizing reasoning agents for precise and cost-effective services. Our focus includes AI incubation, IP creation, applied AI R&D, and AI investment products. By creating models tailored to specific domains and languages, we ensure superior accuracy and efficiency. Collaborating with top universities and industry giants to drive significant advancements in AI technology within the region.

Responsibilities

As a Senior Data Engineer, you will be responsible for developing and maintaining data systems to support the company's strategic goals. Your role will encompass a range of activities focused on data pipeline development, data quality, and cross-functional collaboration.

• Data Pipeline Architecture and Development
  
  

Design, construct, install, test, and maintain highly scalable data pipelines with a focus on machine learning models and analytics.

• Data Integration
  
  

Work closely with data scientists, ML engineers, and stakeholders to ensure that data is accessible, consistent, and reliable for ongoing projects.

• API and Data Services
  
  

Develop and maintain APIs for data access and manipulation, and integrate with external data services as needed.

• Data Storage
  
  

Manage and optimize data storage solutions for both structured and unstructured data, where structured data includes relational databases and unstructured data includes Text Image Audio and Video, Search Engines like Elasticsearch and NoSQL databases, to support the requirements of machine learning models.

Understand data engines and structure to effectively design solutions for transactional, analytics, and search purposes.

• Data Quality and Governance
  
  

Implement processes to monitor data quality and ensure production data is always accurate and available for key stakeholders.

• Collaboration and Support
  
  

Collaborate with ML engineers to assist in data-related technical issues and provide architectural guidance and solutions.

• Security and Compliance
  
  

Ensure compliance with data security and privacy policies.

• Documentation
  
  

Maintain clear and up-to-date documentation including data dictionaries, metadata, and architectural diagrams.

Qualifications

Skills and attributes for success

• Bachelor's degree in Computer Science, Engineering, Mathematics, or a related field; or equivalent work experience.
• 7+ years of experience in a Data Engineering role.
• Programming languages like Python, Java, and Scala and managing huge scale data potentially Terabyte to Petabyte.
• Hands-on experience with big data technologies like Hadoop, Spark and Flink.
• Familiarity with machine learning frameworks such as TensorFlow, PyTorch, or similar.
• Strong understanding of data warehousing concepts, ETL processes, and data modeling.
• Experience with API development and integration with data services.
• Experience with cloud platforms like Azure.
• Knowledge in DevOps, CI/CD methods, and containerization technologies like Docker or Kubernetes.
• Experience with real-time / streaming data processing.
  
  

Technical stack

• Programming Languages: Python, Java, Scala, SQL, Bash
• Big Data Technologies: Hadoop, Spark, Flink
• Unstructured Data: Text, Image, Audio & Video
• Databases: MySQL, PostgreSQL, MongoDB, Cassandra, HBase, Redis
• Cloud Platforms: Azure
• API Development: RESTful APIs, GraphQL, OpenAPI
• Data Services: Kafka, RabbitMQ
• Containers: Docker, Kubernetes
  
  

What we look for

If you are a performance-driven, inquisitive mind with the agility to adapt to ambiguity, you will fit right in. You should be eager to explore opportunities to build meaningful collaborations with stakeholders and aspire to create unique customer-centric solutions. Bias for action and a passion to conquer new frontiers in the AI space is at the heart of the Inception community.

What working at Inception offers

Culture: An open, diverse and inclusive environment with a global vision that encourages personal growth and focuses on ground-breaking, industry-first innovations.

Career: Outstanding learning, development & growth opportunities via structured training programs and innovative, high-tech projects.

Work-Life: A hybrid work policy to strike the perfect balance between office and home.

Rewards: A competitive remuneration package with a host of perks including healthcare, education support, leave benefits and more.

If you can confidently demonstrate that you meet the criteria above, please contact us as soon as possible.

626