Data Engineer

Confidential Jobs; United Arab Emirates; On-site; AED 300,000 - 520,000; Full time

Job summary

Confidential Jobs in the United Arab Emirates is seeking a senior data platform engineer to design, develop, and optimize data platforms that support our analytics goals. You will implement scalable architectures using Databricks and drive automation across ingestion, storage, and processing.

You will lead data engineering pipelines, collaborate with data scientists and BI teams, ensure performance, security, and governance, and mentor a project team to deliver large-scale data solutions on time

Qualifications

• 7+ years of experience in data engineering, data platform development, or a related field.
• Strong knowledge of data modelling, data warehousing, and ETL/ELT processes.
• Certifications in Databricks and relevant big data technologies are required.

Responsibilities

• Implement scalable, high-performance data platform architecture using Databricks.
• Lead development of data engineering pipelines, standards, and tools.
• Collaborate with stakeholders to align platform initiatives with business goals.
• Ensure platforms meet security, compliance, and governance requirements.
• Develop and optimize data storage, ingestion, and processing for batch and real-time data.

Skills

Leadership

Communication

Problem-solving

Teamwork

Presentation

Education

Bachelor's degree

Databricks certifications

Tools

Databricks

Job description

This role is responsible for the design, development, and optimization of data platforms that support the organization's strategic data & analytics goals. This role will work on the implementation of data platform solutions, ensuring scalability, performance, and reliability. This role requires deep expertise in data platform engineering, a strong understanding of data architecture, and the ability to drive innovation and promote automation of work.

Key Responsibilities:

• Implement scalable, high-performance data platform architecture and technologies using Databricks.
• Lead the development of data engineering pipelines, standards, practices, and tools.
• Provide technical expertise for data platform initiatives.
• Collaborate with stakeholders (data scientists, business analysts, and others) to align data platform implementation with business plan and objectives.
• Work on and oversee the development and deployment of data platform solutions (acquisition, storage, transformation, query, analysis, consumption), ensuring they meet performance, security, and compliance requirements.
• Deliver industry best practices data orchestration to minimize data pipeline downtime and efficient use of resources. Optimize data ingestion, storage, retrieval, and processing technology and operations with a focus to improve performance and reduce costs.
• Develop and manage data warehousing solutions, including data models, schema design, and database performance optimization.
• Develop processes for data integration, ETL/ELT processes, and data quality controls within data pipelines. Ensure adherence and compliance to data governance policies.
• Ensure seamless integration of data platforms with existing systems and data sources (including big data), supporting both batch and real-time data ingestion and processing. Implement and maintain data storage structures (e.g., data lake, lake house, ODS etc.), in addition to data warehousing, to support data analysis, reporting, and business intelligence needs.
• Work with cross-functional business and data management teams, including data science, business intelligence, and technology teams, to ensure platform solutions meet diverse needs.

The responsibilities and duties outlined above are not exhaustive and may evolve over time. The role may require additional tasks and responsibilities as assigned by the line manager or higher authorities, in alignment with organizational needs.

What We're Looking For

Education:

• Educated with at least bachelor's degree
• Proficient in English
• Proficient in Arabic is a plus
• Excellent communication and presentation skills for all levels
• Strong teamwork skills
• Certifications in Databricks, and relevant big data and data integration technologies
• Proficiency in Vietnamese, Russian, Turkish, or Central Asian languages is a plus

Experience:

• 7+ years of experience in data engineering, data platform development, or a related field
• Extensive experience with big data technologies and data platforms
• Strong understanding of data modelling, data warehousing, and ETL/ELT processes
• Proven experience in designing and optimizing large-scale data solutions
• Experience in project management
• Excellent leadership, communication, and problem-solving skills
• Able to lead a project team
• Able to deal with varied tasks and to work under pressure to meet deadlines and target
• Able to work effectively and responsibly without close supervision
• Able to foster a culture of collaboration, innovation transparency, and accountability
• Able to execute clear strategies and corporate requirements
• Able to articulate ideas to both technical and non-technical audiences
• Professional demeanour and presentation, leads by example
• Demonstrates ability to manage change and foster teamwork

Why Join Us

• Opportunities for continuous learning, certifications, and professional development.
• Dedicated to building a diverse and inclusive workplace where everyone feels valued and empowered to be their authentic selves.
• Committed to building a stable, forward-thinking organization where innovation thrives.