AI Data Engineer

Michael Page; United Arab Emirates; On-site; AED 120,000 - 200,000; Full time

Job summary

A leading government entity seeks an experienced data engineer to design and maintain data pipelines and infrastructures for AI and ML applications. This role involves collaborating with AI specialists to ensure data quality and accessibility while contributing to national data-driven initiatives. Applicants should have a degree in a relevant field and substantial experience in data engineering, particularly for AI workloads.

Qualifications

• Minimum of 6+ years of experience in data engineering, especially in AI/ML.
• Experience with programming languages such as Python, Scala, or Java.
• Proficiency in big data technologies like Spark and Hadoop.

Responsibilities

• Design and build scalable data pipelines for AI models.
• Collaborate with AI Scientists to optimize data formats.
• Implement data governance best practices for sensitive data.

Skills

Data Engineering

Data Pipeline Optimization

Cloud Platforms

Programming (Python, Scala, Java)

Big Data Technologies

Education

Bachelor's Degree in Computer Science or related field

Tools

Azure Data Factory

AWS Glue

Google Cloud Dataflow

Job description

You will be instrumental in designing, building, and maintaining high-performance data pipelines and infrastructure specifically tailored for AI and ML applications. Your expertise will be critical in ensuring data quality, accessibility, and scalability for training, evaluating, and deploying advanced AI models, with a particular emphasis on the unique data requirements of LLMs and other complex AI techniques. Client Details A pivotal and leading government entity, dedicated to leveraging advanced technology and data to serve national interests and drive strategic objectives Description * Design, build, and optimise scalable, robust, and efficient data pipelines for ingesting, transforming, and preparing large and complex datasets specifically for AI/ML models, including structured and unstructured data sources. * Develop and manage data infrastructure components in cloud environments (e.g., Azure, AWS, GCP) ensuring security, compliance, and performance for AI workloads. * Implement data governance best practices, ensuring data quality, lineage, privacy, and security for sensitive government data used in AI applications. * Collaborate closely with AI Scientists and Machine Learning Engineers to understand their data needs, optimising data formats, storage, and access patterns for model training and inference, particularly for LLMs. * Develop and maintain data versioning, feature stores, and model-serving data layers to support the AI/ML lifecycle. * Troubleshoot and resolve data-related issues, ensuring data reliability and integrity for AI projects. Job Offer An exceptional opportunity to contribute to high-impact national initiatives within a leading government entity.

* Bachelor's degree or equivalent practical experience in Computer Science, Data Engineering, Software Engineering, or a related quantitative field. * Minimum of 6+ years of progressive experience in data engineering, with at least 3 years specifically focused on building data pipelines and infrastructure for AI/ML projects. * Demonstrable strong experience with data engineering concepts and tools for AI, including data preparation for techniques like LLMs, natural language processing, and deep learning. * Expert proficiency in at least one programming language commonly used in data engineering (e.g., Python, Scala, Java). * Strong experience with big data technologies (e.g., Spark, Hadoop, Flink) and distributed data processing frameworks. * Proven experience with cloud data platforms and services (e.g., Azure Data Factory, Azure Databricks, AWS Glue, Google Cloud Dataflow, BigQuery).