Data Engineer

Mohamed bin Zayed University of Artificial Intelligence; Sunnyvale (CA); In loco; USD 150.000 - 450.000; Tempo pieno

Vantaggi offerti da questo lavoro

Comprehensive medical, dental, and vis

Bonus

401K Plan

Generous paid time off

Paid Parental Leave

Employee Assistance Program

Life insurance and disability

Descrizione del lavoro

Mohamed bin Zayed University of Artificial Intelligence in Sunnyvale seeks a Data Engineer focused on NLP and large-scale data processing. You will rapidly gather, curate, and prepare high-quality datasets to support cutting-edge NLP research, delivering data through scalable pipelines built with Python and related technologies.

You will develop web crawlers, APIs, and automated workflows to improve data collection, refine outputs from large language models for training and benchmarking, and

Competenze

• Extensive experience in data engineering and automation using Python.
• Proven ability to design web crawling solutions and data pipelines.
• Strong knowledge of SQL, data structures, and distributed processing.

Mansioni

• Rapidly collect, curate, and preprocess datasets per NLPresearchers' specs.
• Develop and maintain web crawling solutions, APIs, and automated workflows.
• Refine LLM outputs to generate structured datasets for training.
• Implement scalable data pipelines for storage, processing, and distribution.
• Collaborate with researchers to ensure data quality and relevance.
• Document data collection methodologies and pipeline architecture.
• Participate in technical reviews to uphold data quality standards.
• Represent MBZUAI at industry forums showcasing data processing capabilities.

Conoscenze

Python

Data engineering

SQL

Cloud AWS

Spark

Kafka

Kubernetes

Data pipelines

Formazione

Bachelor's degree in Computer Science, Data Science, Engineering, or a related technical field

Master’s degree or PhD in Computer Science, Data Engineering, or related technical field

Strumenti

AWS

Spark

Kafka

Kubernetes

Descrizione del lavoro

About the Institute of Foundation Models

We are a dedicated research lab for building, understanding, using,and risk-managing foundation models. Our mandate is to advanceresearch, nurture the next generation of AI builders, and drivetransformative contributions to a knowledge-driven economy.

As part of our team, you’ll have the opportunity to work on thecore of cutting-edge foundation model training, alongsideworld-class researchers, data scientists, and engineers, tacklingthemost fundamental and impactful challenges in AIdevelopment. You will participate in the development ofgroundbreaking AI solutions that have the potential to reshapeentire industries. Strategic and innovative problem-solving skillswill be instrumental in establishing MBZUAI as a global hub forhigh-performance computing in deep learning, driving impactfuldiscoveries that inspire the next generation of AI pioneers.

The Role

As a Data Engineer specializing in Natural Language Processing(NLP) and large-scale data processing, you will quickly andeffectively gather, curate, and prepare high-quality datasets tosupport cutting-edge NLP research. Your role will be instrumentalin enabling researchers by delivering essential data throughefficient and scalable engineering practices, including webcrawling, LLM-generated content refinement, and robust datapipelines, primarily leveraging Python and relatedtechnologies.

Key Responsibilities

• Rapidly collect, curate, and preprocess datasets based ondetailed specifications provided by NLPresearchers,delivering datawithin tight timelines.
• Develop and maintain efficient web crawling solutions, APIs,and automated workflows to continuously improve data collectionprocesses.
• Refine and evaluate outputs from Large Language Models (LLMs)to generate structured datasets suitable for model training andbenchmarking.
• Implement scalable data pipelines, ensuring efficient dataprocessing, storage, retrieval, and distribution to researchteams.
• Collaborate closely with researchers and engineers to ensurecollected data meets specified quality and relevance criteria.
• Document data collection methodologies, datasetcharacteristics, and pipeline architecture clearly andeffectively.
• Engage with peer teams and participate in technical reviews touphold best practices and data quality standards.
• Represent MBZUAI at industry and research forums, showcasingtechnical capabilities in large-scale data processing and AI datainfrastructure.

Academic Qualifications

• Bachelor's degree in Computer Science, Data Science,Engineering, or a related technical field required
• Master’s degree or PhD degree or equivalent experience inComputer Science, Data Engineering, or related technical fieldspreferred.

Professional Experience - Required

• Extensive experience in data engineering, data processing, andautomation using Python.
• Demonstrated proficiency in designing and deploying webcrawling solutions, automated data extraction, and processingpipelines.
• Strong understanding of data structures, algorithms, databases,SQL, and performance optimization.
• Experience working with cloud infrastructure and distributeddata processing frameworks (e.g., AWS, Spark, Kafka,Kubernetes).
• Excellent problem-solving abilities, attention to detail, andthe capability to rapidly address technical challenges.
• Strong communication and collaboration skills withcross-functional teams.

Professional Experience - Preferred

• Proven track record of supporting NLP or AI research teams withrapid and reliable data delivery.
• Experience working with large language models, includingevaluation, efficient inference, and prompt engineering.
• Experience with refining outputs from large-scale AI models,such as LLM-generated data.
• Contributions to open-source projects, coding competitions, orhigh visibility in coding communities (e.g., GitHub, StackOverflow).
• Familiarity with the latest advancements in NLP data processingand large language model technologies.

$150,000 - $450,000 a year

Compensation

$150,000 – $450,000/year + discretionary performance bonus +benefits

The posted salary range represents the company’s good faithestimate of the base compensation for this position upon hire. Theactual base compensation offered may vary within this rangedepending on individual qualifications, including but not limitedto relevant skills, experience, education, certifications, andspecific business needs. In addition to base salary, totalcompensation includes an annual discretionary performance bonus andcomprehensive benefits.

Benefits Include

• Comprehensive medical, dental, and vision benefits
• Bonus
• 401K Plan
• Generous paid time off, sick leave and holidays
• Paid Parental Leave
• Employee Assistance Program
• Life insurance and disability

We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzingresumes, or assessing responses and identifying potentialinconsistencies or verification signals in application materialsbased on available information. These tools assist our recruitmentteam but do not replace human judgment. Final hiring decisions areultimately made by humans. If you would like more information abouthow your data is processed, please contact us.