Multimodal Data Scientist, Pretraining Specialist

Hark; San Jose (CA); In loco; USD 180.000 - 450.000; Tempo pieno

Descrizione del lavoro

Hark is building the Omni team to advance multimodal AI, handling text, image, and video data for pretraining. The role focuses on creating scalable data pipelines, filtering, and synthetic supervision to improve model understanding across modalities.

You will design datasets, measure data choices with controlled experiments, and work with engineers to scale data processing for production-ready training systems.

Competenze

• Experience building training data pipelines for language or vision-language models.
• Experience designing controlled data experiments and measuring transfer across training scales.
• Strong Python engineering skills and familiarity with distributed processing frameworks (e.g., Spark, Ray, Beam).
• Understanding of data quality issues such as duplication and benchmark contamination.
• Proven record of improving training data or model performance through datasets, systems, open-source contributions, or production work.

Mansioni

• Develop methods to select, filter, deduplicate, and prepare text, code, image, and video datasets at scale.
• Build paired and interleaved datasets, including image-text documents and video-text data, preserving document structure and content alignment.
• Create quality classifiers and sampling methods; measure impact with controlled training experiments.
• Develop synthetic captioning and other supervision methods for image and video data; assess accuracy, coverage, and biases.
• Design data mixtures and curricula across sources, domains, and modalities; evaluate sampling and resolution effects on quality and efficiency.
• Monitor corpus quality and provenance; track dataset versions and contend with benchmark contamination in collaboration with researchers.
• Collaborate with infrastructure engineers to scale data processing and deliver reproducible datasets.

Conoscenze

Python programming

Data processing concepts

Experiment design

Data quality awareness

Strumenti

Spark

Ray

Beam

Descrizione del lavoro

Hark is building the Omni team to advance multimodal AI, handling text, image, and video data for pretraining. The role focuses on creating scalable data pipelines, filtering, and synthetic supervision to improve model understanding across modalities.

You will design datasets, measure data choices with controlled experiments, and work with engineers to scale data processing for production-ready training systems.