Data Scientist

Tawteen; Muscat; On-site; OMR 12,000 - 18,000; Full time

Job summary

Tawteen is seeking a data science professional to design evaluation methods and build test data sets for evaluating AI models. You will run evaluations, benchmark models, and analyze errors in safety and privacy filters across multilingual and adversarial prompts.

The role involves building predictive and classification models on tabular/time-series data and integrating evaluation tests into CI pipelines. Skills in Python, pandas, NumPy, scikit-learn, SQL, and Git are essential, along with

Qualifications

• Degree in data science, statistics, computer science or a related field.
• Proficiency in Python with pandas, NumPy and scikit-learn.
• Knowledge of statistics: precision, recall, confidence intervals, hypothesis testing.
• SQL and Git proficiency.

Responsibilities

• Design evaluation methods and build test data sets including multilingual and adversarial prompts.
• Run evaluations and score results with automated metrics and LLM-as-judge.
• Benchmark models on quality, latency and cost and recommend which to use.
• Analyse errors including false positives and false negatives of safety and privacy filters.
• Build and validate predictive and classification models on tabular, time-series and signal data.
• Add evaluation tests to CI pipelines to prevent regressions.
• Analyse usage cost and quality data and build reports and dashboards.

Skills

Python

Pandas

NumPy

scikit-learn

Statistics

Precision

Recall

Confidence intervals

Hypothesis testing

SQL

Git

Education

Data Science degree

Tools

PyTorch

TensorFlow

MLflow

Weights & Biases

Langfuse

promptfoo

Job description

Role summary

Measure how well AI models and features perform build predictive models where classical machine learning fits better than an LLM and turn usage data into decisions

Responsibilities

• Design evaluation methods and build test data sets expected answers edge cases multilingual and adversarial prompts
• Run evaluations and score the results with automated metrics and LLM-as-judge calibrated against human review
• Benchmark models on quality latency and cost and recommend which to use
• Analyse errors including false positives and false negatives of safety and privacy filters
• Build and validate predictive and classification models on tabular time-series and signal data
• Add evaluation tests to CI pipelines so a regression blocks the release
• Analyse usage cost and quality data and build reports and dashboards

Technical requirements

• Degree in data science statistics computer science or a related field
• Python with pandas NumPy and scikit-learn
• Statistics precision recall confidence intervals hypothesis testing
• SQL
• Git

Nice to have

• More experience is a plus professional years in data science or applied machine learning and models used in production
• Experience evaluating NLP or LLM systems
• PyTorch or TensorFlow
• Time-series or signal-processing models
• LLM evaluation tools Langfuse Ragas promptfoo
• Experiment tracking with MLflow or Weights amp Biases
• Experience mentoring other data scientists
• A second language or a background in healthcare or another regulated domain

Degree in data science, statistics, computer science or a related field Python with pandas, NumPy and scikit-learn Statistics: precision, recall, confidence intervals, hypothesis testing SQL Git