AI Production Readiness & Reliability Engineer

Dicetek LLC; Abu Dhabi; Hybrid; AED 180,000 - 300,000; Full time

Job summary

Dicetek LLC is seeking an AI Quality & Reliability Engineer to own production readiness of cloud-based AI solutions. This hybrid role combines automated testing, AI model evaluation, and SRE for Azure/AWS infrastructure in a banking context.

You will build frameworks that validate AI outputs and ensure resilient, compliant operations, with a strong focus on monitoring, security testing, and cost-conscious testing practices.

Qualifications

• Automation stack with Python for AI testing and framework automation (PyTest, Selenium, or Robot Framework).
• Cloud infrastructure experience with Azure or AWS focusing on networking, scaling, and serverless reliability.
• Understanding of AI/ML evaluation methods such as prompt engineering and LLM benchmarks.
• Monitoring and observability proficiency with Grafana/Prometheus or native cloud tools for real-time dashboards.
• FinOps awareness to identify expensive tests or inefficient cloud usage during testing.

Responsibilities

• AI Quality Automation: design and run automated testing for AI services, incl. model evaluation and content safety latency.
• Resiliency Engineering (SRE): implement chaos engineering and load testing, ensure high availability with automated recovery scripts.
• Automated Regression: build CI/CD-integrated test suites validating app logic and IaC state.
• Observability & SLIs: define and monitor SLIs/SLOs; set up alerts in Azure Monitor or CloudWatch to catch degradation.
• Security & Compliance Testing: automate security scans and compliance checks for banking data residency and privacy requirements.

Skills

Python

PyTest

Selenium

Robot Framework

Azure

AWS

Terraform

Grafana

Prometheus

CI/CD

Tools

GitHub Actions

Terraform

K6

JMeter

Grafana

Prometheus

Job description

Dicetek LLC is seeking an AI Quality & Reliability Engineer to own production readiness of cloud-based AI solutions. This hybrid role combines automated testing, AI model evaluation, and SRE for Azure/AWS infrastructure in a banking context.

You will build frameworks that validate AI outputs and ensure resilient, compliant operations, with a strong focus on monitoring, security testing, and cost-conscious testing practices.