Databricks Data & AI Architect for Scalable AI Foundations

EPAM Systems; Australia; On-site; AUD 150,000 - 210,000; Full time

Job summary

EPAM Systems is seeking a Data and AI Architect in Australia to design shared data architectures for Agentic AI solutions on Databricks. You will shape secure ingestion, modeling and governance for IoT and time-series data, enabling RAG retrieval patterns with Delta Lake, Unity Catalog, Spark and Vector Search.

You will collaborate across engineering, product and security teams to translate complex data and AI requirements into clear architecture decisions and reusable templates.

Qualifications

• Experience designing Databricks-based data and AI architectures in production.
• Proficiency in data modeling for IoT and time-series domains.
• Experience building ingestion pipelines for batch and streaming data.
• Knowledge of governance, lineage and access controls.
• Experience with RAG retrieval patterns and Vector Search considerations.
• Ability to produce reference designs and reusable templates.

Responsibilities

• Design shared data and AI architecture on Databricks across Agentic AI solutions.
• Define data modeling standards for IoT and time-series domains.
• Architect ingestion and processing pipelines using Spark and Delta Lake.
• Establish governance, lineage and access controls with Unity Catalog.
• Define secure data access patterns for internal teams and downstream consumption.
• Design RAG retrieval patterns including indexing strategy and datasets for Vector Search.
• Produce architecture decision records, reference designs and reusable templates.
• Partner with stakeholders to align scope, manage risks and drive implementation quality.

Skills

Databricks architecture

Data modeling

Ingestion pipelines

IoT data

Time-series data

Delta Lake

Unity Catalog

Apache Spark

Security & governance

Tools

Databricks

Delta Lake

Unity Catalog

Apache Spark

Vector Search

Job description

EPAM Systems is seeking a Data and AI Architect in Australia to design shared data architectures for Agentic AI solutions on Databricks. You will shape secure ingestion, modeling and governance for IoT and time-series data, enabling RAG retrieval patterns with Delta Lake, Unity Catalog, Spark and Vector Search.

You will collaborate across engineering, product and security teams to translate complex data and AI requirements into clear architecture decisions and reusable templates.