Databricks Data & AI Architect for Scalable AI Solutions

EPAM Systems; Sydney; On-site; AUD 180,000 - 240,000; Full time

Benefits offered by this job

Great Place To Work

Self-development programs

Mentoring & certifications

Job summary

EPAM Systems is seeking a Data and AI Architect to design shared data architecture for Agentic AI solutions on Databricks. You will shape secure ingestion, modeling and governance for IoT and time-series data and enable RAG retrieval patterns using Delta Lake, Unity Catalog, Spark and Vector Search.

You will collaborate across engineering, product and security teams to translate complex data and AI requirements into scalable, reusable designs and drive implementation quality across the

Qualifications

• Experience designing data architectures for IoT/time-series data
• Proven ability to model data for analytics and RAG patterns
• Experience with ingestion pipelines (batch and streaming)
• Knowledge of secure-by-default architectures and governance
• Experience with large-scale data platforms such as Databricks

Responsibilities

• Design shared data and AI architecture on Databricks across Agentic AI solutions
• Define data modeling standards for IoT and time-series domains
• Architect ingestion and processing pipelines using Spark and Delta Lake
• Establish governance, lineage and access controls with Unity Catalog
• Define secure data access patterns for internal teams and downstream consumption
• Design RAG retrieval patterns including indexing strategy and datasets for Vector Search
• Produce architecture decision records, reference designs and reusable templates
• Partner with stakeholders to align scope, manage risks and drive implementation quality

Skills

Databricks architecture

Data modeling

Ingestion pipelines

IoT data

Time-series data

Delta Lake

Unity Catalog

Apache Spark

Data governance & security

Tools

Delta Lake

Unity Catalog

Spark

Job description

EPAM Systems is seeking a Data and AI Architect to design shared data architecture for Agentic AI solutions on Databricks. You will shape secure ingestion, modeling and governance for IoT and time-series data and enable RAG retrieval patterns using Delta Lake, Unity Catalog, Spark and Vector Search.

You will collaborate across engineering, product and security teams to translate complex data and AI requirements into scalable, reusable designs and drive implementation quality across the