Data And Ai Architect

EPAM Systems Inc

Melbourne

On-site

AUD 180,000 - 280,000

Full time

10 days ago

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Great Place To Work 2024
Top 100 Best Places to Work 2023
Mentoring & Certification support

Job summary

EPAM Systems Inc in Melbourne seeks a Data and AI Architect to design a shared data platform on Databricks for IoT and time-series solutions. You will shape secure ingestion, modeling, governance and retrieval patterns to enable scalable AI-ready data products.

You will collaborate with engineering, product and security teams, translate complex requirements into reusable designs, and deliver reference architectures, templates and decision records that guide implementation across multiple teams.

Qualifications

  • Databricks architecture experience in production environments
  • Data modeling expertise including event, time-series, dimensional and canonical modeling patterns
  • Ingestion pipeline experience across batch and streaming patterns
  • IoT and time-series data experience including scale, retention and query optimization
  • Delta Lake expertise including table design, performance tuning and lifecycle management
  • Databricks Unity Catalog experience covering governance, lineage and fine-grained access control
  • Apache Spark expertise for large-scale processing and optimization
  • Data governance and security experience including policy design and secure-by-default architecture
  • Nice to have experience with Vector Search and embedding lifecycles
  • RAG tuning considerations and retrieval quality awareness
  • Experience enabling shared data products across teams

Responsibilities

  • Design shared data and AI architecture on Databricks across Agentic AI solutions
  • Define data modeling standards for IoT and time-series domains aligned to analytics and RAG needs
  • Architect ingestion and processing pipelines using Spark and Delta Lake
  • Establish governance, lineage and access controls with Unity Catalog
  • Define secure data access patterns for internal teams and downstream consumption
  • Design RAG retrieval patterns including indexing strategy and datasets for Vector Search
  • Produce architecture decision records, reference designs and reusable templates
  • Partner with stakeholders to align scope, manage risks and drive implementation quality

Skills

Databricks architecture
Data modeling
Ingestion pipelines
IoT and time-series
Delta Lake
Unity Catalog
Apache Spark
Data governance and security
Vector Search
RAG tuning considerations
Shared data products

Tools

Databricks
Apache Spark
Delta Lake
Unity Catalog
Vector Search

Job description

We are seeking a Data and AI Architect to design shared data architecture for Agentic AI solutions on Databricks. You will shape secure ingestion, modeling and governance for IoT and time-series data and enable RAG retrieval patterns using Delta Lake, Unity Catalog, Spark and Vector Search. You simplify complexity and guide teams toward scalable, reusable designs.

You will define the shared data architecture for Agentic AI solutions built on Databricks. Your focus is creating a secure, scalable and reusable foundation for ingestion, modeling, governance and retrieval patterns that enable AI-ready data products.

You bring a pragmatic systems mindset, and you can translate complex data and AI requirements into clear architecture decisions. You collaborate across engineering, product and security teams, challenge assumptions constructively and manage risks early.

Responsibilities
  • Design shared data and AI architecture on Databricks across Agentic AI solutions
  • Define data modeling standards for IoT and time-series domains aligned to analytics and RAG needs
  • Architect ingestion and processing pipelines using Spark and Delta Lake
  • Establish governance, lineage and access controls with Unity Catalog
  • Define secure data access patterns for internal teams and downstream consumption
  • Design RAG retrieval patterns including indexing strategy and datasets for Vector Search
  • Produce architecture decision records, reference designs and reusable templates
  • Partner with stakeholders to align scope, manage risks and drive implementation quality
Requirements
  • Databricks architecture experience in production environments
  • Data modeling expertise including event, time-series, dimensional and canonical modeling patterns
  • Ingestion pipeline experience across batch and streaming patterns
  • IoT and time-series data experience including scale, retention and query optimization
  • Delta Lake expertise including table design, performance tuning and lifecycle management
  • Databricks Unity Catalog experience covering governance, lineage and fine-grained access control
  • Apache Spark expertise for large-scale processing and optimization
  • Data governance and security experience including policy design and secure-by-default architecture
Nice to have
  • Experience implementing Vector Search and managing embedding lifecycle
  • Familiarity with retrieval quality, evaluation signals and RAG tuning considerations
  • Experience enabling shared data products across multiple teams
We offer
  • By choosing EPAM, you're getting a job at Great Place To Work-Certified™ in 2024, Glassdoor’s Top 100 Best Places to Work in 2023 & one of Most Loved Workplace, as recognized by Newsweek, 2021 - 2023.
  • Employee ideas are the main driver of our business. We have a very supportive environment where your voice matters.
  • You will be challenged while working side-by-side with the best talent globally. We work with top-notch technologies, constantly seeking new industry trends and best practices.
  • We offer a transparent career path and an individual roadmap to engineer your future & accelerate your journey.
  • At EPAM, you can find vast opportunities for self-development: online courses and libraries, mentoring programs, partial grants of certification, and experience exchange with colleagues around the world. You will learn, contribute, and grow with us.
Life at EPAM
  • We are a leading global provider of digital platform engineering and development services that has rapidly grew in Australia since 2013.
  • We adopt a global growth strategy, thinking and acting like start-ups, working in multi-disciplinary teams, and delivering results. With a multinational talented team, we have been delivering innovative solutions for cloud migration, cyber security, digital transformation, etc.
  • You will work in an open and friendly work environment where your professional skills and experience can stand out.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data and AI Architect
Data and AI Architect

EPAM Systems Inc • Melbourne

On-site
AUD 140,000 - 210,000
Great Place To Work 2024 recognition
Global talent collaboration
Online courses and mentoring programs
Databricks Data & AI Architect for IoT & RAG
Databricks Data & AI Architect for IoT & RAG

EPAM Systems Inc • Melbourne

On-site
AUD 140,000 - 210,000
Great Place To Work 2024 recognition
Global talent collaboration
Online courses and mentoring programs
Senior Data Engineer (Databricks, Data Modeling)
Senior Data Engineer (Databricks, Data Modeling)

EPAM Systems Inc • Sydney

On-site
AUD 140,000 - 190,000
Great Place To Work-Certified
Glassdoor Top 100 Best Places to Work
Most Loved Workplace (Newsweek 2021-23
+1
Senior Manager, Data Analytics and AI Consulting
Senior Manager, Data Analytics and AI Consulting

EPAM Systems Inc • Sydney

On-site
AUD 180,000 - 240,000
Great Place To Work-Certified 2024
Mentoring programs
Access to online courses & libraries
Databricks Data & AI Architect for Scalable AI Foundations
Databricks Data & AI Architect for Scalable AI Foundations

EPAM Systems • Australia

On-site
AUD 150,000 - 210,000
Senior Data Engineer (Databricks, Data Modeling)
Senior Data Engineer (Databricks, Data Modeling)

EPAM Systems • Sydney

On-site
AUD 150,000 - 190,000
Lead Data & AI Engineer
Lead Data & AI Engineer

EPAM Systems • Australia

On-site
AUD 150,000 - 190,000
Data & Ai Solutions Architect
Data & Ai Solutions Architect

Neon • Melbourne

On-site
AUD 120,000 - 180,000
Databricks Data & AI Architect for IoT & RAG
Databricks Data & AI Architect for IoT & RAG

EPAM Systems Inc • Melbourne

On-site
AUD 180,000 - 280,000
Great Place To Work 2024
Top 100 Best Places to Work 2023
Mentoring & Certification support
Sr. Solutions Architect
Sr. Solutions Architect

WinsAbove • Sydney

On-site
AUD 180,000 - 260,000
Top locations