Databricks Platform Architect — AWS, Delta Lake & AI/ML

Shakti Solutions; Chicago (IL); On-site; USD 140,000 - 180,000; Full time

Job summary

Shakti Solutions seeks a Databricks architect to design and implement the end-to-end Databricks platform lifecycle on AWS. You will define Dev/SIT/UAT/PROD environments, implement governance with Unity Catalog, and guide engineering teams on best practices.

Responsibilities include architecture across AWS/Azure/GCP, with Delta Lake, PySpark, Python, Genie, and Databricks Apps expertise to enable AI/ML workflows.

Qualifications

• Design and implement end-to-end Databricks platform lifecycle on AWS.
• Hands-on cloud architecture across AWS, Azure, or GCP.
• Translate business needs into technical blueprints and run POCs.
• Experience with Delta Lake, Unity Catalog, DBX notebooks, Python, PySpark, Genie.

Responsibilities

• Design and implement Dev/SIT/UAT/PROD environments for Databricks.
• Implement data access controls, lineage, and auditing via Unity Catalog.
• Lead engineering teams with best practices and architectural guidance.

Skills

AWS

Spark

Delta Lake

SQL

Cluster management

Python

PySpark

Unity Catalog

Genie

Databricks Apps

AI/ML Enablement

Tools

Databricks

DBX notebooks

Genie

Databricks Apps

Job description

Shakti Solutions seeks a Databricks architect to design and implement the end-to-end Databricks platform lifecycle on AWS. You will define Dev/SIT/UAT/PROD environments, implement governance with Unity Catalog, and guide engineering teams on best practices.

Responsibilities include architecture across AWS/Azure/GCP, with Delta Lake, PySpark, Python, Genie, and Databricks Apps expertise to enable AI/ML workflows.