Databricks Architect: Lakehouse & Governance(Travel 15d/mo)

Intuitive.ai; Charlotte (NC); On-site; USD 130,000 - 190,000; Full time

Job summary

Intuitive.ai is seeking an experienced Databricks Architect to design, develop, and optimize scalable data engineering solutions on the Databricks Lakehouse platform. You will leverage Unity Catalog, Spark, PySpark, SQL, and Delta Lake to build robust pipelines with strong data governance across clouds.

The role covers end-to-end pipeline development, production deployments, and CI/CD, with collaboration across data teams. Travel approximately 15 days per month across the USA is required.

Qualifications

• 5+ years of experience in Data Engineering.
• Hands-on experience with Databricks.
• Strong experience with Unity Catalog.
• Advanced PySpark and Apache Spark skills.
• Strong SQL development and query optimization experience.
• Hands-on experience with Delta Lake / Delta Tables.
• Experience building enterprise-grade ETL/ELT data pipelines.
• Experience with data governance, security, access management, and data lineage.
• Experience with at least one major cloud platform: Azure, AWS, or Google Cloud Platform.

Responsibilities

• Design and develop scalable data pipelines using Databricks, Apache Spark, PySpark, and SQL.
• Build and optimize ETL/ELT pipelines with Delta Lake and Delta Tables.
• Implement and manage Unity Catalog across Databricks environments.
• Configure and manage Catalogs, Schemas, Tables, Views, External Locations, and Storage Credentials.
• Implement data access controls, RBAC, permissions, and governance policies using Unity Catalog.
• Support data lineage, auditing, data discovery, and governance requirements.
• Design and maintain secure data lakehouse architectures across cloud environments.
• Develop Databricks Jobs, Workflows, notebooks, and production‑grade data pipelines.
• Optimize Spark jobs, SQL queries, Delta tables, and overall pipeline performance.
• Implement data quality, validation, monitoring, and error‑handling frameworks.
• Work with cloud storage platforms such as Azure Data Lake Storage, Amazon S3, or Google Cloud Storage.
• Integrate Databricks with enterprise data platforms, databases, APIs, and downstream applications.
• Implement CI/CD processes for Databricks development and deployment.
• Use Git, Terraform, or Databricks Asset Bundles for infrastructure and deployment automation.
• Troubleshoot production data pipelines and resolve performance and data‑quality issues.
• Collaborate with Data Architects, Cloud Engineers, Data Scientists, Business Analysts, and application teams.

Skills

Databricks

PySpark

SQL

Data governance

Data modeling

Cloud platforms

Performance optimization

CI/CD

Tools

Databricks

Unity Catalog

Delta Lake

Delta Tables

Git

Terraform

Airflow

Azure Data Lake Storage

Amazon S3

Google Cloud Storage

Job description

Intuitive.ai is seeking an experienced Databricks Architect to design, develop, and optimize scalable data engineering solutions on the Databricks Lakehouse platform. You will leverage Unity Catalog, Spark, PySpark, SQL, and Delta Lake to build robust pipelines with strong data governance across clouds.

The role covers end-to-end pipeline development, production deployments, and CI/CD, with collaboration across data teams. Travel approximately 15 days per month across the USA is required.