Senior Cloud & Data Architect – AWS & Databricks Expert

Brillio; St. Louis (MO); On-site; USD 180,000 - 260,000; Full time

Job summary

Brillio is seeking a senior cloud architect in St. Louis to design scalable AWS and Databricks solutions. You will lead end-to-end data pipelines, implement medallion architectures, and optimize costs with modern tooling.

The role demands extensive experience with PySpark, SQL, and Terraform in cloud environments. You will collaborate across teams, shape governance and security, and contribute to high-impact data projects for Fortune 1000 clients.

Qualifications

• Experience in cloud architecture with hands-on AWS and Databricks.
• At least 15 years of experience in cloud architecture and design.
• Proven ability to design scalable, secure data platforms and ETL pipelines.

Responsibilities

• Design and architect scalable AWS and Databricks solutions with related services.
• Develop and maintain ETL/ELT pipelines using PySpark and Python in Databricks.
• Implement and optimize medallion architecture (bronze-silver-gold) and Spark tuning.
• Provision infrastructure with Terraform for Databricks, storage, networking, IAM.
• Write SQL for data transformation, validation, and analytics with governance.
• Build CI/CD pipelines using Jenkins and GitHub Actions for notebooks and Terraform.
• Integrate data from diverse sources into cloud storage and processing layers.
• Optimize costs with auto-scaling, spot instances, and efficient scheduling.
• Monitor pipelines, troubleshoot, and implement observability with Databricks or cloud tools.

Skills

AWS Networking

AWS Kinesis

AWS Elastic Cache

AWS Redshift

AWS PaaS (Lambda, S3, Glue)

AWS EKS

AWS Cognito

Databricks

Delta Lake

Unity Catalog

Delta Live Tables

Python

PySpark

SQL

Terraform

Jenkins

GitHub Actions

Education

Bachelor's degree in Computer Science or IT

AWS Certified Solutions Architect - Professional

Databricks Certified Data Engineer Professional or Apache Spark certification

Tools

Terraform

Jenkins

GitHub Actions

Databricks

Job description

Brillio is seeking a senior cloud architect in St. Louis to design scalable AWS and Databricks solutions. You will lead end-to-end data pipelines, implement medallion architectures, and optimize costs with modern tooling.

The role demands extensive experience with PySpark, SQL, and Terraform in cloud environments. You will collaborate across teams, shape governance and security, and contribute to high-impact data projects for Fortune 1000 clients.