Solution Architect – Databricks

Allwyn Corporation; United States; On-site; USD 150,000 - 190,000; Full time

Job summary

Allwyn Corporation is seeking an experienced Solutions Architect to design cloud-native data platforms for real-time analytics and enterprise processing. You will define architectures, lead data engineering pipelines on AWS and Databricks, and collaborate with stakeholders and engineering teams.

The role focuses on streaming and batch processing with Kafka, Lakehouse patterns, and governance. You will mentor engineers, drive best practices, support cloud migration, and evaluate emerging

Qualifications

• 8+ years of experience in software, data engineering, or solution architecture.
• Strong experience designing enterprise data platforms on AWS.
• Hands-on expertise with Databricks and the Lakehouse architecture.
• Experience implementing Apache Kafka for real-time event streaming.
• Strong background in Data Engineering, including ETL/ELT pipeline development.
• Experience building scalable real-time and streaming data processing solutions.
• Strong understanding of distributed systems and cloud architecture principles.
• Experience with SQL, Python, or Scala.
• Knowledge of CI/CD, Infrastructure as Code, and cloud-native development practices.
• Excellent communication and stakeholder management skills.
• Databricks Certified Data Engineer Professional

Responsibilities

• Design end-to-end cloud-native data architectures using AWS and Databricks.
• Architect scalable batch and real-time data processing solutions using Apache Kafka and streaming technologies.
• Lead the design and implementation of enterprise data engineering pipelines and data platforms.
• Define architecture standards, best practices, and governance for data integration and analytics solutions.
• Collaborate with business stakeholders to translate business requirements into scalable technical solutions.
• Provide technical leadership and architectural guidance to development teams throughout the SDLC.
• Optimize data pipelines for performance, scalability, security, and cost efficiency.Design data models and integration strategies for structured and semi-structured data.
• Ensure solutions adhere to enterprise security, compliance, and reliability standards.
• Evaluate emerging technologies and recommend innovative solutions to improve business capabilities.
• Support cloud migration and modernization initiatives.
• Mentor engineers and conduct architecture and design reviews.

Skills

Distributed systems

Stakeholder management

Python

SQL

Scala

Education

Databricks Certified Data Engineer Professional

Tools

Databricks

Kafka

AWS

Delta Lake

Spark Structured Streaming

Terraform

Kubernetes

Docker

Job description

We are seeking an experienced Solutions Architect to design and deliver scalable, cloud-native data solutions that support real-time analytics and enterprise data processing. The ideal candidate will have deep expertise in Databricks, AWS, Kafka, and modern Data Engineering practices, with experience building high-performance streaming and batch data platforms. Exposure to AI/ML solutions is a strong plus.

The Solution Architect will work closely with business stakeholders, engineering teams, and technical leadership to define solution architecture, establish best practices, and drive successful implementation of enterprise data initiatives.

Key Responsibilities

• Design end-to-end cloud-native data architectures using AWS and Databricks.
• Architect scalable batch and real-time data processing solutions using Apache Kafka and streaming technologies.
• Lead the design and implementation of enterprise data engineering pipelines and data platforms.
• Define architecture standards, best practices, and governance for data integration and analytics solutions.
• Collaborate with business stakeholders to translate business requirements into scalable technical solutions.
• Provide technical leadership and architectural guidance to development teams throughout the SDLC.
• Optimize data pipelines for performance, scalability, security, and cost efficiency.Design data models and integration strategies for structured and semi-structured data.
• Ensure solutions adhere to enterprise security, compliance, and reliability standards.
• Evaluate emerging technologies and recommend innovative solutions to improve business capabilities.
• Support cloud migration and modernization initiatives.
• Mentor engineers and conduct architecture and design reviews.

Required Qualifications

• 8+ years of experience in software, data engineering, or solution architecture.
• Strong experience designing enterprise data platforms on AWS.
• Hands-on expertise with Databricks and the Lakehouse architecture.
• Experience implementing Apache Kafka for real-time event streaming.
• Strong background in Data Engineering, including ETL/ELT pipeline development.
• Experience building scalable real-time and streaming data processing solutions.
• Strong understanding of distributed systems and cloud architecture principles.
• Experience with SQL, Python, or Scala.
• Knowledge of CI/CD, Infrastructure as Code, and cloud-native development practices.
• Excellent communication and stakeholder management skills.
• Databricks Certified Data Engineer Professional

Preferred Qualifications

• Experience with AI/ML platforms or deploying machine learning solutions into production.
• Experience with Delta Lake, Spark Structured Streaming, or similar technologies.
• AWS Certification (Solutions Architect or Data Analytics) is preferred.
• Experience with Terraform, Kubernetes, or Docker is a plus.