Solution Architect Data bricks

Hadiamondstar Software Solutions LLC; Washington (District of Columbia); On-site; USD 150,000 - 190,000; Full time

Job summary

Hadiamondstar Software Solutions LLC in Washington, DC is seeking an experienced Enterprise Data Architect to design end-to-end cloud-native data architectures on AWS and Databricks, and to lead scalable batch and real-time processing using Apache Kafka.

You will mentor engineers, define governance standards, collaborate with stakeholders to translate business needs into scalable solutions, and ensure security and cost-efficient platforms across the SDLC.

Qualifications

• 8+ years of experience in software, data engineering, or solution architecture.
• Strong experience designing enterprise data platforms on AWS.
• Hands-on expertise with Databricks and the Lakehouse architecture.
• Experience implementing Apache Kafka for real-time event streaming.
• Strong background in Data Engineering, including ETL/ELT pipeline development.
• Experience building scalable real-time and streaming data processing solutions.
• Strong understanding of distributed systems and cloud architecture principles.
• Experience with SQL, Python, or Scala.
• Knowledge of CI/CD, Infrastructure as Code, and cloud-native development practices.
• Excellent communication and stakeholder management skills.
• Databricks Certified Data Engineer Professional

Responsibilities

• Design end-to-end cloud-native data architectures using AWS and Databricks.
• Architect scalable batch and real-time data processing solutions using Apache Kafka and streaming technologies.
• Lead the design and implementation of enterprise data engineering pipelines and data platforms.
• Define architecture standards, best practices, and governance for data integration and analytics solutions.
• Collaborate with business stakeholders to translate business requirements into scalable technical solutions.
• Provide technical leadership and architectural guidance to development teams throughout the SDLC.
• Optimize data pipelines for performance, scalability, security, and cost efficiency.
• Design data models and integration strategies for structured and semi-structured data.
• Ensure solutions adhere to enterprise security, compliance, and reliability standards.
• Evaluate emerging technologies and recommend innovative solutions to improve business capabilities.
• Support cloud migration and modernization initiatives.
• Mentor engineers and conduct architecture and design reviews.

Skills

AWS

Databricks

Kafka

SQL

Python

Scala

ETL/ELT

CI/CD

Cloud architecture

Stakeholder management

Data engineering

Delta Lake

Terraform

Kubernetes

Docker

Tools

Terraform

Kubernetes

Docker

Delta Lake

Job description

• Design end-to-end cloud-native data architectures using AWS and Databricks.
• Architect scalable batch and real-time data processing solutions using Apache Kafka and streaming technologies.
• Lead the design and implementation of enterprise data engineering pipelines and data platforms.
• Define architecture standards, best practices, and governance for data integration and analytics solutions.
• Collaborate with business stakeholders to translate business requirements into scalable technical solutions.
• Provide technical leadership and architectural guidance to development teams throughout the SDLC.
• Optimize data pipelines for performance, scalability, security, and cost efficiency.
• Design data models and integration strategies for structured and semi-structured data.
• Ensure solutions adhere to enterprise security, compliance, and reliability standards.
• Evaluate emerging technologies and recommend innovative solutions to improve business capabilities.
• Support cloud migration and modernization initiatives.
• Mentor engineers and conduct architecture and design reviews.

Key Responsibilities

• Design end-to-end cloud-native data architectures using AWS and Databricks.
• Architect scalable batch and real-time data processing solutions using Apache Kafka and streaming technologies.
• Lead the design and implementation of enterprise data engineering pipelines and data platforms.
• Define architecture standards, best practices, and governance for data integration and analytics solutions.
• Collaborate with business stakeholders to translate business requirements into scalable technical solutions.
• Provide technical leadership and architectural guidance to development teams throughout the SDLC.
• Optimize data pipelines for performance, scalability, security, and cost efficiency.
• Design data models and integration strategies for structured and semi-structured data.
• Ensure solutions adhere to enterprise security, compliance, and reliability standards.
• Evaluate emerging technologies and recommend innovative solutions to improve business capabilities.
• Support cloud migration and modernization initiatives.
• Mentor engineers and conduct architecture and design reviews.

Required Qualifications

• 8+ years of experience in software, data engineering, or solution architecture.
• Strong experience designing enterprise data platforms on AWS.
• Hands-on expertise with Databricks and the Lakehouse architecture.
• Experience implementing Apache Kafka for real-time event streaming.
• Strong background in Data Engineering, including ETL/ELT pipeline development.
• Experience building scalable real-time and streaming data processing solutions.
• Strong understanding of distributed systems and cloud architecture principles.
• Experience with SQL, Python, or Scala.
• Knowledge of CI/CD, Infrastructure as Code, and cloud-native development practices.
• Excellent communication and stakeholder management skills.
• Databricks Certified Data Engineer Professional

Preferred Qualifications

• Experience with AI/ML platforms or deploying machine learning solutions into production.
• Experience with Delta Lake, Spark Structured Streaming, or similar technologies.
• AWS Certification (Solutions Architect or Data Analytics) is preferred.
• Experience with Terraform, Kubernetes, or Docker is a plus.