Solutions/Databricks Architect

Remotedxb; Dubai; On-site; AED 260,000 - 420,000; Full time

Job summary

MKS2 Technologies, LLC is seeking a senior data engineer to design enterprise lakehouse architectures using Databricks and to develop scalable ETL pipelines for real-time data processing. You will optimize Spark workloads, implement Delta Lake, and guide teams on best practices across federal cloud environments.

The role requires 5+ years in data engineering, 3+ years with Databricks, and strong SQL skills, with eligibility for a Public Trust clearance.

Qualifications

• 5+ years of experience in data engineering, data architecture, or large-scale data platforms.
• 3+ years of hands-on Databricks experience.
• Expertise in Databricks Lakehouse Platform, Delta Lake, Databricks SQL, and Apache Spark.
• Proficiency in Python, Scala, or Java.
• Experience with cloud platforms supporting federal environments (AWS GovCloud, Azure Government).
• Strong knowledge of ETL development, data modeling, data lakes, and data warehousing.
• Experience implementing security, governance, and compliance controls in regulated environments.
• Advanced SQL skills and experience managing large datasets.
• Eligibility to obtain and maintain a Public Trust clearance or higher.

Responsibilities

• Design and implement enterprise data lakehouse architectures using Databricks.
• Develop scalable ETL pipelines, data integrations, and real-time data processing solutions.
• Lead Databricks platform optimization, including Delta Lake, Databricks SQL, and Apache Spark workloads.
• Collaborate with stakeholders to define business requirements and translate them into technical solutions.
• Integrate Databricks with AWS GovCloud, Azure Government, or other federal cloud environments.
• Support advanced analytics, reporting, and machine learning initiatives utilizing MLflow.
• Provide technical leadership, mentoring, and best practices guidance to engineering teams.

Skills

Python

Scala

Java

SQL

Databricks

Spark

Cloud platforms

ETL

Data modeling

Tools

Databricks Platform

Delta Lake

Databricks SQL

Apache Spark

Terraform

CI/CD

Kubernetes

MLflow

Job description

Responsibilities

• Design and implement enterprise data lakehouse architectures using Databricks
• Develop scalable ETL pipelines, data integrations, and real-time data processing solutions
• Lead Databricks platform optimization, including Delta Lake, Databricks SQL, and Apache Spark workloads
• Ensure compliance with federal security and governance requirements, including FedRAMP, FISMA, and NIST standards
• Collaborate with stakeholders to define business requirements and translate them into technical solutions
• Integrate Databricks with AWS GovCloud, Azure Government, or other federal cloud environments
• Support advanced analytics, reporting, and machine learning initiatives utilizing MLflow
• Provide technical leadership, mentoring, and best practices guidance to engineering teams

Requirements

• 5+ years of experience in data engineering, data architecture, or large-scale data platforms
• 3+ years of hands‑on Databricks experience
• Expertise in Databricks Lakehouse Platform, Delta Lake, Databricks SQL, and Apache Spark
• Proficiency in Python, Scala, or Java
• Experience with cloud platforms supporting federal environments (AWS GovCloud, Azure Government)
• Strong knowledge of ETL development, data modeling, data lakes, and data warehousing
• Experience implementing security, governance, and compliance controls in regulated environments
• Advanced SQL skills and experience managing large datasets
• Eligibility to obtain and maintain a Public Trust clearance or higher

Preferred Qualifications

• Experience with CI/CD pipelines and Infrastructure as Code (Terraform, CloudFormation, ARM)
• Experience with Kafka, Azure Event Hubs, or AWS Kinesis
• Familiarity with TensorFlow, PyTorch, Scikit-learn, and MLflow
• Knowledge of Hadoop, Flink, or Elasticsearch
• Databricks Certified Professional Data Engineer or related certifications
• Federal cloud and cybersecurity certifications (CISSP, CCSP, CISM, AWS Security Specialty)
• Prior federal consulting experience

About the Company

MKS2 Technologies, LLC is an award-winning high growth small business creating innovative technology solutions in Cyber Security, Software Engineering, and IT Support Services for government agencies.