Solutions Architect

Quantzig; Bengaluru; On-site; INR 3,500,000 - 7,500,000; Full time

Job summary

Quantzig in Bengaluru is seeking a seasoned Solution Architect to design and implement Azure Databricks based data and ML platforms across edge and cloud environments. You will lead end-to-end architecture, governance, and CI/CD for production ML workloads.

The role requires deep expertise in lakehouse architectures, Unity Catalog governance, and OT/IT integration, with responsibilities spanning project leadership, client interaction, and cross-functional collaboration.

Qualifications

• 10+ years in data platform architecture or enterprise analytics.
• 5+ years on Azure-based data platforms.
• Strong expertise in Azure Databricks Lakehouse architecture.
• Experience designing ML platform infrastructure.
• Proven edge/industrial data system integration with cloud.

Responsibilities

• Architect integrated data and ML solutions across edge, cloud, and ML platform layers.
• Build and govern enterprise data platforms with lakehouse architecture and governance.
• Enable scalable ML platforms with feature and model stores, training and deployment pipelines.
• Oversee edge-to-cloud integration pipelines and OT/IT data extraction.
• Define monitoring, security, and governance standards for multi-zone data lakes.
• Lead CI/CD automation for ML workflows and platform deployments.
• Collaborate with data eng, ML eng, and DevOps teams across client projects.

Skills

Architectural leadership

Client communication

Translate business problems into IT

Cross-team collaboration

Multi-vendor delivery

Tools

Azure Databricks

Docker

RabbitMQ

OPC UA

GitHub Actions

Azure DevOps

Unity Catalog

Azure Data Factory

Job description

About Quantzig:

Quantzig is a global analytics and advisory firm with offices in the US, UK, Canada, China, and India. we have assisted our clients across the globe with end-to-end advanced analytics, visual storyboarding , Machine Learning and data engineering solutions implementation for prudent decision making. We are a rapidly growing organization that is built and operated by high-performance champions

Company Website: https://www.quantzig.com/

Solution Architect Azure Databricks Data & ML Platform Role Overview

We are seeking an experienced Solution Architect with strong expertise in Azure Databricks, enterprise data platforms, edge-to-cloud integration, and MLOps to lead architecture design and implementation across large-scale industrial and analytics ecosystems. The role involves designing scalable data platforms, enabling production-grade machine learning environments, and integrating OT systems with enterprise cloud data infrastructure. This role is ideal for architects working in client-facing consulting environments, delivering secure, governed, and scalable solutions across multi-layer data platforms and edge-connected intelligent systems.

Key Responsibilities

1. 1. Architect Integrated Data & ML Solutions  • Lead architecture design across edge, cloud, and ML platform layers.  • Define scalable architectures supporting telemetry ingestion, operational analytics, and ML-driven insights.  • Design zone-based lakehouse architectures aligned with enterprise governance standards.
2. 2. Build and Govern Enterprise Data Platform  • Architect ingestion and transformation pipelines across Raw fi Bronze fi Silver fi Gold layers.  • Implement enterprise-grade Unity Catalog-based governance frameworks.  • Define consumption-layer strategies supporting BI, ML, and operational workloads.
3. 3. Enable Scalable Machine Learning Platform  • Architect infrastructure supporting feature engineering, training, validation, versioning, and deployment.  • Enable Feature Store and Model Store implementations using Azure Databricks ML workspace.  • Support cloud and edge inference deployment strategies.  • Define lifecycle cadence models: Weekly refresh  • Monthly retraining  • Quarterly model revamp.
4. 4. Edge Integration and Automation  • Architect secure OTIT integration pipelines.  • Design edge-to-cloud ingestion using Docker, Portainer, RabbitMQ, and OPC UA interfaces for PLC connectivity.  • Implement IDMZ-compliant deployment patterns.
5. 5. Monitor and Optimize Data & ML Pipelines  • Define monitoring frameworks for ETL and ML workloads.  • Enable near real-time monitoring (e.g., 1-minute intervals for model telemetry).  • Optimize performance, reliability, and cost efficiency across workloads.
6. 6. Governance and Security Compliance  • Define enterprise governance standards using Unity Catalog.  • Implement access policies, metadata tagging, and lineage tracking.  • Ensure compliance across multi-zone data lake environments.
7. 7. Lead CI/CD Automation  • Architect CI/CD pipelines using Azure DevOps, GitHub Actions, and GitHub self-hosted runners.  • Enable automation for ML workflows and platform deployments.

Technical Expertise

Azure Cloud Stack & DevOps  • Azure Databricks (including ML workspace, Feature Store, Model Store)  • Azure Data Factory (ADF) orchestration  • Azure Data Lake Storage (ADLS) with Medallion architecture  • Azure Event Hub (topics, consumer groups, event-driven integration)  • Azure Stream Analytics for real-time telemetry processing  • Azure Key Vault, Azure App Service, Azure Container Registry (ACR)  • Azure IoT Hub for edge device connectivity  • Azure DevOps, GitHub Actions for CI/CD automation, GitHub self-hosted runners Edge and On-Prem Integration  • OT–IT integration architecture and data extraction from industrial/PLC environments  • Edge VM deployment using Docker, Portainer, and RabbitMQ messaging services  • OPC UA integration with PLC systems (e.g., compressors, filtration systems)  • IDMZ deployment strategies and secure edge-to-cloud service integration Machine Learning Platform & MLOps  • End-to-end ML lifecycle: feature engineering, training, validation, versioning, deployment  • Model export and versioning strategies; production deployment architectures  • Monitoring deployed models at frequent intervals; training vs inference architecture design  • Cloud vs edge inference strategies; GitHub monorepo-based ML workflow management  • Hands-on expertise: Azure Databricks ML workspace, Feature Store, Model Store Data Architecture & Integration  • Medallion architecture (Raw fi Bronze fi Silver fi Gold) across enterprise lakehouse environments  • Unity Catalog integration for governance and sharing  • CDC ingestion tools (e.g., Aecorsoft for SAP ingestion); streaming ingestion frameworks  • API-based ingestion architectures; template-driven ingestion with configuration-based mapping Governance and Data Modeling  • Define enterprise data governance frameworks; access control strategies across DLZ zones  • Design scalable dimensional and feature-driven data models  • Enable metadata tagging and lineage tracking  • Support analytics and ML feature readiness through optimized modeling Qualifications Required

Qualifications

• 10+ years in data platform architecture or enterprise analytics
• 5+ years on Azure-based data platforms
• Strong expertise in Azure Databricks Lakehouse architecture
• Experience designing ML platform infrastructure
• Proven edge/industrial data system integration with cloud
• Experience implementing Unity Catalog governance frameworks
• Strong stakeholder management in enterprise delivery

Preferred Qualifications

• Industrial analytics or manufacturing environments
• Exposure to IoT telemetry architectures
• SAP CDC ingestion framework experience
• Near real-time streaming analytics platform design
• Consulting or client-facing enterprise program experience

Soft Skills

• Strong architectural leadership
• Excellent client communication skills
• Translate business problems into platform architecture
• Cross-team collaboration (Data Eng, ML Eng, DevOps)
• Multi-vendor enterprise delivery experience

Engagement Model

Client-facing architecture role Works with enterprise stakeholders, ML teams, platform engineers, OT system teams Leads technical governance & architecture decision-making across platform layers

Primary Technology Stack

Azure Databricks ADLS ADF Unity Catalog Event Hub Stream Analytics Azure IoT Hub Docker RabbitMQ OPC UA GitHub Actions Azure DevOps Feature Store Model Store Edge ML Deployment CDC (SAP) Medallion Architecture