Databricks Architect

Impetus; Bengaluru; Vor Ort; INR 4.000.000 - 7.000.000; Vollzeit

Zusammenfassung

Impetus Technologies in Bengaluru is seeking a hands-on Senior Data Architect to lead enterprise-scale data initiatives. You will shape the technical direction for analytics platforms using Databricks and cloud migrations across AWS and Azure.

You will design scalable data pipelines, establish governance and security, mentor teams, and translate business needs into robust data blueprints for executive stakeholders.

Qualifikationen

• 12–14 years of experience designing, architecting, implementing, and optimizing large-scale data engineering and analytics solutions.
• Hands-on Databricks - Spark/PySpark, Delta Lake, lakehouse, medallion architecture, Unity Catalog, workflows, and optimization.
• Expert-level PySpark and Spark SQL with production-grade ETL/ELT and streaming architectures.
• Strong AWS and Azure cloud experience including IAM/security, cost optimization, and reliability patterns.
• Experience with Snowflake interoperability with Databricks; data warehousing patterns.
• Data orchestration, CI/CD for data, and Infrastructure-as-Code (Terraform); exposure to Docker/Kubernetes.
• Deep expertise in data governance, quality, lineage, cataloging, and security across the data lifecycle.
• Experience leading cloud/data platform migrations to Databricks and the cloud.
• Ability to set architecture standards, lead design reviews, and mentor multiple engineering teams.
• Strong communication and stakeholder-management skills for technical and executive audiences.

Aufgaben

• Set architecture standards and drive technical direction for enterprise-scale data programs.
• Lead design reviews and provide mentorship across engineering teams.
• Translate business needs into robust data blueprints and enforce governance across platforms.
• Collaborate with cloud, data, and analytics stakeholders to deliver scalable solutions.

Kenntnisse

Databricks

PySpark

AWS

Azure

Snowflake

Terraform

Docker

Kubernetes

Unity Catalog

Delta Lake

ETL/ELT frameworks

Data governance

Data modeling

Tools

EMR

Glue

Azure Data Factory

Synapse

Jobbeschreibung

Impetus Technologies is a digital engineering company focused on delivering expert services and products to help enterprises achieve their transformation goals. We solve the analytics, AI, and cloud puzzle, enabling businesses to drive unmatched innovation and growth.

Founded in 1991, we are cloud and data engineering leaders providing solutions to fortune 100 enterprises, headquartered in Los Gatos, California, with development centers in Noida, Indore, Gurugram, Bengaluru, Chennai, Pune, and Hyderabad with over 3000 global team members. We also have offices in Canada and Australia and collaborate with a number of established companies, including American Express, Bank of America, Capital One, Toyota, United Airlines, and Verizon.

Experience: 12+ Years

The ideal candidate is a hands-on architect with deep Databricks and PySpark expertise, strong AWS/Azure cloud fluency, and a proven record of setting technical direction on enterprise-scale programs.

Key requirements:

• 12–14 years of experience in designing, architecting, implementing, and optimizing large-scale (TB/PB) data engineering and analytics solutions and distributed data platforms.
• Deep, hands-on architecture expertise with Databricks - Spark/PySpark, Delta Lake, lakehouse and medallion architecture, Unity Catalog, Databricks Workflows, Delta Live Tables, and performance/cost optimization.
• Expert-level PySpark and Spark SQL, with strong Python for building reusable, production-grade ETL/ELT frameworks and streaming architectures (batch and real-time).
• Strong hands-on experience with AWS and Azure Cloud, including networking, IAM/security, cost optimization, and reliability patterns; comfortable architecting across cloud data services (e.g., AWS S3, EMR, Glue; Azure Data Factory, Synapse, ADLS).
• Strong experience architecting solutions with Snowflake and modern data warehousing - data modeling, performance tuning, security, and cost governance - and defining Databricks–Snowflake interoperability patterns.
• Strong experience with data orchestration, CI/CD for data, and Infrastructure-as-Code (Terraform or equivalent); exposure to Docker/Kubernetes.
• Deep expertise in data governance, data quality, lineage, cataloging, and security across the data lifecycle (e.g., Unity Catalog, RBAC, encryption, audit).
• Experience leading cloud/data platform migration and modernization programs - migrating legacy or on-premises platforms to Databricks and the cloud.
• Proven ability to set architecture standards, lead design reviews, and provide technical leadership and mentorship across multiple engineering teams.
• Strong communication and stakeholder-management skills; able to drive architecture decisions and translate business needs into technical blueprints for both technical and executive audiences.

Good to Have:

Exposure to Generative AI / LLM solution architecture — RAG pipelines, embedding and vector store design, and native Databricks GenAI services (Mosaic AI Model Serving, Vector Search, Agent Framework)