Principal / Lead Data Architect (AWS Focus)

Orcapod Consulting Services

Bengaluru

On-site

INR 3,500,000 - 5,500,000

Full time

14 days+

Get more replies from employers

Send a job-specific resume in minutes.

Job summary

Orcapod Consulting Services in Bengaluru seeks a seasoned Lead/Principal Data Architect with 10+ years of experience to design, build, and scale a next-generation data platform. You will bridge complex business requirements with robust technical execution, focusing on Databricks and Snowflake within the AWS ecosystem.

You will lead architecture strategy, govern data solutions, mentor teams, and collaborate with AI/Data Science units to enable real-time data processing and scalable analytics.

Qualifications

Bachelor of Tech in a relevant field.
10+ years of progressive data engineering/architecture experience.
Leadership of enterprise-scale data platforms.

Responsibilities

End-to-end design of scalable, secure data architectures on cloud data ecosystems.
Architect and oversee streaming and batch data pipelines (ETL/ELT).
Architect within AWS with IAM and networking integration.
Design ingestion frameworks using Databricks APIs and external REST/GraphQL APIs.
Create real-time processing frameworks for low-latency use cases.
Harmonize relational warehousing with lakehouse patterns.
Provide technical governance and mentor engineering teams.
Collaborate with Data Science/AI to support AI/LLM pipelines.

Skills

Databricks

Snowflake

Structured Streaming

Apache Kafka

Flink/AWS Kinesis

Python

Scala

SQL

Education

B.Tech

Job description

Job Title

Principal / Lead Data Architect(AWS Focus)

Locations

Bengaluru

Minimum Experience

Mandatory Skills

Databricks,Python,Scala,SQL,Snowflake,Structured Streaming,Apache Kafka,Flink/AWS Kinesis'

Skill to Evaluate

Databricks,Python,Scala,SQL,Snowflake,Structured Streaming,Apache Kafka,Flink/AWS Kinesis

Experience

Above 10 Years

Location

Bengaluru

Job Description

Architecture & Strategy: End-to-end design of scalable, secure, and highly available data architectures leveraging modern cloud data ecosystems (Databricksand Snowflake).

Pipeline Engineering: Architect, optimize, and oversee the deployment of reliable streaming and batch data pipelines (ETL/ELT) to process complex, large-scale datasets.

Cloud Architecture: Architect and deploy scalable enterprise data platform components natively within the AWS ecosystem, ensuring tight integration with core security, IAM, and networking protocols.

API Ingestion & Orchestration: Design and implement robust data ingestion frameworks leveraging Databricks APIsand external REST/GraphQL APIs for automated workflows, platform orchestration, and data delivery.

Real-time Processing: Design and implement robust frameworks for real-time data ingestion and processing to solve business-critical, low-latency use cases.

Hybrid Data Modelling: Harmonize "old school" relational data warehousing patterns (Kimball/Inmon, Star/Snowflake schemas) with unstructured/semi-structured modern paradigms.

Technical Leadership: Act as a core problem-solver for complex data bottlenecks, provide technical governance, and mentor engineering teams on data best practices.

AI Integration: Collaborate with Data Science and AI teams to architect data layers that seamlessly support LLMs, Machine Learning pipelines, and advanced analytics solutions.

Education Qualification

B. Tech/MTech

Roles & Responsibilities

Position Summary

We are seeking a highly seasoned Lead/Principal Data Architectwith over a decade of experience to design, build, and scale our next-generation data platform. In this role, you will be the mastermind behind our data strategy, bridging the gap between complex business requirements and robust technical execution.

You will bring exceptional problem-solving abilities, deep expertise in Databricks and Snowflake, and a proven track record of engineering high-throughput, real-time data pipelines. The ideal candidate possesses a rare blend of deep respect for traditional data warehousing alongside a forward-thinking mastery of modern Lakehouse architectures, NoSQL systems, and AI-driven data solutions.

Key Responsibilities

Architecture & Strategy: End-to-end design of scalable, secure, and highly available data architectures leveraging modern cloud data ecosystems (Databricksand Snowflake).
Pipeline Engineering: Architect, optimize, and oversee the deployment of reliable streaming and batch data pipelines (ETL/ELT) to process complex, large-scale datasets.
Cloud Architecture: Architect and deploy scalable enterprise data platform components natively within the AWS ecosystem, ensuring tight integration with core security, IAM, and networking protocols.
API Ingestion & Orchestration: Design and implement robust data ingestion frameworks leveraging Databricks APIsand external REST/GraphQL APIs for automated workflows, platform orchestration, and data delivery.
Real-time Processing: Design and implement robust frameworks for real-time data ingestion and processing to solve business-critical, low-latency use cases.
Hybrid Data Modelling: Harmonize "old school" relational data warehousing patterns (Kimball/Inmon, Star/Snowflake schemas) with unstructured/semi-structured modern paradigms.
Technical Leadership: Act as a core problem-solver for complex data bottlenecks, provide technical governance, and mentor engineering teams on data best practices.
AI Integration: Collaborate with Data Science and AI teams to architect data layers that seamlessly support LLMs, Machine Learning pipelines, and advanced analytics solutions.

Required Qualifications & Skills

Qualification -

Bachelor of Technology (B.Tech)

Core Experience -

10+ years of progressive experience in Data Engineering, Data Warehousing, and Data Architecture. Demonstrated experience leading architectural decisions for enterprise-scale data platforms.

Technical Expertise

Databricks Mastery: Deep hands-on experience with the Databricks Lakehouse platform, Delta Lake, Unity Catalog, and optimizing Spark performance.
Data Pipeline Excellence: Exceptional expertise in designing distributed, fault-tolerant data pipelines using Python, Scala, or SQL.
Snowflake Proficiency: Strong hands-on experience architectural design, performance tuning, and cost-optimization within Snowflake.
Real-time Systems: Proven track record with stream processing technologies (e.g., Structured Streaming, Apache Kafka, Flink, or AWS Kinesis) for real-time use cases.
Polyglot Persistence: Solid foundation in traditional Data Warehousing and relational database management systems (RDBMS). Hands-on experience with NoSQL ecosystems.

Soft Skills & Problem Solving

Elite Problem-Solving: A strong analytical mindset with a track record of troubleshooting complex distributed systems and performance degradation issues.
Communication: Ability to articulate complex technical architectures clearly to both engineering teams and non-technical business stakeholders.

Preferred / Good-to-Have Qualifications

AI/ML Data Readiness: Exposure to architecting data solutions tailored for AI, such as vector databases (e.g., Pinecone, Milvus), feature stores, or building data pipelines for generative AI/LLM applications.
Relevant certifications (e.g., Databricks Certified Data Architect, Snowflake Certified Advanced Architect).

Type of Employment

Contract

Project Details

Architecture & Strategy: End-to-end design of scalable, secure, and highly available data architectures leveraging modern cloud data ecosystems (Databricksand Snowflake).
Pipeline Engineering: Architect, optimize, and oversee the deployment of reliable streaming and batch data pipelines (ETL/ELT) to process complex, large-scale datasets.
Cloud Architecture: Architect and deploy scalable enterprise data platform components natively within the AWS ecosystem, ensuring tight integration with core security, IAM, and networking protocols.
API Ingestion & Orchestration: Design and implement robust data ingestion frameworks leveraging Databricks APIsand external REST/GraphQL APIs for automated workflows, platform orchestration, and data delivery.
Real-time Processing: Design and implement robust frameworks for real-time data ingestion and processing to solve business-critical, low-latency use cases.
Hybrid Data Modelling: Harmonize "old school" relational data warehousing patterns (Kimball/Inmon, Star/Snowflake schemas) with unstructured/semi-structured modern paradigms.
Technical Leadership: Act as a core problem-solver for complex data bottlenecks, provide technical governance, and mentor engineering teams on data best practices.
AI Integration: Collaborate with Data Science and AI teams to architect data layers that seamlessly support LLMs, Machine Learning pipelines, and advanced analytics solutions.