Databricks Architect - India

Cogniify, Inc.; Hyderabad; Remote; INR 3,000,000 - 6,000,000; Full time

Benefits offered by this job

Unlimited PTO

Generous parental leave

Open communication with management

Small, dynamic teams

Medical, Dental, Vision coverage

Disability & Life insurance

Mental health support

Employer Stock Purchase Program (ESPP)

Yearly team building experiences

Mentorship and sponsorship

Manager resources and support

Diversity & inclusion commitment

Job summary

Cognify Analytics is seeking a Databricks Architect (Senior/Specialist) in India to lead data platform architecture on the Databricks Lakehouse. You will bridge data engineering, analytics, and AI/ML with hands-on expertise in Databricks, Python, and SQL, guiding governance and scalable delivery across client engagements.

You will own lakehouse design, data mesh strategies, and integration with AI/ML workflows, while mentoring engineers and coordinating with multiple teams.

Qualifications

• Databricks on the Lakehouse platform in production-grade scale.
• Strong Python proficiency for data engineering and pipelines.
• Advanced SQL for data modeling, transformation, and performance tuning.

Responsibilities

• Own the architecture and design of Databricks-based data platforms.
• Serve as senior technical authority on platform design and standards.
• Design and build ingestion, transformation, and orchestration pipelines using Python, SQL, PySpark, and Databricks Workflows.
• Architect data mesh and data product strategies with domain ownership and data contracts.
• Establish best practices across ingestion, transformation, modeling, quality, observability and governance.

Skills

Databricks

Python

SQL

Education

Bachelor's, Master's or equivalent in CS/DS/Stats

Tools

dbt

Airflow/Dagster

Fivetran/Airbyte

Spark

Kafka

Cloud data services

Job description

Engineering · Data Science / Data Engineering

Databricks Architect - India

The Role

We're seeking a Databricks Architect (Senior/Specialist level) to serve as a senior technical authority for data architecture and analytics engineering on the Databricks Lakehouse platform across Cognify Analytics and our client engagements. In this role, you will design and own Databricks-based data platform architecture, lead complex data engineering initiatives, and ensure our data capabilities are enterprise-grade, governed, and future-ready. You will bridge data engineering, analytics, and AI/ML — combining deep hands-on expertise in Databricks, Python, and SQL with strong architectural judgment and stakeholder communication.

Must-Have Skills (Non-Negotiable)

Databricks — hands-on architecture and engineering experience on the Databricks Lakehouse Platform (production-grade, at scale)

Python — strong professional proficiency for data engineering and pipeline development

SQL — advanced proficiency for data modeling, transformation, and performance tuning

Candidates without demonstrable, hands-on experience in all three of the above will not be considered.

What You'll Do

Own the architecture and design of Databricks-based data platforms, including lakehouse design (Delta Lake), medallion architecture, and unified analytics layers.

Serve as the senior technical authority on Databricks platform design, providing guidance on architecture, tooling, modeling, and engineering standards across teams and projects.

Design and build ingestion, transformation, and orchestration pipelines using Python, SQL, PySpark, and Databricks Workflows.

Architect data mesh and data product strategies, defining domain ownership, data contracts, and self-service consumption patterns on Databricks.

Establish and evangelize best practices across the data lifecycle: ingestion, transformation, modeling, quality, observability, governance, and consumption.

Drive the integration of Databricks with AI/ML capabilities, including feature engineering pipelines, vector data infrastructure for RAG, and MLflow-based model workflows.

Lead complex data migration, platform modernization, and consolidation initiatives for enterprise clients across industries.

Evaluate emerging data technologies (Apache Iceberg, Unity Catalog, Delta Live Tables, Mosaic AI) to inform architectural decisions.

Solve complex, ambiguous, high-impact data architecture problems that span multiple teams, platforms, or organizational boundaries.

Drive cross-functional alignment between data engineering, analytics, AI/ML, platform engineering, security, and product teams.

Mentor senior data engineers and analysts, fostering a culture of technical excellence.

Represent Cognify Analytics in discussions with client stakeholders and leadership on data capabilities, strategy, and technical roadmaps.

Define and enforce data governance, compliance (GDPR, HIPAA, SOC2), and responsible data management standards across all Databricks environments.

Drive FinOps maturity for the Databricks platform, including compute optimization, cluster policies, storage lifecycle management, and cost forecasting.

What We're Looking For

Bachelor's, Master's, or equivalent professional experience in Computer Science, Data Science, Statistics, or a related field.

6-9 years of professional experience in data engineering, analytics engineering, or data architecture, with demonstrated technical leadership on at least a few large-scale engagements.

Mandatory, hands-on expertise in Databricks, Python, and SQL in production environments.

Strong working knowledge of the modern data stack: dbt, Airflow/Dagster, Fivetran/Airbyte, Spark, Kafka, and cloud-native data services.

Solid grasp of data modeling methodologies (Kimball, Data Vault, Activity Schema, OBT) and the judgment to apply them across contexts.

Experience with cloud data infrastructure on AWS, Azure, or GCP (any combination is acceptable, alongside Databricks).

Proven ability to influence technical direction and align technical and business stakeholders on complex architecture topics.

Strong understanding of data governance, data quality, cataloging, lineage, and regulatory compliance frameworks.

Experience mentoring engineers and contributing to a high-performing data team.

Strong understanding of how data platforms serve AI/ML workloads, including feature engineering, vector data, and model input/output pipelines.

Preferred Qualifications

Experience architecting Databricks-based platforms that directly serve LLM-based systems, RAG pipelines, and agentic AI architectures at enterprise scale.

Deep familiarity with lakehouse table formats: Delta Lake, Apache Iceberg, and Apache Hudi.

Track record of implementing data mesh, data product, or federated data governance patterns.

Experience with real-time analytics and streaming architectures: Kafka, Flink, Spark Structured Streaming.

Experience with advanced Databricks features: Unity Catalog, Delta Live Tables, Databricks Workflows, MLflow, and Mosaic AI.

Contributions to open-source data projects, data architecture publications, or industry standard bodies.

Background in financial services, healthcare, SaaS, or enterprise consulting requiring high data compliance, security, and operational rigor.

Experience leading data engineering across geographically distributed teams and multi-client engagements.

Why Join Cognify Analytics?

Join a team of industry veterans from Google, Meta, and top-tier tech companies.

Work on impactful, high-scale data and analytics projects with leading global clients.

Enjoy a flexible, remote-first culture focused on innovation and excellence.

Competitive salary, equity options, and continuous learning opportunities.

Shape the future of modern data platforms and AI-powered analytics at a rapidly growing company

Perks and Benefits

Unlimited PTO

Generous parental leave, above industry standards

Open communication with management and company leadership

Small, dynamic teams = massive impact

Medical, Dental, and Vision coverage

Access to Disability & Life insurance

Mental health and wellbeing support

Employer Stock Purchase Program (ESPP)

Yearly team building experiences

Mentorship and sponsorship opportunities

Manager resources and support

We are an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, veteran status, or any other protected characteristic.