Data Engineer

nCircle Tech Co; Pune District; Hybrid; INR 4,000,000 - 7,000,000; Full time

Benefits offered by this job

Performance bonus

Hybrid work arrangements

Databricks training budget

Engineering-first data culture

Career progression path

Job summary

nCircle Tech Private Limited is seeking a hands-on Data Architect with deep Databricks expertise to design, build, and optimise enterprise-scale data platforms. You will own the end-to-end data engineering lifecycle—ingestion, transformation, and serving—ensuring reliability, scalability, and governance across a lakehouse architecture.

You will collaborate with data engineers, analytics engineers, and product teams to translate business requirements into robust data solutions on the Databricks

Qualifications

• Minimum 4+ years hands-on with Databricks
• Strong PySpark and Spark SQL skills
• Deep Delta Lake knowledge including ACID, time travel, OPTIMIZE
• Experience with Unity Catalog and data governance
• Proficient in SQL; knowledge of dbt a plus
• Cloud platform experience (Azure/AWS/GCP)
• CI/CD and IaC (Terraform) experience
• Familiarity with streaming pipelines (Kafka, Kinesis, etc.)

Responsibilities

• Design and maintain the lakehouse architecture using Databricks and Delta Lake.
• Define scalable data ingestion and transformation pipelines.
• Own schema evolution and data quality governance across platforms.
• Collaborate with engineers and product teams to deliver robust data products.
• Implement CI/CD for data pipelines and infrastructure as code.

Skills

Databricks

PySpark

Spark SQL

Delta Lake

Databricks Workflows

Delta Live Tables

Unity Catalog

Data modelling

Streaming pipelines

SQL

Cloud platforms (Azure/AWS/GCP)

GitHub Actions

Terraform

dbt

Kafka

Tools

Databricks

Spark

Delta Lake

Unity Catalog

dbt

Terraform

GitHub Actions

Azure

AWS

GCP

Job description

nCircle Tech Private Limited (Incorporated in 2012) empowers passionate innovators to createimpactful 3D visualization software for desktop, mobile and cloud. Our domain expertise in CADand BIM customization is driving automation with the ability to integrate advanced technologieslike AI/ML and AR/VR, which empowers our clients to reduce time to market and meet businessgoals. nCircle has a proven track record of technology consulting and advisory services for AECand Manufacturing industry across the globe. Our team of dedicated engineers, partnerecosystem and industry veterans are on a mission to redefine how you design and visualize.

Job Description

Data Architect — Databricks

Experience

5 – 8 Years

Level

Mid-Level

EmploymentType

Full-Time

Location

PrimaryStack

Domain

About the Role

We are looking for a hands-on DataArchitect with deep expertise in Databricks to design, build, and optimiseenterprise-scale data platforms. You will own the end-to-end data engineeringlifecycle — from ingestion and transformation to serving — while ensuringreliability, scalability, and governance across our lakehouse architecture.

You will collaborate closely withdata engineers, analytics engineers, and product teams to translate businessrequirements into robust, reusable data solutions on the Databricks LakehousePlatform.

Key Responsibilities

• Design and maintain theorganisation's lakehouse architecture using Databricks and Delta Lake.
• Define data modellingstandards (dimensional, Data Vault 2.0, or medallion architecture) acrossBronze, Silver, and Gold layers.
• Architect scalableingestion frameworks using structured and unstructured data sources (Kafka,JDBC, REST APIs, cloud storage).
• Own schema evolutionstrategy and ensure backward-compatibility across data assets.
• Build and maintainproduction-grade ETL/ELT pipelines using PySpark, Spark SQL, and DatabricksWorkflows.
• Optimise Spark jobs forperformance — partitioning, Z-ordering, caching, and cluster right-sizing.
• Establish CI/CD practicesfor data pipelines using tools such as GitHub Actions, Azure DevOps, orDatabricks Asset Bundles.

DataGovernance & Quality

• Implement Unity Catalog fordata discovery, lineage tracking, fine-grained access control, and compliance.
• Define and enforce dataquality rules using Great Expectations, DLT expectations, or equivalentframeworks.
• Work with data governanceteams to document metadata, business glossary, and data contracts.

Platform& Infrastructure

• Manage Databricks workspaceconfiguration: clusters, pools, secrets, and access policies.
• Collaborate with cloud andDevOps teams on infrastructure-as-code (Terraform) for Databricks on Azure /AWS / GCP.
• Monitor platform health,SLAs, and cost using Databricks system tables and cloud-native monitoringtools.
• Partner with data consumers(analysts, data scientists, ML engineers) to define SLAs and publish clean,well-documented data products.
• Review code and providearchitectural guidance to junior engineers.
• Contribute to and championinternal data engineering best practices, runbooks, and documentation.

Required Skills & Experience

CoreDatabricks & Spark

• 4+ years of hands-onexperience with Databricks (Unified Data Analytics Platform).
• Strong proficiency inPySpark and Spark SQL for large-scale data transformation.
• Deep knowledge of DeltaLake — ACID transactions, time travel, OPTIMIZE, VACUUM.
• Experience with DatabricksWorkflows, Jobs, and Delta Live Tables (DLT).
• Familiarity with UnityCatalog and Databricks governance features.
• Solid understanding of datamodelling paradigms: dimensional modelling, Data Vault, or medallionarchitecture.
• Experience designing andoperating streaming pipelines (Structured Streaming, Kafka, Event Hubs, orKinesis).
• Proficiency in SQL;experience with dbt is a strong plus.
• Hands-on experience withcloud platforms: Azure (ADLS, ADF), AWS (S3, Glue), or GCP (BigQuery, GCS).

SoftwareEngineering Practices

• Version control with Git;experience with branching strategies and code review workflows.
• Ability to write testable,modular pipeline code with unit and integration tests.
• Familiarity with CI/CDpipelines and infrastructure-as-code (Terraform preferred).

Nice to Have

• Databricks Certified DataEngineer Associate or Professional certification.
• Experience with data meshor data product frameworks.
• Exposure to ML pipelines,MLflow, or Feature Store on Databricks.
• Knowledge of datacataloguing tools (Alation, Collibra, or Databricks Unity Catalog).
• Experience with ApacheIceberg or Apache Hudi as alternative table formats.
• Familiarity with real-timeanalytics or OLAP systems (Druid, ClickHouse, Redshift).

What We Offer

• Competitive salary withperformance-linked bonus.
• Flexible / hybrid workingarrangements.
• Access to Databrickstraining and certification budget.
• Collaborative,engineering-first data culture with modern tooling.
• Clear career progressionpath to Senior Data Architect or Data Platform Lead.
• Comprehensive health,wellness, and retirement benefits.

Skills / responsibilities the panel was screening for

• Databricks platform depth: Unity Catalog (governance, lineage, three-level namespace, external/managed tables, row/column-level masking), Delta Lake, cluster management (all-purpose vs. job vs. serverless compute), Databricks Workflows
• Data modeling: medallion architecture (bronze/silver/gold), SCD Type 1/2, surrogate key generation, star vs. snowflake schema design
• Data quality & governance: schema evolution/enforcement, null handling, data contracts, naming standards/glossaries, tools like DQX, Great Expectations, Atlan
• Emerging tool fluency: interest in AI-assisted data engineering (Claude skills/plugins for profiling reports, dynamic dashboards) — they're actively moving in this direction
• Business/domain awareness: understanding how a construction company (multi-domain: HR, risk, project delivery, competitive intelligence) uses data for things like cost prediction and bid-win-probability modeling

This is key for us, we need to sahre this with Cangra