Databricks Architect

Zone IT Solutions; Hyderabad; On-site; INR 2,500,000 - 4,500,000; Full time

Job summary

Zone IT Solutions in Hyderabad, India seeks an experienced Databricks Architect to design scalable, secure data solutions on the Databricks Lakehouse platform. You will architect end-to-end data solutions, define ingestion and processing strategies, and implement Delta Lake, Unity Catalog, and Databricks workflows.

The role requires hands-on PySpark, Spark SQL, and SQL expertise, along with experience on Azure/AWS/GCP and data governance.

Qualifications

• Strong experience in Databricks and Data Architecture.
• Hands-on expertise with Apache Spark, PySpark, Spark SQL, and SQL.
• Knowledge of Delta Lake, Unity Catalog, and Databricks Workflows.
• Experience designing ETL/ELT pipelines and Lakehouse architectures.
• Experience with at least one cloud platform: Azure / AWS / GCP.
• Knowledge of data warehousing, data modeling, data governance, and data security.
• Excellent problem-solving and communication skills.
• Experience optimizing Spark/Databricks workloads.

Responsibilities

• Design and architect end-to-end data solutions using Databricks Lakehouse architecture.
• Define scalable data ingestion, transformation, storage, and processing strategies.
• Design and implement Delta Lake, Delta Live Tables (DLT), Unity Catalog, and Databricks workflows.
• Develop scalable data pipelines using PySpark, Spark SQL, and SQL.
• Integrate Databricks with cloud platforms such as AWS, Azure, or GCP.
• Establish best practices for data governance, security, access control, and data quality.
• Optimize Spark jobs, Databricks workloads, and data pipelines for performance and cost.
• Collaborate with data engineers, architects, analysts, and business teams to translate requirements into technical solutions.
• Lead data migration and modernization initiatives involving Databricks.
• Evaluate new Databricks features and technologies and recommend appropriate solutions.

Skills

Databricks

Data Architecture

Apache Spark

PySpark

Spark SQL

SQL

Data governance

Security

Performance tuning

Communication

Tools

Delta Lake

Unity Catalog

Databricks Workflows

Delta Live Tables

Job description

We are looking for an experienced Databricks Architect to design and implement scalable, secure, and high-performance data solutions using the Databricks Lakehouse platform. The ideal candidate should have strong expertise in data architecture, Databricks, cloud platforms, data engineering, and modern data technologies.

Key Responsibilities

• Design and architect end-to-end data solutions using Databricks Lakehouse architecture.
• Define scalable data ingestion, transformation, storage, and processing strategies.
• Design and implement Delta Lake, Delta Live Tables (DLT), Unity Catalog, and Databricks workflows.
• Develop scalable data pipelines using PySpark, Spark SQL, and SQL.
• Integrate Databricks with cloud platforms such as AWS, Azure, or GCP.
• Establish best practices for data governance, security, access control, and data quality.
• Optimize Spark jobs, Databricks workloads, and data pipelines for performance and cost.
• Collaborate with data engineers, architects, analysts, and business teams to translate requirements into technical solutions.
• Lead data migration and modernization initiatives involving Databricks.
• Evaluate new Databricks features and technologies and recommend appropriate solutions.

Key Skills & Requirements

• Strong experience in Databricks and Data Architecture.
• Hands-on expertise with Apache Spark, PySpark, Spark SQL, and SQL.
• Strong knowledge of Delta Lake, Unity Catalog, and Databricks Workflows.
• Experience designing ETL/ELT pipelines and Lakehouse architectures.
• Strong experience with at least one cloud platform: Azure / AWS / GCP.
• Knowledge of data warehousing, data modeling, data governance, and data security.
• Experience with performance tuning and optimization of Spark/Databricks workloads.Strong understanding of modern data engineering and distributed computing concepts.
• Excellent architecture, problem-solving, and communication skills.

About Us

We specialize in Digital, ERP, and larger IT Services. We offer flexible, efficient and collaborative solutions to any organization that requires IT, experts. Our agile, agnostic, and flexible solutions will help you source the IT Expertise you need. If you are looking for new opportunities, send your profile at [e-mail].

Also follow our LinkedIn page for new job opportunities and more.

Zone IT Solutions is an equal opportunity employer and our recruitment process focuses on essential skills and abilities. We encourage applications from a diverse array of backgrounds, including individuals of various ethnicities, cultures, and linguistic backgrounds, as well as those with disabilities.