Databricks Architect

InfoCepts; Maharashtra; On-site; INR 900,000 - 1,800,000; Full time

Job summary

InfoCepts in India is seeking a skilled data engineering lead to design, develop, and implement scalable data solutions using Databricks on cloud platforms. You will architect data pipelines, optimize Spark jobs, and ensure governance and security across enterprise analytics initiatives.

You will collaborate with data engineers, analysts, and business stakeholders to translate requirements into robust technical solutions, integrate Databricks with Azure, AWS or GCP, and monitor performance to

Qualifications

• Experience designing scalable data architectures on Databricks.
• Proficiency in building ETL/ELT pipelines for structured and unstructured data.
• Experience with cloud platforms (Azure/AWS/GCP) and data governance.

Responsibilities

• Architecture Design: Develop and maintain scalable, secure, and high-performance data architecture using Databricks.
• Data Pipeline Development: Lead the creation and optimization of ETL/ELT pipelines for structured and unstructured data.
• Cloud Integration: Ensure seamless integration of Databricks with cloud platforms like Azure, AWS, or GCP.
• Performance Optimization: Monitor and enhance the performance of data workflows and Spark jobs.
• Stakeholder Collaboration: Work closely with data engineers, analysts, and business stakeholders to translate requirements into technical solutions.

Skills

Databricks

Spark

Data modeling

ETL/ELT design

Cloud platforms

Tools

Azure

AWS

GCP

SQL

Job description

Purpose of the Position: To lead the design, development, and implementation of scalable data solutions using Databricks on cloud platforms. The role involves architecting data pipelines, optimizing performance, and ensuring data governance and security across enterprise-level analytics initiatives.

Type of Employment: Full time

Key Result Areas and Activities:

• Architecture Design: Develop and maintain scalable, secure, and high-performance data architecture using Databricks
• Data Pipeline Development: Lead the creation and optimization of ETL/ELT pipelines for structured and unstructured data.
• Cloud Integration: Ensure seamless integration of Databricks with cloud platforms like Azure, AWS, or GCP.
• Performance Optimization: Monitor and enhance the performance of data workflows and Spark jobs.
• Stakeholder Collaboration: Work closely with data engineers, analysts, and business stakeholders to translate requirements into technical solutions.