Sr. Lead- Databricks

Azimuth Grc; Gurugram District; On-site; INR 4,000,000 - 7,000,000; Full time

Job summary

Azimuth Grc in Gurugram seeks a seasoned Data Architect to define and communicate the data platform strategy, technical design, and roadmaps for large-scale analytics. You will lead Databricks-based implementations (Delta Lake, cloud data warehouses), create scalable pipelines in Azure Data Factory, and guide teams on data governance and performance optimization.

The role requires deep expertise in PySpark, Python data extraction, Unity Catalog, DevOps practices, and cloud modernization to

Qualifications

• Bachelor's degree in computer science or a related field.
• 15+ years of experience with Azure Databricks, Azure Data Factory, and PySpark.
• Experience extracting data from unstructured files (XML, PDF, HTML).
• Hands-on knowledge of data security, ELT, and cloud computing.
• Experience with Unity Catalog, dashboards, and Databricks deployment tools.
• Experience in Agile environments and Azure DevOps.

Responsibilities

• Conceptualize and communicate data architecture strategy and technology roadmap.
• Lead design and development of data architecture for large-scale Databricks solutions.
• Guide teams on data processes and architectural approaches for data strategy.
• Provide data solutions enabling BI integrations, self-service analytics, and AI-driven products.
• Document cloud modernization value propositions for stakeholders.
• Architect Databricks use cases (streaming and batch) and integrate with other components.
• Translate complex data problems into actionable technical approaches.
• Identify issues, perform root cause analysis, and communicate results.
• Propose opportunities and participate in proofs-of-concept.

Skills

Azure Databricks

PySpark

Python

Databricks Lakehouse

Data architecture

Azure Data Factory

Azure SQL/Data Warehouse

Delta Lake

Spark

Cloud modernization

Agile

Communication skills

Project leadership

Education

Bachelor's degree in Computer Science, Information Systems, or related field

Tools

Unity Catalog

Job Clusters

Databricks Deployment Bundles

Databricks Dashboards

Azure DevOps

Job description

RESPONSIBILITIES:

• Conceptualize and communicate Data architecture strategy, technical design and technology roadmap for data platform solutions and services.
• Lead design and development of data architecture to support implementation of large-scale data solutions in Databricks to support multiple use cases (delta lake, cloud Datawarehouse migrations, reporting and analytics).
• Guiding the organization and development teams on the overall data processes, architectural approaches for data strategy design and implementation.
• Providing data solutions that enable business intelligence data integrations, data services, self-service analytics, and data-driven digital products and services.
• Articulating the value proposition of cloud modernization/transformation to stakeholders; creating detailed documentation that empowers other engineers and customers.
• Architecting the solutions for Databricks-specific use cases (streaming and batch) as well as integrating it into other solution components in the customer architecture landscape.
• Translating the advanced business data, integration and analytics problems into technical approaches that yield actionable recommendations, across multiple, diverse domains.
• Implement architectural improvements for existing solutions using legacy modernization and cloud adoptions strategies.
• Creating Azure Data Factory processes for SQL and NoSQL data pipelines, and work in Azure SQL, to create tables, views, stored procedures, and other database objects.
• Creating applications through Scala and Python.
• Developing Spark tasks for data aggregation and transformation
• Writing Scala style documents with each code.
• Identifying technical issues (performance bottlenecks, data discrepancies, system defects) in data and software, perform root cause analysis, and communicate results effectively to the development team and other stakeholders.
• Providing recommendations on opportunities and improvements and participating in technology and analytics proof-of-concepts

QUALIFICATION AND EXPERIENCE:

• Bachelors degree in computer science, Data, Information Systems, or related field.
• 15 years of experience with Azure Databricks, Azure Data Factory, and Pyspark
• Experience with Python extracting data from unstructured files, such as XML, PDF, HTML.
• Hands‑on working knowledge of topics such as data security, messaging patterns, ELT, Data wrangling and cloud computing and proficiency in data integration/EAI and DB technologies, sophisticated analytics tools, programming languages or visualization platforms
• Hands‑on experience with Unity Catalog, Job Clusters, Databricks Deployment Bundles, Databricks Dashboards, Automations, and AI/BI Dashboards.
• Highly proficient in use of Spark and Azure Databricks
• Experience with deployment and builds using Azure DevOps
• Experience in an agile development environment, or an understanding of the concepts of Agile software development
• Excellent communication, organization, technical, and project management skills
• Experience in leading projects relating to cloud modernization, data migration, data warehousing experience with cloud-based data platforms (Databricks/Apache Spark); preferably Databricks certified; experience driving technical workshops with technical and business clients to derive value added services and implementation.