Solutions Architect (Databricks)

Perficient India; Hyderabad; Hybrid; INR 4,000,000 - 6,000,000; Full time

Job summary

Perficient India is seeking a Solutions Architect (Databricks) to design, build, and optimize scalable data pipelines on Azure Databricks. You will lead end-to-end data engineering initiatives, collaborate with analytics teams, and implement best practices for reliability and performance.

The ideal candidate has 10+ years of programming and 7+ years in data engineering, with Azure Databricks, PySpark, Python, and SQL expertise, plus cloud experience (AWS/Azure/GCP).

Qualifications

• 10+ years of programming experience.
• 7+ years of Data Engineering experience with Azure Databricks, Python, PySpark and SQL.
• Certification in Data Engineering (Associate or Professional).
• Experience with two or more Cloud ecosystems (AWS, Azure, GCP) and Databricks product knowledge.

Responsibilities

• Design and maintain ETL/ELT pipelines using Azure Databricks.
• Write PySpark and Python code for large-scale data processing.
• Develop and optimize complex SQL queries for transformation and reporting.
• Work with Delta Lake, ADLS, and Azure Data Factory; monitor pipeline performance.
• Collaborate with data analysts and business teams to deliver reliable datasets.

Education

Data Engineering Associate or Professional certification

Tools

Azure Databricks

Python

PySpark

SQL

Delta Lake

Azure Data Factory

ADLS

CI/CD

MLOps

Job description

We are looking for a Solutions Architect (Databricks)

Locations: #Hyderabad |#Bangalore | #Chennai | #Pune | #Nagpur

Experience: 10+ Years

Employment Type: Full-time | Hybrid (3 days work from office)

Interview Mode: Virtual followed by mandatory in-person

Notice Period: Immediate- 15 Days

Job Title: Solutions Architect (Databricks)

Job Description:

We are looking for a skilled Data Engineer with strong expertise in Azure Databricks, PySpark, SQL, and Python to join our team. The ideal candidate will design, build, and optimize scalable data pipelines and support analytics initiatives on the Azure platform.

Responsibilities:

• Design and maintain ETL/ELT pipelines using Azure Databricks
• Write efficient PySpark and Python code for large-scale data processing
• Develop and optimize complex SQL queries for data transformation and reporting
• Work with Delta Lake, Azure Data Lake Storage (ADLS), and Azure Data Factory
• Monitor and troubleshoot pipeline performance
• Collaborate with data analysts and business teams to deliver reliable datasets

Requirements:

• Minimum 10+ years of programming experience with (7+ years of Data Engineer; Azure Databricks, Python, PySpark, SQL).
• Completed Data Engineering Associate or Professional certification
• Minimum 6-8+ projects delivered with hands-on experience in development on Databricks
• Working knowledge of two or more common Cloud ecosystems (AWS, Azure, GCP) with deep expertise in at least one
• Deep experience with distributed computing with Spark with knowledge of Spark runtime internals
• Familiarity with CI/CD for production deployments
• Working knowledge of MLOps
• Current knowledge across the breadth of Databricks product and platform features
• Familiarity with optimizations for performance and scalability

Preferred Certifications

• Databricks Certified Data Engineer Associate or Professional