Solution Architect (Databricks)

Diggibyte Technologies; Krishnagiri District, Ahmedabad District, Bengaluru; On-site; INR 3,000,000 - 5,500,000; Full time

Job summary

Diggibyte Technologies is seeking a Databricks Solution Architect to design and implement scalable data solutions on the Databricks Lakehouse Platform. You will collaborate with data engineers and business stakeholders to translate requirements into robust data architectures.

The role demands strong expertise in data engineering, cloud platforms, and big data technologies, with hands-on experience in PySpark, SQL, and data governance.

Qualifications

• Experience designing scalable data solutions on Databricks Lakehouse Platform.
• Strong background in data engineering, cloud platforms, and big data tech.
• Ability to translate business needs into technical data architectures.
• Hands-on with data governance, security, and performance tuning.

Responsibilities

• Design and implement data pipelines and scalable data architectures using Databricks Lakehouse Platform.
• Collaborate with data engineers, architects, and business teams to translate requirements into technical solutions.
• Develop and optimize ETL/ELT workflows using Databricks, PySpark, and SQL.
• Build data lakes and lakehouse architectures on cloud platforms such as AWS, Azure, or GCP.
• Ensure best practices in data governance, security, and performance optimization.
• Support migration of legacy data platforms to Databricks-based modern data platforms, Unity Catalog.
• Provide technical guidance and mentoring to junior engineers and team members.
• Participate in solution design discussions, architecture reviews, and client presentations.
• Troubleshoot performance issues and optimize data processing workloads.

Skills

Databricks

Apache Spark

PySpark

Python

SQL

Data pipelines

ETL frameworks

Cloud platforms

Delta Lake

Data governance

Security

CI/CD

DevOps

Tools

Databricks

Azure

AWS

GCP

Delta Lake

Job description

We are looking for a skilled Databricks Solution Architect who will be responsible for designing and implementing scalable data solutions using the Databricks Lakehouse Platform. The ideal candidate should have strong experience in data engineering, cloud platforms, and big data technologies, and will work closely with clients, data engineers, and business stakeholders to deliver robust data architectures.

Key Responsibilities

• Design and implement data pipelines and scalable data architectures using Databricks Lakehouse Platform.
• Collaborate with data engineers, architects, and business teams to understand business requirements and translate them into technical solutions.
• Develop and optimize ETL/ELT workflows using Databricks, PySpark, and SQL.
• Build data lakes and lakehouse architectures on cloud platforms such as AWS, Azure, or GCP.
• Ensure best practices in data governance, security, and performance optimization.
• Support migration of legacy data platforms to Databricks-based modern data platforms, Unity Catalog.
• Provide technical guidance and mentoring to junior engineers and team members.
• Participate in solution design discussions, architecture reviews, and client presentations.
• Troubleshoot performance issues and optimize data processing workloads.
  

Required Skills

• Strong experience with Databricks and Apache Spark.
• Proficiency in PySpark, Python, and SQL.
• Experience with data pipeline development and ETL frameworks.
• Hands-on experience with cloud platforms (Azure / AWS / GCP).
• Good understanding of data lake, data warehouse, and lakehouse architectures.
• Experience working with Delta Lake, Data Engineering workflows, and data modeling.
• Knowledge of CI/CD pipelines and DevOps practices for data platforms.
  

Preferred Qualifications

• Experience with Azure Data Factory, Azure Synapse, or AWS Glue.
• Databricks certification such as Databricks Data Engineer Associate / Professional.
• Exposure to streaming technologies like Kafka or Spark Streaming.
• Strong problem-solving and stakeholder communication skills.