Resident Solution Architect

Zephico Technologies; India; Remote; INR 4,500,000 - 6,000,000; Full time

Job summary

Zephico Technologies is seeking an experienced Data Engineer/Architect to design and deliver scalable Databricks-based data platforms and analytics solutions. You will work with clients to translate business needs into robust technical architectures and supervise end-to-end project delivery from design to production.

The role requires hands-on expertise with Databricks, Spark, PySpark, and SQL, plus strong cloud experience and MLOps familiarity.

Qualifications

• 7+ years of experience in Data Engineering, Data Platforms, and Analytics.
• 10+ years of consulting experience.
• Hands-on delivery experience in 68+ Databricks projects.
• Strong expertise in Databricks, Apache Spark, PySpark, and SQL.
• Deep understanding of distributed computing and Spark architecture/runtime internals.
• Working knowledge of at least two cloud platforms: AWS, Azure, or GCP.
• Deep expertise in at least one major cloud ecosystem.
• Experience with CI/CD and production deployments.
• Working knowledge of MLOps.
• Strong understanding of Databricks Lakehouse architecture and platform capabilities.
• Experience with performance tuning, optimization, and scalability.
• Strong client-facing communication, consulting, and solution-architecture skills.

Responsibilities

• Design and architect scalable data engineering and analytics solutions using Databricks.
• Work closely with clients to understand business requirements and translate them into technical solutions.
• Lead the architecture and delivery of complex Databricks projects from design through production.
• Develop and optimize data pipelines using Apache Spark, PySpark, SQL, and Databricks.
• Apply Spark performance optimization techniques and understand Spark runtime internals.
• Design solutions across cloud platforms and recommend appropriate cloud architecture.
• Support CI/CD and production deployment of data engineering workloads.
• Provide technical leadership and guidance to engineering teams.
• Advise clients on Databricks platform capabilities, architecture, scalability, security, and best practices.
• Support MLOps initiatives and integration of machine learning workloads with data platforms.
• Troubleshoot complex performance, scalability, and production issues.
• Stay current with the latest Databricks products, features, and platform capabilities.

Skills

Databricks

Apache Spark

PySpark

SQL

Cloud platforms

CI/CD

MLOps

Spark internals

Data engineering

Data platforms

Job description

Key Responsibilities

• Design and architect scalable data engineering and analytics solutions using Databricks.
• Work closely with clients to understand business requirements and translate them into technical solutions.
• Lead the architecture and delivery of complex Databricks projects from design through production.
• Develop and optimize data pipelines using Apache Spark, PySpark, SQL, and Databricks.
• Apply Spark performance optimization techniques and understand Spark runtime internals.
• Design solutions across cloud platforms and recommend appropriate cloud architecture.
• Support CI/CD and production deployment of data engineering workloads.
• Provide technical leadership and guidance to engineering teams.
• Advise clients on Databricks platform capabilities, architecture, scalability, security, and best practices.
• Support MLOps initiatives and integration of machine learning workloads with data platforms.
• Troubleshoot complex performance, scalability, and production issues.
• Stay current with the latest Databricks products, features, and platform capabilities.

Required Skills & Experience

• 7+ years of experience in Data Engineering, Data Platforms, and Analytics.
• 10+ years of consulting experience.
• Hands-on delivery experience in 68+ Databricks projects.
• Strong expertise in Databricks, Apache Spark, PySpark, and SQL.
• Deep understanding of distributed computing and Spark architecture/runtime internals.
• Working knowledge of at least two cloud platforms: AWS, Azure, or GCP.
• Deep expertise in at least one major cloud ecosystem.
• Experience with CI/CD and production deployments.
• Working knowledge of MLOps.
• Strong understanding of Databricks Lakehouse architecture and platform capabilities.
• Experience with performance tuning, optimization, and scalability.
• Strong client-facing communication, consulting, and solution-architecture skills.

Certification

• Databricks Certified Data Engineer Professional certification required/preferred.