Resident Solution Architect

Zephico Technologies; United States; Remote; USD 150,000 - 200,000; Full time

Job summary

Zephico Technologies in the United States seeks an experienced Data Engineering Lead to design and deliver scalable Databricks-based data platforms. You will translate client requirements into robust architectures and supervise end-to-end delivery from design to production.

You bring deep expertise in Databricks, Spark, PySpark and SQL, with a track record on 68+ Databricks projects, and strong consulting and client-facing skills.

Qualifications

• 7+ years of experience in Data Engineering, Data Platforms, and Analytics.
• 10+ years of consulting experience.
• Hands-on delivery experience in 68+ Databricks projects.
• Strong expertise in Databricks, Apache Spark, PySpark, and SQL.
• Deep understanding of distributed computing and Spark architecture/runtime internals.
• Working knowledge of at least two cloud platforms: AWS, Azure, or GCP.
• Deep expertise in at least one major cloud ecosystem.
• Experience with CI/CD and production deployments.
• Working knowledge of MLOps.
• Strong knowledge of Databricks Lakehouse architecture and platform capabilities.
• Experience with performance tuning, optimization, and scalability.
• Strong client-facing communication, consulting, and solution-architecture skills.
• Databricks Certified Data Engineer Professional certification.

Responsibilities

• Design and architect scalable data engineering and analytics solutions using Databricks.
• Work closely with clients to translate business requirements into technical solutions.
• Lead architecture and delivery of complex Databricks projects from design through production.
• Develop and optimize data pipelines using Apache Spark, PySpark, SQL, and Databricks.
• Apply Spark performance optimization techniques and understand Spark runtime internals.
• Design cloud-agnostic solutions and recommend cloud architecture.
• Support CI/CD and production deployment of data engineering workloads.
• Provide technical leadership to engineering teams.
• Advise clients on Databricks platform capabilities, architecture, scalability, security, and best practices.
• Support MLOps initiatives and integration of ML workloads with data platforms.
• Troubleshoot performance, scalability, and production issues.
• Stay current with Databricks products, features, and platform capabilities.

Skills

Data engineering

Data platforms

Analytics

Consulting

Client-facing

Solution architecture

MLOps

CI/CD

Distributed computing

Spark internals

Education

Databricks Certified Data Engineer Professional

Tools

Databricks

Apache Spark

PySpark

SQL

Cloud platforms

AWS

Azure

GCP

CI/CD tools

MLOps tooling

Job description

Key Responsibilities

• Design and architect scalable data engineering and analytics solutions using Databricks.
• Work closely with clients to understand business requirements and translate them into technical solutions.
• Lead the architecture and delivery of complex Databricks projects from design through production.
• Develop and optimize data pipelines using Apache Spark, PySpark, SQL, and Databricks.
• Apply Spark performance optimization techniques and understand Spark runtime internals.
• Design solutions across cloud platforms and recommend appropriate cloud architecture.
• Support CI/CD and production deployment of data engineering workloads.
• Provide technical leadership and guidance to engineering teams.
• Advise clients on Databricks platform capabilities, architecture, scalability, security, and best practices.
• Support MLOps initiatives and integration of machine learning workloads with data platforms.
• Troubleshoot complex performance, scalability, and production issues.
• Stay current with the latest Databricks products, features, and platform capabilities.

Required Skills & Experience

• 7+ years of experience in Data Engineering, Data Platforms, and Analytics.
• 10+ years of consulting experience.
• Hands‑on delivery experience in 68+ Databricks projects.
• Strong expertise in Databricks, Apache Spark, PySpark, and SQL.
• Deep understanding of distributed computing and Spark architecture/runtime internals.
• Working knowledge of at least two cloud platforms: AWS, Azure, or GCP.
• Deep expertise in at least one major cloud ecosystem.
• Experience with CI/CD and production deployments.
• Working knowledge of MLOps.
• Strong understanding of Databricks Lakehouse architecture and platform capabilities.
• Experience with performance tuning, optimization, and scalability.
• Strong client‑facing communication, consulting, and solution‑architecture skills.

Certification

• Databricks Certified Data Engineer Professional certification required/preferred.