Databricks Architect | Tech Club Inc | Malvern, PA

Tech Junction Ltd; Pennsylvania; Híbrido; USD 180.000 - 240.000; Jornada completa

Descripción de la vacante

Tech Club Inc in Malvern, PA seeks a senior Databricks Architect to design and scale enterprise-grade Databricks Lakehouse platforms and Delta Lake pipelines. You will own the enterprise data architecture, optimize Spark-based pipelines, and enforce governance across multi-cloud environments.

You will collaborate with business units and executive stakeholders, mentor engineers, and ensure compliance with data privacy and security baselines while delivering cost-efficient solutions.

Formación

• Bachelor’s or Master’s degree in a related field.
• 10+ years of professional software and data engineering experience with 5+ years in big data architecture.
• Deep expertise in Databricks Lakehouse, Delta Lake, Spark, and PySpark.
• Strong programming in Python/Scala/SQL for big data processing.
• Experience with Unity Catalog governance and data security controls.

Responsabilidades

• Define enterprise data architecture and scalable Delta Lake pipelines in multi-cloud Databricks environments.
• Architect, build, and optimize Databricks Lakehouse platforms and data storage ecosystems.
• Lead governance, access controls, and data security policies using Unity Catalog.

Conocimientos

Databricks Lakehouse

Delta Lake

Apache Spark

PySpark

Python/Scala/SQL

Unity Catalog governance

Multi-cloud architecture

Data governance

Leadership

Educación

Bachelor’s or Master’s degree in Computer Science or related field

Herramientas

Databricks

Airflow

Terraform

Databricks Workflows

Descripción del empleo

Position Summary

Tech Club Inc is seeking an experienced, highly technical, and accomplished Databricks Architect with a strong background in modern data lakehouse architectures to join our client delivery team on a full-time, onsite basis in Malvern, PA. In this specialized enterprise data engineering and cloud architecture role, you will spearhead designing, building, and scaling enterprise-grade Databricks Lakehouse platforms, Delta Lake pipelines, and advanced analytics solutions. You will work closely with C-level stakeholders, enterprise data engineers, data scientists, and infrastructure teams to deliver high-performance data platforms. Ideal candidates bring a robust academic degree in computer science or engineering, deep practical mastery of Apache Spark, Databricks, Python, and immediate availability for onsite deployment in Malvern.

Detailed Job Description

As a Databricks Architect at Tech Club Inc in Malvern, PA, you will take full ownership of defining the enterprise data architecture, designing scalable data pipelines, and establishing governance frameworks across multi-cloud Databricks environments. Your day-to-day responsibilities encompass architecting Delta Lake storage structures, optimizing Spark cluster performance, implementing Unity Catalog governance, and mentoring engineering teams. You will collaborate with business units to translate complex data analytics requirements into robust, cost-effective data solutions. Working in a fast-paced onsite environment in Malvern, PA, you will drive enterprise intelligence and modern data engineering excellence.

Key Responsibilities

• Architect, design, and deploy enterprise-grade Databricks Lakehouse platforms and cloud data storage ecosystems.
• Build, scale, and optimize high-throughput data pipelines, ETL/ELT workflows using Apache Spark, PySpark, and Delta Lake.
• Implement and manage enterprise data governance, access controls, and security policies utilizing Databricks Unity Catalog.
• Optimize cluster compute configurations, job scheduling, query performance, and cloud infrastructure cost efficiency.
• Collaborate closely with enterprise architects, data scientists, software engineers, and executive stakeholders in Malvern, PA.
• Establish CI/CD data operations (DataOps) pipelines for automated testing, deployment, and monitoring of data workflows.
• Provide technical leadership, architectural guidance, and mentorship to junior and mid-level data engineers.
• Ensure strict adherence to enterprise data privacy standards, regulatory compliance, and security baselines.
• Maintain comprehensive architectural blueprints, data lineage documentation, and engineering runbooks.
• Participate in executive steering committees and technology roadmap planning for enterprise data transformation.

Required Qualifications & Skills

• Bachelor’s or Master’s degree in Computer Science, Data Analytics, Information Systems, Software Engineering, or a related field.
• Minimum 10+ years of professional software and data engineering experience, with at least 5+ years dedicated to big data architecture.
• Deep specialized expertise in Databricks Lakehouse platform architecture, Delta Lake, Apache Spark, and PySpark.
• Strong programming proficiency in Python, Scala, or SQL for big data processing.
• Proven experience implementing data governance, lineage, and access controls using Unity Catalog.
• Extensive experience designing cloud-native data pipelines on AWS, Azure, or GCP.
• Exceptional analytical, system design, troubleshooting, and cross-functional communication capabilities.
• Professional availability for onsite employment in Malvern, PA.

Nice-to-Have Skills

• Professional certifications such as Databricks Certified Data Engineer Professional or Databricks Certified Solutions Architect.
• Advanced experience with orchestration frameworks (Apache Airflow, Databricks Workflows) and Terraform infrastructure automation.
• Prior working experience in financial services, healthcare, or enterprise SaaS consulting environments.
• Strong background in streaming data ingestion using Apache Kafka and structured streaming.
• Contributions to open-source data engineering projects or published technical papers on lakehouse architecture.

Application Information

• Salary/Rate: Competitive executive remuneration package commensurate with experience
• Deadline: Open until filled
• Notice Period: Immediate to short notice preferred
• Contract Duration: Permanent / Full-Time (Onsite in Malvern, PA)

Recruitment Pro Tip

When applying for senior Databricks architect roles, ensure your resume explicitly highlights your Unity Catalog governance experience, Delta Lake optimization track record, and big data pipeline design leadership.