Senior Databricks Engineer

HIKE2; United (PA); On-site; USD 135,000 - 185,000; Full time

Benefits offered by this job

Medical benefits

401(k)

Vacation

Job summary

HIKE2 is seeking a Senior Data Engineer to design and build enterprise-grade data platforms using Databricks in Pennsylvania. This role involves leading technical directions, collaborating with teams on data solutions, and optimizing performance across large-scale data environments.

The ideal candidate will have deep expertise in Databricks and experience with CI/CD automation. We're looking for someone who is passionate about guiding teams and ensuring the quality and efficiency of data engineering practices.

Qualifications

• Deep Databricks-native expertise required.
• Experience with end-to-end lakehouse solutions.
• Familiarity with infrastructure-as-code tools like Terraform.

Responsibilities

• Design and implement enterprise-grade data platforms.
• Optimize data pipelines for performance and reliability.
• Mentor and guide other engineers.

Skills

Databricks expertise

Delta Lake

Pipeline optimization

CI/CD automation

Terraform

Tools

Azure

Snowflake

Kafka

Job description

Who We Are

HIKE2 is an advisory and innovation partner helping organizations design and build what’s next.

We work with complex and regulated industries – including law firms, financial services, insurance, professional services, and high‑growth SaaS companies – to modernize operating models and embed data, automation, and AI into the core of how work gets done. Our clients are rethinking workflows, activating governed AI, and designing environments where humans and intelligent systems collaborate. We believe meaningful transformation happens when strategy, data, design, and engineering move together.

Job Description

Senior Data Engineer – Databricks & Modern Data Platforms

We are looking for a hands‑on engineer who can lead technical direction and support the growth of other engineers in large, complex enterprise environments.

You’ll work closely with clients and internal teams to design and implement enterprise‑grade data platforms and pipelines from the ground up and operate across the full lifecycle from architecture to delivery and optimization.

What You’ll Do

• Design and build large‑scale data platforms on Databricks (Delta Lake, Spark, Unity Catalog) in Azure.
• Develop and maintain batch and streaming data pipelines for high‑volume, complex data sources.
• Implement medallion/lakehouse architectures from the ground up in greenfield environments.
• Build and optimize data models to support analytics, reporting, and downstream applications.
• Integrate Databricks with enterprise systems (APIs, event streams, warehouses, ML workflows).
• Tune Spark jobs and pipelines for performance, reliability, and cost at scale.
• Support production deployments, including CI/CD pipelines, testing, and release management.

How You’ll Work

• Partner directly with enterprise clients to translate requirements into working technical solutions.
• Collaborate with architects, engineers, and data scientists across multiple workstreams.
• Balance speed and quality, knowing when to move fast and when to harden solutions.
• Make pragmatic decisions in ambiguous, evolving environments (especially greenfield builds).
• Contribute hands‑on while also guiding design and approach across the team.
• Communicate trade‑offs clearly to both technical and non‑technical stakeholders.
• Work within modern engineering practices (version control, code reviews, automated testing).
• Mentor and guide data engineers and analysts.

What You’ll Own

• End‑to‑end delivery of Databricks‑based data solutions—from design through production support.
• Technical direction and key architecture decisions for large‑scale implementations.
• Data pipeline reliability, monitoring, and incident response in production environments.
• Performance and cost efficiency of workloads running in Databricks and Azure.
• Data quality, governance alignment, and adherence to enterprise security standards.
• Reusable patterns, frameworks, and standards for scaling future implementations.
• Mentorship and technical development of other engineers on the team.

Desired Skills and Experience

• Deep Databricks‑native expertise, including advanced experience with Delta Lake, Unity Catalog, workflow orchestration, governance, performance tuning, and operational monitoring.
• Experience architecting and implementing end‑to‑end lakehouse solutions primarily on Databricks.
• Familiarity with infrastructure‑as‑code (Terraform, Bicep), environment provisioning, and CI/CD automation (GitHub, Azure DevOps) for Databricks‑based platforms.
• Strong learning agility, technical curiosity, and comfort using AI‑enabled development workflows or automation tools.
• Familiarity with other modern cloud data architectures and tools, including cloud‑native data warehouses (Snowflake, BigQuery, Redshift), data lakes, orchestration frameworks (Airflow, Astronomer), transformation tools (dbt), catalog/governance platforms, and scalable batch or streaming data processing services (Kafka, Kinesis).

Compensation and Benefits

Salary range: $135,000–185,000. Benefits include medical, dental, vision, 401(k), holiday pay, vacation, personal and family sick leave, and more.

Accommodation Statement

We are committed to providing reasonable accommodations for qualified individuals with disabilities and sincerely held religious beliefs throughout the hiring process. If you require an accommodation to complete the application or participate in the interview process, please let us know.

Background Check Notice

Employment may be contingent upon successful completion of a background check, conducted in compliance with applicable federal, state, and local laws.

US Citizenship Requirement

All applicants must reside in the U.S. and hold U.S. citizenship. HIKE2 cannot sponsor work authorizations or visas.

EEO Statement

We are an equal opportunity employer and are committed to creating an inclusive environment for all employees and applicants. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex (including pregnancy, childbirth, or related medical conditions), sexual orientation, gender identity or expression, national origin, age, disability, genetic information, protected veteran status, marital status or any other status protected by applicable federal, state, and local law.

Disclaimer

This job description is not intended to be an exhaustive list of duties, responsibilities, or qualifications and may be modified at any time to meet business needs.