Databricks Data Engineer - Hybrid, Spark & Lakehouse

Techgene Solutions; Pasadena (CA); Hybrid; USD 120,000 - 180,000; Full time

Job summary

Techgene Solutions in Pasadena, CA is seeking an experienced Data Engineer to design and maintain scalable data pipelines using Databricks, PySpark, and Python. The role focuses on ETL/ELT, data lakehouse architectures, and cloud-based processing.

You will collaborate with Data Architects, Data Scientists, and BI developers, implement CI/CD for data code, and ensure data quality, governance, and security across pipelines.

Qualifications

• Proven experience building scalable data pipelines with Databricks and PySpark.
• Strong Python development skills for data processing workflows.
• Experience with Delta Lake and data lakehouse architectures.
• Hands-on knowledge of Spark performance tuning and optimization.
• Experience implementing CI/CD for data engineering code and git workflows.

Responsibilities

• Design, develop, and maintain scalable data pipelines using Databricks, PySpark, and Python.
• Develop ETL/ELT workflows to ingest, transform, cleanse, and integrate data from multiple sources.
• Build and optimize data processing jobs using PySpark and Spark SQL.
• Work with Delta Lake for storage, transformation, and incremental processing.
• Collaborate with data architects and stakeholders to understand data requirements.

Skills

Databricks

PySpark

Python

Delta Lake

Spark SQL

Git

CI/CD

Cloud Platforms (AWS/GCP/Azure)

Job description

Techgene Solutions in Pasadena, CA is seeking an experienced Data Engineer to design and maintain scalable data pipelines using Databricks, PySpark, and Python. The role focuses on ETL/ELT, data lakehouse architectures, and cloud-based processing.

You will collaborate with Data Architects, Data Scientists, and BI developers, implement CI/CD for data code, and ensure data quality, governance, and security across pipelines.