Databricks Lakehouse Engineer for Scalable Data Pipelines

Tredence Inc.; San Jose (CA); On-site; USD 140,000 - 190,000; Full time

Job summary

Tredence Inc. is seeking a Databricks Developer to design, develop, and optimize scalable data pipelines on a Databricks Lakehouse platform. The role focuses on PySpark, Python, SQL, Delta Lake, and enterprise data modeling to support analytics, reporting, and AI/ML initiatives.

Responsibilities include building ETL/ELT processes, ensuring data quality and governance, and collaborating with stakeholders to deliver robust data solutions in a modern cloud environment.

Qualifications

• 4-8 years in data engineering and analytics.
• 3+ years hands-on Databricks and PySpark experience.
• Strong SQL development and data modeling skills.
• Experience implementing data quality and governance controls.
• Familiar with ETL/ELT pipelines and cloud data platforms.

Responsibilities

• Design and build scalable data pipelines on Databricks Lakehouse.
• Implement Delta Lake features: ACID, time travel, schema evolution.
• Collaborate with stakeholders to translate requirements into technical solutions.
• Optimize workloads for performance, scalability and cost.
• Govern data quality, lineage, metadata management and governance processes.
• Support cloud modernization and data platform initiatives.

Skills

Databricks

PySpark

Python

SQL

Delta Lake

Data Modeling

Data Quality

Data Governance

ETL/ELT

Data Pipelines

Education

Bachelor's or Master’s degree in Computer Science / Information Technology / Data Engineering

Tools

Databricks

Unity Catalog

Data Catalog

Azure Databricks

Azure Data Factory

Azure Data Lake Storage

Azure Synapse Analytics

CI/CD (Azure DevOps, GitHub Actions)

GitHub Actions

Job description

Tredence Inc. is seeking a Databricks Developer to design, develop, and optimize scalable data pipelines on a Databricks Lakehouse platform. The role focuses on PySpark, Python, SQL, Delta Lake, and enterprise data modeling to support analytics, reporting, and AI/ML initiatives.

Responsibilities include building ETL/ELT processes, ensuring data quality and governance, and collaborating with stakeholders to deliver robust data solutions in a modern cloud environment.