Senior Data Engineer - Databricks & Spark Pipelines

TIU Consulting LLC; Glendale (CA); Hybrid; USD 140,000 - 190,000; Full time

Job summary

TIU Consulting LLC in Glendale, CA, is hiring a data engineer for a hybrid role (2-4 days onsite). You will design and maintain large-scale data pipelines using Databricks and Spark, partner with stakeholders to translate requirements into scalable data platform solutions, and govern the platform with Unity Catalog, ACLs, and lineage.

The role requires 5+ years of experience, strong SQL, and proficiency with Airflow, AWS, Docker, and Kubernetes. Collaboration in an agile environment is essential.

Qualifications

• 5+ years of data engineering experience developing and operating large-scale data pipelines.
• Deep hands-on Databricks and Spark (batch and streaming) experience.
• Proficiency with SQL and advanced performance tuning.
• Experience with Airflow (MWAA) for orchestration.
• Familiar with CI/CD and DevOps practices (Git workflows).
• AWS experience; Kubernetes and containerization understanding.

Responsibilities

• Design, write, test, and deploy data pipelines using PySpark, Scala, SQL, Python.
• Translate stakeholder requirements into scalable data platform solutions.
• Govern Databricks platform: Unity Catalog, ACLs, lineage, privacy tooling.
• Explain Spark architecture and pipeline behavior to stakeholders.
• Provide solution architecture across AWS, Databricks, Kubernetes, and Airflow.
• Collaborate with engineers, architects, and product managers in agile/scrum.
• Maintain documentation of platform changes and pipeline configurations.

Skills

Databricks experience

Advanced SQL

Data pipelines and workflows

Education

Bachelor’s Degree in Computer Science or related field

Tools

Databricks

Apache Spark

Airflow (MWAA)

Docker

Kubernetes

Snowflake

Job description

TIU Consulting LLC in Glendale, CA, is hiring a data engineer for a hybrid role (2-4 days onsite). You will design and maintain large-scale data pipelines using Databricks and Spark, partner with stakeholders to translate requirements into scalable data platform solutions, and govern the platform with Unity Catalog, ACLs, and lineage.

The role requires 5+ years of experience, strong SQL, and proficiency with Airflow, AWS, Docker, and Kubernetes. Collaboration in an agile environment is essential.