Hybrid Data Engineer: Databricks & Spark Platform Expert

Xforia Global Talent & Technology Solutions; Glendale (CA); Hybrid; USD 120,000 - 180,000; Full time

Job summary

Xforia Global Talent & Technology Solutions seeks a Senior Data Engineer in Glendale, CA, to design and optimize large-scale data pipelines and platform services. You will leverage Databricks, Spark, PySpark, and SQL to deliver scalable data solutions and work with stakeholders to meet evolving needs.

You’ll deploy on AWS with Kubernetes and Airflow, govern Databricks resources, and contribute to platform governance, security, and cost monitoring.

Qualifications

• 5+ years of data engineering experience building large-scale pipelines.
• Strong Databricks and Spark (batch and streaming) experience.
• Experience with PySpark/Scala for pipeline development.
• Proficiency in SQL with performance tuning capabilities.
• Hands-on with Airflow for orchestration and MWAA familiarity.
• Experience with AWS, Docker, Kubernetes, and Snowflake.

Responsibilities

• Design, write, test, and deploy data pipelines using PySpark, Scala, SQL, Python.
• Collaborate with stakeholders to translate requirements into scalable data platform solutions.
• Govern Databricks: Unity Catalog, ACLs, lineage, data discovery and privacy tooling.
• Operate Kubernetes containers and containerized utilities for data services.
• Provide solution architecture across AWS, Databricks, Kubernetes, and Airflow.
• Audit platform activity, diagnose errors, and automate updates across pipelines.
• Explain Spark architecture to stakeholders and drive data platform success.
• Maintain platform documentation, standards, and pipeline configurations.

Skills

Databricks experience

Advanced SQL

PySpark

Scala

Airflow (MWAA)

Kubernetes

Docker

AWS

Snowflake

CI/CD / DevOps

Data quality & monitoring

Spark architecture & performance

Python

Scrum/Agile

Education

Bachelor’s Degree in Computer Science, Information Systems, or related field

Tools

Databricks

Apache Spark

Unity Catalog

MLflow / tooling

Git

Job description

Xforia Global Talent & Technology Solutions seeks a Senior Data Engineer in Glendale, CA, to design and optimize large-scale data pipelines and platform services. You will leverage Databricks, Spark, PySpark, and SQL to deliver scalable data solutions and work with stakeholders to meet evolving needs.

You’ll deploy on AWS with Kubernetes and Airflow, govern Databricks resources, and contribute to platform governance, security, and cost monitoring.