Senior Data Engineer - Databricks & Spark Expert

TechLink Resources, Inc; Glendale (CA); On-site; USD 120,000 - 180,000; Full time

Job summary

TechLink Resources, Inc. is seeking a Senior Data Engineer to design and maintain scalable data platforms on AWS, leveraging Databricks, Spark, Python, and SQL to support batch and streaming workloads.

You will own data pipelines, optimize performance, and collaborate with engineering teams to translate business requirements into scalable technical solutions while ensuring governance and reliability across cloud-based architectures.

Qualifications

• 5+ years of professional data engineering experience.
• Hands-on Databricks experience required.
• Strong Python/Scala programming.
• Advanced SQL and performance tuning.
• Experience with AWS data infrastructure.
• Familiarity with Docker, Kubernetes, and cloud-based data architectures.
• Experience with Databricks governance, Unity Catalog, and access management.

Responsibilities

• Design, develop, and optimize scalable data pipelines using PySpark, Scala, Python, and SQL.
• Build and maintain batch and streaming pipelines using Databricks and Apache Spark.
• Manage Databricks platform administration, governance, access controls, and Unity Catalog.
• Develop and maintain workflow orchestration using Apache Airflow (MWAA).
• Optimize Spark performance, troubleshoot pipeline failures, and improve platform reliability.
• Design and support cloud-based data architectures across AWS, Kubernetes, and Databricks.
• Implement automation, CI/CD processes, monitoring, and data quality solutions.
• Collaborate with engineering teams and stakeholders to translate business requirements into scalable technical solutions.

Tools

Databricks

Apache Spark

Python

SQL

Airflow (MWAA)

AWS

Docker

Kubernetes

Unity Catalog

Job description

TechLink Resources, Inc. is seeking a Senior Data Engineer to design and maintain scalable data platforms on AWS, leveraging Databricks, Spark, Python, and SQL to support batch and streaming workloads.

You will own data pipelines, optimize performance, and collaborate with engineering teams to translate business requirements into scalable technical solutions while ensuring governance and reliability across cloud-based architectures.