Big Data Engineer: Databricks, ETL & Cloud Pipelines

Agilisium

Westlake Village (CA)

On-site

USD 150,000 - 190,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Agilisium is seeking a Big Data Specialist to design and optimize data pipelines across Databricks, PySpark, and AWS. This role involves working on client sites nationwide with potential relocation paid by the employer.

You will implement ETL/ELT workflows, ensure data quality, and lead data integration efforts. A strong background in SQL optimization, governance, and scalable architectures is required.

Qualifications

  • Bachelor’s degree in CS/CE or related field.

Responsibilities

  • Develop Databricks notebooks to implement Snaplogic data cleansing and processing.
  • Optimize and manage data manipulation tasks.
  • Implement and automate data flow orchestration.
  • Design modular, standardized pipelines to meet client performance expectations.
  • Develop validation processes to ensure data integrity.
  • Establish a standardized framework for CDL layers for scalability.
  • Create reusable code modules and maintain version control.
  • Lead data integration to drive system improvements.
  • Develop automated processes for ingesting Google Analytics data.

Skills

Data pipeline design
PySpark
Databricks
SQL optimization
ETL/ELT design
GitLab CI/CD
Data governance
Cloud data architecture (AWS)

Education

Bachelor’s degree in Computer Science, Computer Engineering, or related field

Tools

Databricks
PySpark
GitLab
Amazon Redshift
Airflow
SnapLogic
Informatica PowerCenter
AWS S3
AWS EC2
AWS Kinesis
AWS Lambda
Delta Lake
Parquet

Job description

Agilisium is seeking a Big Data Specialist to design and optimize data pipelines across Databricks, PySpark, and AWS. This role involves working on client sites nationwide with potential relocation paid by the employer.

You will implement ETL/ELT workflows, ensure data quality, and lead data integration efforts. A strong background in SQL optimization, governance, and scalable architectures is required.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Big Data Engineer - Databricks & Data Pipelines
Big Data Engineer - Databricks & Data Pipelines

Agilisium Onshore • Westlake Village (CA)

On-site
USD 140,000 - 210,000
Data Engineer - Databricks + ADF Data Pipelines
Data Engineer - Databricks + ADF Data Pipelines

Ethereum Technologies LLC • Plano (TX)

On-site
USD 118,000 - 136,000
Remote Databricks Data Engineer for Scalable Pipelines
Remote Databricks Data Engineer for Scalable Pipelines

Eliassen Group • Madison (WI)

Remote
USD 150,000 - 154,000
Comprehensive benefits
Remote work
Remote Databricks Data Engineer — Pipelines & Data Lakes
Remote Databricks Data Engineer — Pipelines & Data Lakes

Eliassen Group • Olympia (WA)

Remote
USD 150,000 - 154,000
Medical insurance
Dental insurance
Vision insurance
+4
Databricks Data Engineer — Pipelines & Big Data
Databricks Data Engineer — Pipelines & Big Data

Cyber Space Technologies LLC • Columbus (OH)

On-site
USD 90,000 - 115,000
Cloud Data Engineer: AWS, Databricks & PySpark Pipelines
Cloud Data Engineer: AWS, Databricks & PySpark Pipelines

Advanced Tech Placement • Atlanta (GA)

On-site
USD 120,000 - 150,000
Senior AWS Data Engineer - Databricks & DMS Pipelines
Senior AWS Data Engineer - Databricks & DMS Pipelines

CloudBigD • Wilmington (DE)

On-site
USD 120,000 - 180,000
Senior Databricks Data Engineer - Cloud Pipelines
Senior Databricks Data Engineer - Cloud Pipelines

Alephys LLC • United States

On-site
USD 120,000 - 190,000
Databricks Data Engineer - Azure, CDC Pipelines, PySpark
Databricks Data Engineer - Azure, CDC Pipelines, PySpark

CloudBigD • Phoenix (AZ)

On-site
USD 81,000 - 115,000
Hybrid Data Engineer: Databricks Pipelines & ETL
Hybrid Data Engineer: Databricks Pipelines & ETL

Infinitive • North Dakota

Hybrid
USD 75,000 - 120,000