Big Data Specialist (Multiple Positions) | ZR_925_JOB

Agilisium

Westlake Village (CA)

Presencial

USD 150 000 - 190 000

Tempo integral

14 dias+

Uma candidatura completa num minuto

Currículo e carta de apresentação personalizados, prontos a enviar.

Ultrapassa os filtros ATS

Resumo da oferta

Agilisium is seeking a Big Data Specialist to design and optimize data pipelines across Databricks, PySpark, and AWS. This role involves working on client sites nationwide with potential relocation paid by the employer.

You will implement ETL/ELT workflows, ensure data quality, and lead data integration efforts. A strong background in SQL optimization, governance, and scalable architectures is required.

Qualificações

  • Bachelor’s degree in CS/CE or related field.

Responsabilidades

  • Develop Databricks notebooks to implement Snaplogic data cleansing and processing.
  • Optimize and manage data manipulation tasks.
  • Implement and automate data flow orchestration.
  • Design modular, standardized pipelines to meet client performance expectations.
  • Develop validation processes to ensure data integrity.
  • Establish a standardized framework for CDL layers for scalability.
  • Create reusable code modules and maintain version control.
  • Lead data integration to drive system improvements.
  • Develop automated processes for ingesting Google Analytics data.

Conhecimentos

Data pipeline design
PySpark
Databricks
SQL optimization
ETL/ELT design
GitLab CI/CD
Data governance
Cloud data architecture (AWS)

Formação académica

Bachelor’s degree in Computer Science, Computer Engineering, or related field

Ferramentas

Databricks
PySpark
GitLab
Amazon Redshift
Airflow
SnapLogic
Informatica PowerCenter
AWS S3
AWS EC2
AWS Kinesis
AWS Lambda
Delta Lake
Parquet

Descrição da oferta de emprego

Big Data Specialist (Multiple Positions) | ZR_925_JOB

Westlake Village, United States | Posted on 11/04/2026

TITLE: Big Data Specialist (Multiple Positions)

LOCATION: Westlake Village, CA and various and unanticipated locations throughout the U.S. (Must be willing to work anywhere in the U.S. as the position may involve relocation to various and unanticipated client site locations; any relocation to be paid by employer pursuant to internal policy.)

Duties
  • Develop Databricks notebooks to implement Snaplogic pipeline data cleansing and processing.
  • Optimize and manage data manipulation tasks.
  • Implement and automate data flow orchestration.
  • Design and implement modular, standardized pipelines to meet client performance expectations.
  • Develop and execute comprehensive validation processes to ensure data integrity throughout the pipeline.
  • Establish a standardized framework for multiple CDL layers, ensuring scalability and consistency.
  • Create reusable code modules and maintain version control practices.
  • Lead data integration, to drive system improvements.
  • Develop automated processes for ingesting Google Analytics data, enabling actionable insights.
  • EOE
Requirements
  • Must have a Bachelor’s degree or foreign equivalent in Computer Science, Computer Engineering, or a related field plus five (5) years of experience in the position offered, as a Software Engineer, or a related position.
  • Must have five (5) years of experience with all of the following: Developing and optimizing data pipelines in PySpark and Databricks for large-scale data ingestion and processing, integrating GitLab version control for code management and automated deployments; Implementing advanced SQL development and optimization techniques for complex data modeling, query performance tuning, and relational warehouse management in Amazon Redshift and SQL DW; Engineering and automating ETL/ELT workflows leveraging Apache Airflow, SnapLogic, and Informatica PowerCenter, ensuring reliability, reusability, and end-to-end data integration across heterogeneous systems; Architecting and operating cloud-native big-data ecosystems on AWS, integrating services including AWS S3 for storage, AWS EC2 for compute, AWS Kinesis for real-time data streaming, and AWS Lambda for event-driven processing to enable low-latency ingestion and analytics pipelines; Designing and implementing data quality and governance frameworks within the Medallion Architecture (Bronze, Silver, and Gold layers) to ensure data reliability, lineage, and compliance with enterprise governance standards; Implementing Delta Lake optimization strategies, including Z-Ordering, OPTIMIZE, and VACUUM, to improve query performance, data compaction, and storage efficiency, while designing partition pruning and columnar data formats (Parquet) to minimize I/O overhead and accelerate analytical workloads; and Architecting and optimizing dimensional and relational data models using Star and Snowflake schemas within Databricks and Amazon Redshift, ensuring scalable performance, schema consistency, and efficient analytical processing across large datasets.
Obtém a tua avaliação gratuita e confidencial do currículo.

ou arrasta e larga o ficheiro aqui.

Similar jobs

Ofertas semelhantes que vale a pena comparar

Big Data Specialist (Multiple Positions) | ZR_925_JOB
Big Data Specialist (Multiple Positions) | ZR_925_JOB

Agilisium Onshore • Westlake Village (CA)

Presencial
USD 140 000 - 210 000
Big Data Specialist
Big Data Specialist

Hyrhub • New York (NY)

Presencial
USD 150 000 - 180 000
Databricks Data Engineer
Databricks Data Engineer

Compunnel Inc. • Spring (TX)

Presencial
USD 110 000 - 140 000
Senior Data Engineer – Databricks & AWS
Senior Data Engineer – Databricks & AWS

Robotic Process Automation LLC • Sunnyvale (CA)

Presencial
USD 83 000 - 102 000
Senior Data Engineer – Databricks & AWS
Senior Data Engineer – Databricks & AWS

Valiant Technologies LLC • Sunnyvale (CA)

Presencial
USD 83 000 - 103 000
Senior Data Engineer – Databricks & AWS
Senior Data Engineer – Databricks & AWS

Ethereum Technologies LLC • Sunnyvale (CA)

Presencial
USD 76 000 - 108 000
Senior Data Engineer – Databricks & AWS
Senior Data Engineer – Databricks & AWS

Quantum Technologies LLC • Sunnyvale (CA)

Presencial
USD 76 000 - 108 000
Databricks Data Engineer
Databricks Data Engineer

Valiant Technologies LLC • Dallas (TX)

Presencial
USD 120 000 - 190 000
Databricks Data Engineer
Databricks Data Engineer

Ethereum Technologies LLC • Dallas (TX)

Presencial
USD 120 000 - 170 000
Databricks Data Engineer
Databricks Data Engineer

Robotic Process Automation LLC • Dallas (TX)

Presencial
USD 140 000 - 180 000