Data Engineer: Scalable Pipelines with Airflow & Spark

Fierceli Inc, MBE, SBE

Sunnyvale (CA)

On-site

USD 110,000 - 180,000

Full time

3 days ago

Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Fierceli Inc is seeking a hands-on Data Engineer to design and maintain data ingestion and processing pipelines, consolidating data from multiple sources into a centralized repository. You will use Python, SQL, and Apache Airflow to build ETL/ELT processes, work with JSON, CSV, Parquet, Avro, and collaborate with infrastructure teams to ensure secure, scalable data movement.

Ideal candidates optimize pipelines for performance, monitor data accuracy, and document procedures while supporting

Qualifications

Hands-on experience building and maintaining data ingestion and processing pipelines.
Strong Python and SQL programming skills.
Experience with Apache Airflow for workflow orchestration and scheduling.
Experience with multiple data formats (JSON, CSV, Parquet, Avro).

Responsibilities

Design, develop, and maintain data ingestion pipelines into a centralized repository.
Build and optimize data processing workflows for large data volumes.
Ensure data governance, quality, security, and compliance.
Develop ETL/ELT pipelines using Python, SQL, and Airflow.
Work with Spark and Flink for distributed/stream processing.
Troubleshoot pipeline failures and resolve bottlenecks.
Document pipelines and processing procedures.