Data Engineer

Fierceli Inc, MBE, SBE; Sunnyvale (CA); Sur place; USD 110 000 - 180 000; Plein temps

Résumé du poste

Fierceli Inc is seeking a hands-on Data Engineer to design and maintain data ingestion and processing pipelines, consolidating data from multiple sources into a centralized repository. You will use Python, SQL, and Apache Airflow to build ETL/ELT processes, work with JSON, CSV, Parquet, Avro, and collaborate with infrastructure teams to ensure secure, scalable data movement.

Ideal candidates optimize pipelines for performance, monitor data accuracy, and document procedures while supporting

Qualifications

• Hands-on experience building and maintaining data ingestion and processing pipelines.
• Strong Python and SQL programming skills.
• Experience with Apache Airflow for workflow orchestration and scheduling.
• Experience with multiple data formats (JSON, CSV, Parquet, Avro).

Responsabilités

• Design, develop, and maintain data ingestion pipelines into a centralized repository.
• Build and optimize data processing workflows for large data volumes.
• Ensure data governance, quality, security, and compliance.
• Develop ETL/ELT pipelines using Python, SQL, and Airflow.
• Work with Spark and Flink for distributed/stream processing.
• Troubleshoot pipeline failures and resolve bottlenecks.
• Document pipelines and processing procedures.

Connaissances

Python

SQL

Data pipelines

Troubleshooting

Problem solving

Data governance

Outils

Apache Airflow

Apache Spark

Apache Flink

JSON

CSV

Parquet

Avro

Description du poste

We are seeking a skilled Data Engineer to develop and maintain data ingestion and processing pipelines that consolidate data from multiple sources and formats into a centralized repository. The ideal candidate will have hands‑on experience with data engineering, data processing frameworks, workflow orchestration, and infrastructure fundamentals to ensure reliable, scalable, and compliant data integration.

Key Responsibilities

• Design, develop, and maintain data ingestion pipelines to collect and integrate data from multiple sources into a centralized repository.
• Build and optimize data processing workflows to handle diverse data formats and large volumes of data.
• Ensure data ingestion and processing adhere to applicable data governance, quality, security, and compliance requirements.
• Develop and maintain ETL/ELT pipelines using Python, SQL, and Apache Airflow.
• Work with distributed data processing technologies such as Apache Spark and Apache Flink.
• Troubleshoot data pipeline failures, investigate data quality issues, and resolve processing bottlenecks.
• Collaborate with infrastructure, networking, and platform teams to support reliable data movement and integration.
• Monitor pipeline performance, data accuracy, and system reliability.
• Optimize data workflows for scalability, efficiency, and operational stability.
• Document data pipelines, processing logic, and technical procedures.

Required Skills and Qualifications

• Strong programming experience with Python and SQL.
• Hands‑on experience building and maintaining data ingestion and processing pipelines.
• Experience with Apache Airflow for workflow orchestration and scheduling.
• Experience working with multiple data formats, such as JSON, CSV, Parquet, Avro, or similar.
• Understanding of ETL/ELT processes, data integration, and centralized data repositories.
• Knowledge of data engineering infrastructure, networking fundamentals, and data transfer mechanisms.
• Strong troubleshooting, debugging, and problem‑solving skills.

Preferred / Plus Skills

• Experience with Apache Spark for distributed data processing.
• Experience with Apache Flink for stream processing.
• Experience with additional data querying, processing, and analytics tools.
• Familiarity with big data technologies and scalable data platforms.
• Experience with data governance, compliance, and data quality controls.

Ideal Candidate

The ideal candidate is a hands‑on Data Engineer who can build reliable data pipelines, integrate data from heterogeneous sources, and deliver compliant, high‑quality data into a centralized repository. They should be comfortable working across data processing, orchestration, infrastructure, and networking environments.