Senior Data Engineer - GCP, BigQuery & Airflow (Onsite)
Factspan
Aumenta le possibilità di ricevere una risposta da questo datore di lavoro
Il tuo curriculum e una lettera di presentazione personalizzati, che corrispondono esattamente a ciò che sta cercando.
Descrizione del lavoro
Factspan seeks a senior Data Engineer for an onsite Atlanta role to support a Java Spring Boot based data platform. You will design scalable pipelines and collaborate across teams to ensure production readiness.
You will build data ingestion, transformation, and orchestration on Google Cloud (BigQuery, Dataflow, Cloud Composer) and Airflow, while implementing data quality checks and performance tuning.
Competenze
- Must have expertise in Java-based Spring Boot applications.
- Strong hands-on experience with GCP services: BigQuery, Dataflow, GCS, Cloud Composer.
- Proficient in SQL for data profiling and optimization.
- Proficient in Python for pipeline automation and orchestration.
- Experience building batch and streaming data pipelines.
Mansioni
- Design and develop data ingestion, transformation, and orchestration pipelines using GCP and Airflow.
- Implement and maintain data integrations between MongoDB and BigQuery for performance and scalability.
- Build and optimize ETL/ELT workflows for warehousing.
- Implement data quality checks, monitoring, and alerting frameworks.
- Translate requirements into data pipelines with Lead DE and SME.
- Support performance tuning, data reconciliation, and defect resolution during testing.
- Collaborate with QA for end-to-end validation and production readiness.
- Participate in daily Agile ceremonies and sprint planning.
Conoscenze
Formazione
Descrizione del lavoro
Factspan seeks a senior Data Engineer for an onsite Atlanta role to support a Java Spring Boot based data platform. You will design scalable pipelines and collaborate across teams to ensure production readiness.
You will build data ingestion, transformation, and orchestration on Google Cloud (BigQuery, Dataflow, Cloud Composer) and Airflow, while implementing data quality checks and performance tuning.
'/%3e%3cpath%20d='M101.744%2012.4726L102.894%2016.1947L99.0021%2013.2201L101.744%2012.4726ZM105.3%208.40937H100.488L99.0026%203.59473L97.5116%208.41047L92.7%208.40385L96.5965%2011.3834L95.1055%2016.1942L99.0021%2013.2196L101.408%2011.3834L105.3%208.40992V8.40937Z'%20fill='white'/%3e%3cdefs%3e%3clinearGradient%20id='paint0_linear_5593_1381'%20x1='90'%20y1='9.44471'%20x2='99'%20y2='9.44471'%20gradientUnits='userSpaceOnUse'%3e%3cstop%20stop-color='%2300AD70'/%3e%3cstop%20offset='0.998594'%20stop-color='%2300B67A'/%3e%3cstop%20offset='1'%20stop-color='%23DCDCE6'/%3e%3c/linearGradient%3e%3c/defs%3e%3c/svg%3e){kind=small}
