Remote Apache Hudi Engineer - Real-Time Data Lake
BairesDev
Get a reply from this employer
Your resume and cover letter, tailored to exactly what they’re hiring for.
Benefits offered by this job
Job summary
BairesDev is seeking a Software Engineer specializing in Apache Hudi to design and manage large-scale incremental data processing on data lakes. You will develop efficient pipelines handling upserts, deletes, and CDC, while optimizing storage formats to boost query performance and reduce latency.
The role emphasizes ACID transactions, schema evolution, and collaboration with data teams to integrate Hudi with streaming and batch engines. Excellent English skills required.
Qualifications
- 4+ years of software or data engineering experience.
- Proven expertise with Apache Hudi in incremental data processing.
- Experience with upserts, deletes, and streaming ingestion into data lake storage.
- Strong understanding of data lakehouse architectures and distributed systems.
- Advanced English proficiency.
Responsibilities
- Design, implement, and manage Apache Hudi tables for large-scale incremental processing.
- Build data pipelines handling high-frequency upserts, deletes, and CDC.
- Optimize storage formats and indexing to reduce data-lake latency.
- Implement ACID transactions and schema evolution for consistency.
- Collaborate with data teams to integrate Hudi with streaming and batch engines.
- Monitor and tune table services like compaction and clustering.
Skills
Tools
Job description
BairesDev is seeking a Software Engineer specializing in Apache Hudi to design and manage large-scale incremental data processing on data lakes. You will develop efficient pipelines handling upserts, deletes, and CDC, while optimizing storage formats to boost query performance and reduce latency.
The role emphasizes ACID transactions, schema evolution, and collaboration with data teams to integrate Hudi with streaming and batch engines. Excellent English skills required.
'/%3e%3cpath%20d='M101.744%2012.4726L102.894%2016.1947L99.0021%2013.2201L101.744%2012.4726ZM105.3%208.40937H100.488L99.0026%203.59473L97.5116%208.41047L92.7%208.40385L96.5965%2011.3834L95.1055%2016.1942L99.0021%2013.2196L101.408%2011.3834L105.3%208.40992V8.40937Z'%20fill='white'/%3e%3cdefs%3e%3clinearGradient%20id='paint0_linear_5593_1381'%20x1='90'%20y1='9.44471'%20x2='99'%20y2='9.44471'%20gradientUnits='userSpaceOnUse'%3e%3cstop%20stop-color='%2300AD70'/%3e%3cstop%20offset='0.998594'%20stop-color='%2300B67A'/%3e%3cstop%20offset='1'%20stop-color='%23DCDCE6'/%3e%3c/linearGradient%3e%3c/defs%3e%3c/svg%3e){kind=small}
