Senior AWS Data Engineer - Remote, Lakehouse & Automation
BairesDev
A complete application in a minute
Tailored resume and cover letter, ready to send.
Benefits offered by this job
Job summary
BairesDev is seeking a Senior AWS Data Engineer to re-engineer an existing ingestion and lakehouse platform and redesign retention, compaction, and recovery for automated restores. This role offers 100% remote work and opportunities to tackle production-scale challenges across Lambda, Glue, EventBridge, and Step Functions.
You will work with Apache Iceberg tables and Kafka on MSK, designing automated recovery, retention, and write-audit-publish patterns.
Qualifications
- 5+ years of data engineering experience with AWS.
- Strong experience managing Apache Iceberg tables.
- Experience with AWS Glue ETL.
- Proficiency in Python.
- Experience with Kafka (MSK).
- Strong code-reading and re-engineering skills for existing systems.
Responsibilities
- Read and re-engineer existing ingestion and lakehouse code across Lambda, Glue, EventBridge, and Step Functions.
- Manage Apache Iceberg tables in the Gold zone, including retention and compaction strategy.
- Redesign recovery processes to automate restores that were previously manual.
- Introduce write-audit-publish patterns to prevent bad writes from landing silently.
- Work with Kafka on MSK for event processing across the pipeline.
Skills
Tools
Job description
BairesDev is seeking a Senior AWS Data Engineer to re-engineer an existing ingestion and lakehouse platform and redesign retention, compaction, and recovery for automated restores. This role offers 100% remote work and opportunities to tackle production-scale challenges across Lambda, Glue, EventBridge, and Step Functions.
You will work with Apache Iceberg tables and Kafka on MSK, designing automated recovery, retention, and write-audit-publish patterns.
'/%3e%3cpath%20d='M101.744%2012.4726L102.894%2016.1947L99.0021%2013.2201L101.744%2012.4726ZM105.3%208.40937H100.488L99.0026%203.59473L97.5116%208.41047L92.7%208.40385L96.5965%2011.3834L95.1055%2016.1942L99.0021%2013.2196L101.408%2011.3834L105.3%208.40992V8.40937Z'%20fill='white'/%3e%3cdefs%3e%3clinearGradient%20id='paint0_linear_5593_1381'%20x1='90'%20y1='9.44471'%20x2='99'%20y2='9.44471'%20gradientUnits='userSpaceOnUse'%3e%3cstop%20stop-color='%2300AD70'/%3e%3cstop%20offset='0.998594'%20stop-color='%2300B67A'/%3e%3cstop%20offset='1'%20stop-color='%23DCDCE6'/%3e%3c/linearGradient%3e%3c/defs%3e%3c/svg%3e){kind=small}
