Data Engineer, Databricks + GCP Automation Pipelines
AgileEngine, LLC.
Una candidatura hecha para este puesto de trabajo — un currículum y una carta de presentación adaptados que responden directamente a la oferta.
Ventajas ofrecidas por este puesto de trabajo
Descripción de la vacante
AgileEngine is seeking a Data Engineer with strong Databricks and GCP experience to develop scalable ETL/ELT pipelines and productionize agentic workflows for data engineering automation.
You will design, build, and optimize pipelines using PySpark, SQL, and Delta Lake, integrating with BigQuery and GCP environments while driving data quality and production readiness across multiple data domains.
Formación
- 5+ years hands-on Databricks and PySpark experience.
- Advanced SQL and data-processing skills are required.
- GCP experience, especially BigQuery.
- Experience with Delta Lake and data-lake architectures.
- Proven ability to build reliable, scalable data pipelines.
- Strong analytical and troubleshooting skills.
Responsabilidades
- Design, develop, and optimize scalable ETL/ELT pipelines using Databricks, PySpark, and SQL.
- Operationalize agentic workflows to automate data engineering tasks.
- Integrate with Databricks, GCP, BigQuery, and Delta Lake environments.
- Develop pipelines to support new datasets and business needs.
- Build reusable data engineering frameworks for multiple use cases.
- Implement data quality checks, monitoring, and production controls.
- Optimize PySpark/SQL workloads for performance and reliability.
- Support testing, deployment, and productionization of data solutions.
- Troubleshoot complex data issues and implement durable fixes.
- Collaborate with stakeholders to identify automation opportunities.
Conocimientos
Herramientas
Descripción del empleo
AgileEngine is seeking a Data Engineer with strong Databricks and GCP experience to develop scalable ETL/ELT pipelines and productionize agentic workflows for data engineering automation.
You will design, build, and optimize pipelines using PySpark, SQL, and Delta Lake, integrating with BigQuery and GCP environments while driving data quality and production readiness across multiple data domains.
'/%3e%3cpath%20d='M101.744%2012.4726L102.894%2016.1947L99.0021%2013.2201L101.744%2012.4726ZM105.3%208.40937H100.488L99.0026%203.59473L97.5116%208.41047L92.7%208.40385L96.5965%2011.3834L95.1055%2016.1942L99.0021%2013.2196L101.408%2011.3834L105.3%208.40992V8.40937Z'%20fill='white'/%3e%3cdefs%3e%3clinearGradient%20id='paint0_linear_5593_1381'%20x1='90'%20y1='9.44471'%20x2='99'%20y2='9.44471'%20gradientUnits='userSpaceOnUse'%3e%3cstop%20stop-color='%2300AD70'/%3e%3cstop%20offset='0.998594'%20stop-color='%2300B67A'/%3e%3cstop%20offset='1'%20stop-color='%23DCDCE6'/%3e%3c/linearGradient%3e%3c/defs%3e%3c/svg%3e){kind=small}
