Senior Data Engineer - Databricks, PySpark & GCP | Flexible
AgileEngine, LLC.
An application made for this job — a tailored resume and cover letter that speak straight to the posting.
Benefits offered by this job
Job summary
AgileEngine, LLC. is seeking a Data Engineer with strong Databricks and GCP experience to design and implement scalable ETL/ELT pipelines and agentic workflows.
You will integrate Databricks, BigQuery, and Delta Lake environments, build reusable data frameworks, and ensure data quality and production stability. The role emphasizes performance optimization, collaboration with cross-functional teams, and ongoing automation of data engineering processes to support diverse datasets and business
Qualifications
- 5+ years of hands-on experience with Databricks and PySpark
- Advanced SQL and data-processing skills
- GCP experience, particularly BigQuery
- Experience with Delta Lake and modern data lake architectures
- Strong ETL/ELT design and data quality
- Production-grade data solutions experience
- Strong analytical and troubleshooting skills
- Software engineering practices (testing, version control, deployment, monitoring)
Responsibilities
- Design, develop, and optimize scalable ETL/ELT data pipelines using Databricks, PySpark, and SQL
- Build and operationalize agentic workflows to automate data engineering and operational processes
- Integrate agentic capabilities with Databricks, GCP, BigQuery, and Delta Lake environments
- Develop data pipelines and processing solutions for new business requirements
- Build reusable data engineering frameworks and platform components
- Implement data quality checks, monitoring, and production controls
- Optimize PySpark and SQL workloads for performance and scalability
- Support testing, deployment, and productionization of data solutions
- Troubleshoot complex data and production issues and implement sustainable solutions
- Collaborate with business, data engineering, and platform teams to identify automation opportunities
Skills
Tools
Job description
AgileEngine, LLC. is seeking a Data Engineer with strong Databricks and GCP experience to design and implement scalable ETL/ELT pipelines and agentic workflows.
You will integrate Databricks, BigQuery, and Delta Lake environments, build reusable data frameworks, and ensure data quality and production stability. The role emphasizes performance optimization, collaboration with cross-functional teams, and ongoing automation of data engineering processes to support diverse datasets and business
'/%3e%3cpath%20d='M101.744%2012.4726L102.894%2016.1947L99.0021%2013.2201L101.744%2012.4726ZM105.3%208.40937H100.488L99.0026%203.59473L97.5116%208.41047L92.7%208.40385L96.5965%2011.3834L95.1055%2016.1942L99.0021%2013.2196L101.408%2011.3834L105.3%208.40992V8.40937Z'%20fill='white'/%3e%3cdefs%3e%3clinearGradient%20id='paint0_linear_5593_1381'%20x1='90'%20y1='9.44471'%20x2='99'%20y2='9.44471'%20gradientUnits='userSpaceOnUse'%3e%3cstop%20stop-color='%2300AD70'/%3e%3cstop%20offset='0.998594'%20stop-color='%2300B67A'/%3e%3cstop%20offset='1'%20stop-color='%23DCDCE6'/%3e%3c/linearGradient%3e%3c/defs%3e%3c/svg%3e){kind=small}
