Data Engineer - Databricks, GCP & AI-Driven Pipelines

AgileEngine, LLC.; Lisboa; Híbrido; EUR 70 000 - 100 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

Professional growth

Competitive USD-based compensation

Exciting projects

Flextime

Resumo da oferta

AgileEngine, LLC. is seeking a Data Engineer in Lisbon to design and operate scalable ETL/ELT pipelines using Databricks and PySpark, integrating with GCP and Delta Lake. You will build agentic workflows for automated data engineering and production processes.

The role emphasizes data quality, performance tuning, and collaboration across data, platform, and business teams to extend automation capabilities and support new data requirements.

Qualificações

• 5+ years hands-on Databricks and PySpark experience.
• Advanced SQL and data-processing skills.
• Hands-on GCP experience, especially BigQuery.
• Experience with Delta Lake and modern data lake/lakehouse architectures.
• Strong ETL/ELT, data pipeline design, and data quality focus.
• Experience building reliable, scalable production-grade data solutions.
• Strong analytical and troubleshooting skills.
• Understanding of testing, version control, deployment, monitoring, and production support

Responsabilidades

• Design, develop, and optimize scalable ETL/ELT data pipelines using Databricks, PySpark, and SQL.
• Build and operationalize agentic workflows to automate data engineering and operational processes.
• Integrate agentic capabilities with Databricks, GCP, BigQuery, and Delta Lake environments.
• Develop data pipelines to support new business datasets and requirements.
• Create reusable data engineering frameworks and components.
• Implement data quality checks, monitoring, and production controls.
• Optimize PySpark and SQL workloads for performance and reliability.
• Support testing, deployment, productionization, and ongoing enhancements.
• Troubleshoot complex data and production issues and implement sustainable solutions.
• Collaborate with business, data engineering, and platform teams to identify automation opportunities

Conhecimentos

Databricks

PySpark

SQL

GCP BigQuery

Delta Lake

ETL/ELT

Production-grade data solutions

Troubleshooting

Data pipelines

Software engineering practices

Descrição da oferta de emprego

AgileEngine, LLC. is seeking a Data Engineer in Lisbon to design and operate scalable ETL/ELT pipelines using Databricks and PySpark, integrating with GCP and Delta Lake. You will build agentic workflows for automated data engineering and production processes.

The role emphasizes data quality, performance tuning, and collaboration across data, platform, and business teams to extend automation capabilities and support new data requirements.