Data Engineer: Databricks + GCP Automation Pipelines

AgileEngine, LLC.; Porto; Híbrido; EUR 88 000 - 159 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

Professional growth

Competitive USD-based compensation

Projects with Fortune 500 clients

Flextime / hybrid work model

Resumo da oferta

AgileEngine is seeking a Data Engineer with strong Databricks and GCP experience to develop scalable ETL/ELT pipelines and productionize agentic workflows for data engineering automation.

You will design and optimize pipelines using Databricks, PySpark and SQL, integrate with BigQuery and Delta Lake, and build reusable components to support multiple use cases while ensuring data quality and production readiness.

Qualificações

• 5+ years hands-on Databricks + PySpark experience.
• Advanced SQL and data-processing skills.
• GCP experience, BigQuery.
• Delta Lake and modern lakehouse architectures.
• ETL/ELT, data pipeline design and optimization.
• Production-grade data solutions experience.
• Strong analytical and troubleshooting skills.
• Software engineering practices: testing, version control, deployment, monitoring, production support.

Responsabilidades

• Design, develop, and optimize scalable ETL/ELT data pipelines using Databricks, PySpark, and SQL.
• Build and operationalize agentic workflows to automate data engineering and operational processes.
• Integrate agentic capabilities with Databricks, GCP, BigQuery, and Delta Lake environments.
• Develop data pipelines and processing solutions for new business requirements and datasets.
• Create reusable frameworks and components across data engineering use cases.
• Implement data quality checks, monitoring, validation, and production controls.
• Optimize PySpark and SQL workloads for performance and scalability.
• Support testing, deployment, productionization, and ongoing enhancement of data solutions.

Conhecimentos

Databricks & PySpark experience

Advanced SQL

GCP BigQuery exposure

Delta Lake knowledge

ETL/ELT design & optimization

Production-grade data solutions

Analytical & troubleshooting skills

Software engineering practices

Ferramentas

BigQuery

Delta Lake

Descrição da oferta de emprego

AgileEngine is seeking a Data Engineer with strong Databricks and GCP experience to develop scalable ETL/ELT pipelines and productionize agentic workflows for data engineering automation.

You will design and optimize pipelines using Databricks, PySpark and SQL, integrate with BigQuery and Delta Lake, and build reusable components to support multiple use cases while ensuring data quality and production readiness.