Data Engineer: Databricks, GCP & Automated Pipelines

AgileEngine, LLC.; Województwo pomorskie; Hybrid; PLN 473,000 - 709,000; Full time

Benefits offered by this job

Professional growth and mentorship

Competitive USD-based compensation

Exciting projects with Fortune 500/Top

Flextime: remote and office options

Job summary

AgileEngine, LLC. is seeking a Data Engineer with strong Databricks and GCP experience to build scalable ETL/ELT pipelines and productionize agentic workflows for data automation.

You will design robust data pipelines, optimize performance, and integrate with Databricks, GCP, BigQuery, and Delta Lake. A focus on data quality, testing, and production support is essential, with an English level of upper-intermediate.

Qualifications

• Databricks and PySpark hands-on experience (5+ years).
• Advanced SQL and data-processing skills.
• Hands-on GCP experience, especially BigQuery.
• Experience with Delta Lake and modern lakehouse architectures.
• Strong ETL/ELT, pipeline design, performance optimization, and data quality.
• Experience building reliable, scalable production-grade data solutions.
• Strong analytical and troubleshooting abilities.
• Understanding of software engineering practices (testing, VCS, deployment, monitoring).
• Upper-intermediate English level.

Responsibilities

• Design, develop, and optimize scalable ETL/ELT data pipelines using Databricks, PySpark, and SQL.
• Build and operationalize agentic workflows to automate data engineering and production processes.
• Integrate agentic capabilities with Databricks, GCP, BigQuery, and Delta Lake environments.
• Develop data pipelines and processing solutions for new business requirements and datasets.
• Create reusable frameworks and components for multiple data use cases.
• Implement data quality checks, monitoring, validation, and production controls.
• Optimize PySpark and SQL workloads for performance, reliability, and scalability.
• Support testing, deployment, productionization, and ongoing enhancement of data and agentic solutions.
• Troubleshoot complex data and production issues; implement sustainable fixes.
• Collaborate with business, data engineering, and platform teams to identify automation opportunities.

Skills

Databricks PySpark

Advanced SQL

GCP BigQuery

Delta Lake

ETL/ELT pipelines

Production-grade data solutions

Data quality & testing

English (upper-intermediate)

Job description

AgileEngine, LLC. is seeking a Data Engineer with strong Databricks and GCP experience to build scalable ETL/ELT pipelines and productionize agentic workflows for data automation.

You will design robust data pipelines, optimize performance, and integrate with Databricks, GCP, BigQuery, and Delta Lake. A focus on data quality, testing, and production support is essential, with an English level of upper-intermediate.