Data Engineer - Databricks & PySpark on GCP | Flexible Hours

AgileEngine, LLC.; Poznań; Vor Ort; PLN 430.000 - 548.000; Vollzeit

Benefits dieser Stelle

Professional growth

Competitive USD-based compensation

Exciting projects

Flextime

Zusammenfassung

AgileEngine is seeking a Data Engineer with strong Databricks and GCP experience to design scalable ETL/ELT pipelines and productionize agentic workflows for data engineering automation.

You will design, build, and optimize data flows using Databricks, PySpark, and SQL; integrate with BigQuery and Delta Lake; and collaborate across teams to deliver reliable data solutions for Fortune 500 clients.

Qualifikationen

• 5+ years of hands-on Databricks and PySpark experience.
• Advanced SQL and data-processing skills.
• GCP exposure, especially BigQuery.
• Delta Lake and modern data lake architectures experience.
• ETL/ELT design, performance optimization, and data quality.
• Production-grade data solutions development.
• Strong analytical and troubleshooting abilities.
• Software engineering practices: testing, version control, deployment, monitoring.

Aufgaben

• Design, develop, and optimize scalable ETL/ELT pipelines using Databricks, PySpark, and SQL.
• Build and operationalize agentic workflows to automate data engineering and operations.
• Integrate agentic capabilities with Databricks, GCP, BigQuery, and Delta Lake environments.
• Develop pipelines and processing solutions for new business requirements and datasets.
• Create reusable data engineering frameworks and components.
• Implement data quality checks, monitoring, validation, and production controls.
• Optimize PySpark and SQL workloads for performance and reliability.
• Support testing, deployment, productionization, and enhancement of data and agentic solutions.
• Troubleshoot complex data and production issues and implement sustainable fixes.
• Collaborate with business, data engineering, and platform teams to identify automation opportunities.

Kenntnisse

Databricks

PySpark

SQL

Data pipelines

Data quality

Troubleshooting

Production-grade data solutions

Tools

BigQuery

Delta Lake

GCP

Jobbeschreibung

AgileEngine is seeking a Data Engineer with strong Databricks and GCP experience to design scalable ETL/ELT pipelines and productionize agentic workflows for data engineering automation.

You will design, build, and optimize data flows using Databricks, PySpark, and SQL; integrate with BigQuery and Delta Lake; and collaborate across teams to deliver reliable data solutions for Fortune 500 clients.