Senior Data Engineer - Databricks & GCP (Flextime)

AgileEngine, LLC.; Wrocław; Hybrid; PLN 180,000 - 300,000; Full time

Benefits offered by this job

Professional growth

Competitive USD-based compensation

A selection of exciting projects

Flextime

Job summary

AgileEngine, LLC. is seeking a Data Engineer skilled in Databricks, PySpark, and SQL to design scalable ETL/ELT pipelines and automate data workflows. The role emphasizes productionizing agentic processes and integrating with GCP/BigQuery and Delta Lake.

Ideal candidates bring 5+ years of hands-on experience, strong data quality discipline, and a passion for scalable, reliable data platforms. Hybrid work options and a dynamic, award-winning environment are offered.

Qualifications

• 5+ years of hands-on Databricks and PySpark experience.
• Advanced SQL and data-processing skills.
• Hands-on GCP experience, esp. BigQuery.
• Experience with Delta Lake and modern data lake architectures.
• Strong ETL/ELT design and data quality focus.
• Experience building scalable data solutions for production.
• Strong analytical and troubleshooting abilities.
• Knowledge of software engineering practices incl. testing and deployments.

Responsibilities

• Design, develop, and optimize scalable ETL/ELT pipelines with Databricks, PySpark, and SQL.
• Build and operationalize agentic workflows to automate data engineering tasks.
• Integrate agentic capabilities with Databricks, GCP, BigQuery, and Delta Lake.
• Develop data pipelines and solutions for new datasets and requirements.
• Create reusable data engineering frameworks and components.
• Implement data quality checks, monitoring, and production controls.
• Optimize PySpark and SQL workloads for performance and reliability.
• Support testing, deployment, and productionization of data solutions.
• Troubleshoot complex data problems and provide sustainable fixes.
• Collaborate with business and platform teams to find more automation opportunities.

Skills

Databricks

PySpark

SQL

GCP BigQuery

Delta Lake

ETL/ELT

Data quality

Production support

Troubleshooting

Tools

Databricks

PySpark

SQL

Job description

AgileEngine, LLC. is seeking a Data Engineer skilled in Databricks, PySpark, and SQL to design scalable ETL/ELT pipelines and automate data workflows. The role emphasizes productionizing agentic processes and integrating with GCP/BigQuery and Delta Lake.

Ideal candidates bring 5+ years of hands-on experience, strong data quality discipline, and a passion for scalable, reliable data platforms. Hybrid work options and a dynamic, award-winning environment are offered.