Senior Databricks Data Engineer — Scalable ETL Architect

CGI Group, Inc.; Miami (FL); Hybrid; USD 92,000 - 204,000; Full time

Benefits offered by this job

Competitive compensation

Comprehensive insurance options

401(k) match and share purchase plan

Paid time off for vacation, holidays,和

Paid parental leave

Learning opportunities and tuitionass

Job summary

CGI Group, Inc. in Miami, FL is seeking a Senior Databricks Data Engineer to design, develop, and optimize end-to-end ETL/ELT pipelines using Databricks, Spark, PySpark, and SQL.

You’ll own the full engineering lifecycle, implement data quality controls, and collaborate with data architects and stakeholders in a hybrid work setup. The role requires 5+ years of hands-on experience with Databricks, Spark, Python, and cloud platforms (AWS/Azure/GCP), plus expertise in orchestration tools like

Qualifications

• Minimum 5 years of relevant experience in data engineering, data integration, or related disciplines.
• Strong hands-on development experience with Databricks.
• Proven experience designing, building, and implementing production ETL/ELT pipelines.
• Advanced proficiency in Apache Spark, PySpark, and Python with hands-on engineering experience.
• Strong SQL skills with complex joins, window functions, data transformations, and query performance tuning.
• Experience with at least one major cloud platform: AWS, Azure, or Google Cloud Platform (GCP).
• Experience with workflow orchestration tools such as Apache Airflow, Databricks Workflows, or equivalents.
• Experience with CI/CD practices, source control, and automated deployment of data pipelines.
• Demonstrated ability to own full development lifecycle from design to production support.
• Ability to evaluate requirements and implement approved solutions.
• Strong problem-solving, accountability, and ownership in fast-paced environments.

Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines using Databricks, Spark, Python/PySpark, and SQL.
• Build batch and incremental data integration pipelines from files, databases, and enterprise sources.
• Create reusable, maintainable data processing solutions.
• Optimize Spark workloads for performance, scalability, reliability, and cost efficiency.
• Implement data quality controls, reconciliation, monitoring, and error handling.
• Own the complete engineering lifecycle: requirements analysis, design, review, implementation, deployment.
• Prepare and present technical solution designs for architecture reviews.
• Incorporate feedback and implement approved designs; make technical decisions and resolve challenges.
• Ensure deliverables, quality, timelines, and production readiness.
• Develop complex SQL transformations for cleansing, enrichment, and aggregation.
• Maintain curated datasets, dimensional models, and reporting views.
• Support enterprise analytics, reporting, and downstream data consumers.
• Implement and maintain pipeline orchestration using Airflow, Databricks Workflows, or equivalents.
• Apply CI/CD practices for version control and automated deployment.

Skills

Databricks

Apache Spark

Python

PySpark

SQL

Airflow

Databricks Workflows

CI/CD

AWS

Azure

Google Cloud Platform (GCP)

Delta Lake

Unity Catalog

REST API

Education

Bachelor's or Master's degree in Computer Science, Engineering, or a related field

Job description

CGI Group, Inc. in Miami, FL is seeking a Senior Databricks Data Engineer to design, develop, and optimize end-to-end ETL/ELT pipelines using Databricks, Spark, PySpark, and SQL.

You’ll own the full engineering lifecycle, implement data quality controls, and collaborate with data architects and stakeholders in a hybrid work setup. The role requires 5+ years of hands-on experience with Databricks, Spark, Python, and cloud platforms (AWS/Azure/GCP), plus expertise in orchestration tools like