Senior Databricks Data Engineer

CGI Group, Inc.; Miami (FL); Hybrid; USD 92,000 - 204,000; Full time

Benefits offered by this job

Competitive compensation

Comprehensive insurance options

401(k) match and share purchase plan

Paid time off for vacation, holidays,和

Paid parental leave

Learning opportunities and tuitionass

Job summary

CGI Group, Inc. in Miami, FL is seeking a Senior Databricks Data Engineer to design, develop, and optimize end-to-end ETL/ELT pipelines using Databricks, Spark, PySpark, and SQL.

You’ll own the full engineering lifecycle, implement data quality controls, and collaborate with data architects and stakeholders in a hybrid work setup. The role requires 5+ years of hands-on experience with Databricks, Spark, Python, and cloud platforms (AWS/Azure/GCP), plus expertise in orchestration tools like

Qualifications

• Minimum 5 years of relevant experience in data engineering, data integration, or related disciplines.
• Strong hands-on development experience with Databricks.
• Proven experience designing, building, and implementing production ETL/ELT pipelines.
• Advanced proficiency in Apache Spark, PySpark, and Python with hands-on engineering experience.
• Strong SQL skills with complex joins, window functions, data transformations, and query performance tuning.
• Experience with at least one major cloud platform: AWS, Azure, or Google Cloud Platform (GCP).
• Experience with workflow orchestration tools such as Apache Airflow, Databricks Workflows, or equivalents.
• Experience with CI/CD practices, source control, and automated deployment of data pipelines.
• Demonstrated ability to own full development lifecycle from design to production support.
• Ability to evaluate requirements and implement approved solutions.
• Strong problem-solving, accountability, and ownership in fast-paced environments.

Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines using Databricks, Spark, Python/PySpark, and SQL.
• Build batch and incremental data integration pipelines from files, databases, and enterprise sources.
• Create reusable, maintainable data processing solutions.
• Optimize Spark workloads for performance, scalability, reliability, and cost efficiency.
• Implement data quality controls, reconciliation, monitoring, and error handling.
• Own the complete engineering lifecycle: requirements analysis, design, review, implementation, deployment.
• Prepare and present technical solution designs for architecture reviews.
• Incorporate feedback and implement approved designs; make technical decisions and resolve challenges.
• Ensure deliverables, quality, timelines, and production readiness.
• Develop complex SQL transformations for cleansing, enrichment, and aggregation.
• Maintain curated datasets, dimensional models, and reporting views.
• Support enterprise analytics, reporting, and downstream data consumers.
• Implement and maintain pipeline orchestration using Airflow, Databricks Workflows, or equivalents.
• Apply CI/CD practices for version control and automated deployment.

Skills

Databricks

Apache Spark

Python

PySpark

SQL

Airflow

Databricks Workflows

CI/CD

AWS

Azure

Google Cloud Platform (GCP)

Delta Lake

Unity Catalog

REST API

Education

Bachelor's or Master's degree in Computer Science, Engineering, or a related field

Job description

Senior Databricks Data Engineer role focused on building scalable enterprise ETL/ELT pipelines end to end.

Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines using Databricks, Apache Spark, Python/PySpark, and SQL
• Build batch and incremental data integration pipelines from files, databases, and enterprise data sources
• Create reusable, maintainable data processing solutions
• Optimize Spark workloads for performance, scalability, reliability, and cost efficiency
• Implement data quality controls, reconciliation, monitoring, and error handling
• Own the complete engineering lifecycle: requirements analysis, solution design, technical review, implementation, and deployment
• Prepare and present technical solution designs for architecture and engineering review
• Incorporate review feedback and implement approved designs
• Make appropriate technical decisions and proactively resolve engineering challenges
• Take accountability for deliverables, quality, timelines, and production readiness
• Develop complex SQL transformations for cleansing, enrichment, and aggregation
• Design and maintain curated datasets, dimensional models, and reporting views
• Support enterprise analytics, reporting, and downstream data consumers
• Implement and maintain pipeline orchestration using Airflow, Databricks Workflows, or comparable tools
• Apply CI/CD practices for version control, automated deployment, and release management
• Troubleshoot pipeline failures and production performance issues
• Collaborate with data architects, integration teams, engineering teams, and business stakeholders
• Document technical designs, implementation decisions, and operational procedures

Requirements

• Minimum 5 years of relevant experience in data engineering, data integration, or related disciplines
• Strong, recent hands-on development experience with Databricks
• Proven experience designing, building, and implementing production ETL/ELT pipelines
• Advanced proficiency in Apache Spark, PySpark, and Python with demonstrated hands-on engineering experience
• Strong SQL skills, including complex joins, window functions, data transformations, and query performance tuning
• Experience with at least one major cloud platform: AWS, Azure, or Google Cloud Platform (GCP)
• Experience with workflow orchestration tools such as Apache Airflow, Databricks Workflows, or equivalent platforms
• Experience with CI/CD practices, source control, and automated deployment of data pipelines
• Demonstrated ability to independently own the full development lifecycle, including solution design, technical review, implementation, and production support
• Proven ability to evaluate requirements, make sound technical design decisions, and implement approved solutions
• Strong problem-solving skills with a track record of independent execution, accountability, and ownership in fast-paced environments

Technologies

• Databricks
• Apache Spark
• Python
• PySpark
• SQL
• Airflow
• Databricks Workflows
• CI/CD
• AWS
• Azure
• Google Cloud Platform (GCP)
• Delta Lake
• Unity Catalog
• REST API

Nice to Haves

• Calling external APIs from Databricks notebooks, including REST API integration, authentication, request handling, and ingestion of API responses (building or exposing APIs is not required)
• Streaming and event driven data processing
• Delta Lake and medallion architecture
• Unity Catalog, governance, and lineage
• Metadata driven and reusable pipeline frameworks
• Enterprise scale Databricks implementations
• Experience with complex enterprise data integration environments
• Spark performance tuning and cost optimization
• Batch and incremental data processing
• Data quality, reconciliation, monitoring, and observability
• Data warehousing, fact/dimension modeling, and curated data layers
• Production troubleshooting and operational reliability
• Technical documentation and design review participation

Benefits

• Competitive compensation
• Comprehensive insurance options
• Matching contributions through the 401(k) plan and the share purchase plan
• Paid time off for vacation, holidays, and sick time
• Paid parental leave
• Learning opportunities and tuition assistance
• Wellness and Well being programs

Location: Miami, FL (hybrid)

Salary: USD 91,800 - 203,800 per yearly

Education: Bachelor's or Master's degree in Computer Science, Engineering, or a related field

Additional info: CGI anticipates accepting applications through November 13, 2026. Compensation range estimate for the U.S.: $91,800.00 - $203,800.00.