Data Engineer (AWS Glue, Airflow, Lambda, Postgres) (contract)

Capgemini; Fort Mill (SC); On-site; USD 54,911 - 85,810; Full time

Benefits offered by this job

Medical benefits

Dental benefits

Vision benefits

Retirement benefits

Job summary

Capgemini is seeking a skilled Data Engineer (AWS Glue, Airflow, Lambda, Postgres) in Fort Mill, South Carolina. The role involves designing and building scalable data pipelines and performing analytics on AWS and Postgres platforms.

The ideal candidate should have 6+ years of experience in Data Engineering, with strong skills in AWS services and SQL, especially Postgres. This position offers a pay range of $39.86/hour to $62.29/hour and various benefits including medical and retirement plans.

Qualifications

• 6+ years of experience in Data Engineering or a related field.
• Strong hands‑on expertise in AWS Glue, Lambda, EventBridge, and S3.
• Experience building workflows using Apache Airflow (MWAA preferred).
• Strong proficiency in SQL (Postgres preferred).
• Experience with Python/PySpark for ETL development.

Responsibilities

• Design and develop scalable data pipelines using AWS Glue and Lambda.
• Build and maintain AWS Glue ETL jobs for data ingestion.
• Develop and manage Airflow DAGs for orchestrating data workflows.
• Implement event‑driven architectures using AWS Lambda.
• Optimize complex SQL queries in Postgres.

Skills

AWS Glue

AWS Lambda

Apache Airflow

SQL (Postgres)

Python/PySpark

AWS EventBridge

Data pipeline design

Tools

AWS Glue Data Catalog

CI/CD (GitHub Actions)

Terraform

Job description

Data Engineer (AWS Glue, Airflow, Lambda, Postgres)

We are seeking a skilled Data Engineer with strong hands‑on experience in AWS‑native data services, particularly Glue, Lambda, EventBridge, and workflow orchestration using Airflow. The role focuses on building scalable, event‑driven data pipelines and supporting data processing and analytics on AWS and Postgres platforms.

Key Responsibilities

• Design and develop scalable data pipelines using AWS Glue, Lambda, and EventBridge for event‑driven and batch processing.
• Build and maintain AWS Glue ETL jobs (PySpark/Python) for data ingestion, transformation, and curation across data lake layers.
• Develop and manage Airflow DAGs (MWAA or self‑managed) to orchestrate data workflows, triggers, and dependencies across AWS services.
• Implement event‑driven architectures using AWS Lambda and EventBridge for near real‑time data processing.
• Write, optimize, and maintain complex SQL queries in Postgres for data validation, transformation, and reporting.
• Manage metadata and schema definitions using AWS Glue Data Catalog, ensuring proper governance and discoverability.
• Build and support robust data ingestion frameworks for batch and near real‑time data pipelines.
• Monitor and troubleshoot pipeline performance using AWS monitoring tools (CloudWatch, logs, alerts).
• Collaborate with downstream teams (BI, analytics, and Snowflake if applicable) to ensure reliable data consumption.
• Contribute to system reliability, scalability, and performance optimization of the data platform.

Required Skills & Qualifications

• 6+ years of experience in Data Engineering or a related field.
• Strong hands‑on expertise in AWS Glue, Lambda, EventBridge, and S3.
• Experience building and orchestrating workflows using Apache Airflow (MWAA preferred).
• Strong proficiency in SQL (Postgres preferred) for complex transformations and performance tuning.
• Experience with Python/PySpark for ETL development.
• Solid understanding of event‑driven data architecture and pipeline design.
• Familiarity with AWS IAM, CloudWatch, and logging/monitoring frameworks.
• Experience with CI/CD pipelines (e.g., GitHub Actions) for deployment automation.

Preferred Skills

• Experience with Terraform or infrastructure‑as‑code tools.
• Knowledge of data lake formats (Parquet, Iceberg, Delta).
• Understanding of data modeling, partitioning, and schema evolution.
• Familiarity with AWS services such as Lake Formation, SNS, and CloudTrail.
• Exposure to Snowflake or other data warehouse platforms.

The pay range that the employer in good faith reasonably expects to pay for this position is $39.86/hour - $62.29/hour. Our offered benefits include medical, dental, vision and retirement benefits.

We are an equal opportunity employer, and we do not discriminate on the basis of race, religion, color, national origin, sex, sexual orientation, age, veteran status, disability, genetic information or other applicable legally protected characteristic. Qualified applicants with arrest or conviction records will be considered for employment in accordance with applicable law, including the Los Angeles County Fair Chance Ordinance for Employers and the California Fair Chance Act. Unincorporated LA County workers: we reasonably believe that criminal history may have a direct adverse and negative relationship with the following job duties, potentially resulting in the withdrawal of a conditional offer of employment: client provided property, including hardware (both of which may include data) entrusted to you from theft, loss or damage; return all portable client computer hardware in your possession (including the data contained therein) upon completion of the assignment; and maintain the confidentiality of client proprietary, confidential, or non‑public information. In addition, job duties require access to secure and protected client information technology systems and related data security obligations.