Opportunity AWS Data Engineer professional

Atos; Chennai District, Bengaluru, Mumbai; On-site; INR 2,500,000 - 4,000,000; Full time

Job summary

Atos in India is seeking an experienced AWS Data Engineer to own enterprise-grade data pipelines and data products. You will baseline Lambda functions, Dagster pipelines, dbt models, MySQL flows, and Kafka feeds, while guiding architectural decisions and best practices.

Responsibilities include designing Glue-based ingestion, S3 bronze Landing, and Redshift conformance; implementing data quality gates; and mentoring engineers in an Agile environment.

Qualifications

• 10+ years in data engineering with 5+ years on AWS data platforms.
• Deep Amazon Redshift knowledge: modelling, tuning, RBAC.
• Hands-on AWS Glue: jobs, Catalog, Crawlers; Bronze/Silver/Gold medallion.
• Experience with AWS Lambda, Step Functions, S3, IAM, CloudWatch, Secrets Manager.
• Advanced SQL and dimensional data modelling; ETL/ELT frameworks.
• Python and PySpark for Glue jobs; MySQL operational data store; CDC patterns.

Responsibilities

• Inventory and baseline current Lambda functions, Dagster pipelines, dbt models, MySQL ODS flows and Kafka feeds; document lineage, schedules, SLAs, and failure paths.
• Design and build Glue-based ingestion, transformation, and cataloguing patterns with Lambda/Step Functions orchestration.
• Implement S3 Bronze landing with lifecycle policies, Redshift Silver conformance and Gold canonical data products (Customer, Loan, Collections, Risk).
• Create data quality gates, reconciliation checks, and support enterprise data dictionary and certified reports.
• Establish IAM, encryption, Secrets Manager and RBAC/least-privilege access.
• Set up Dev/Test environments, Git-based CI/CD, schema version control (Liquibase), and controlled releases; monitor via CloudWatch.
• Migrate workloads wave by wave against baselines; decommission ODS dependencies and replace with governed interfaces.
• Optimize Redshift performance and cost (WLM, keys, materialised views).
• Mentor engineers and ensure technical quality; collaborate with Data Owners and Atos architects in Agile.

Skills

Data engineering

AWS data platforms

SQL

Python

PySpark

Leadership

Mentoring

Agile

Tools

AWS Redshift

AWS Glue

AWS Lambda

S3

IAM

CloudWatch

Secrets Manager

Liquibase

Terraform

Dagster

Kafka

Apache Spark/EMR

Tableau

Metabase

Liquibase or equivalent

Job description

Dear Candidate,

This is with reference for AWS Data Engineer Professionals

PFB the Job Description

Key responsibilities

• Inventory and baseline current Lambda functions, Dagster pipelines, dbt models, MySQL ODS flows and Kafka feeds, document lineage, schedules, SLAs and failure paths.
• Design and build Gluebased ingestion, transformation, and cataloguing patterns (Glue jobs, Glue Catalog, Crawlers) with Lambda/Step Functions/Event Bridge orchestration.
• Implement S3 Bronze landing with lifecycle/archive policies, Redshift Silver conformance and Gold canonical data products (Customer, Loan, Collections, Risk).
• Build automated dataquality gates, reconciliation checks, and sourcetoreport lineage; support the enterprise data dictionary and certification of businesscritical reports
• Implement IAM, AWSnative encryption, Secrets Manager and Goldlayer RBAC/leastprivilege access.
• Establish Dev/Test environments, Gitbased CI/CD, schema version control (Liquibase) and controlled release promotion; monitoring and recovery via CloudWatch.
• Migrate and reconcile workloads wave by wave against baselines; decommission ODS dependencies and replace reverse flows with governed interfaces.
• Optimise Redshift performance and cost (WLM, sort/Dist keys, materialised views).
• Mentor engineers, own technical quality assurance of deliverables, and work with Data Owners/Champions and Atos architects in an Agile setup.

Mandatory skills

• 10+ years in data engineering with 5+ Experience in AWS data platforms.
• Amazon Redshift (deep): data modelling, performance tuning, WLM, RBAC, external schemas/Spectrum.
• AWS Glue (jobs, Glue Catalog, Crawlers) and handson medallion (Bronze/Silver/Gold) implementation on S3 + Redshift.
• AWS Lambda, Step Functions, S3, IAM, CloudWatch, Secrets Manager.
• Advanced SQL, dimensional/canonical data modelling, ETL/ELT frameworks.
• Python (primary) PySpark for Glue jobs.
• MySQL (operational data store) and CDC/batch extraction patterns.
• dbt must be able to read, reverseengineer, and migrate existing dbt models.
• Dataquality frameworks, reconciliation, and lineage/metadata practices.
• Git,CI/CD pipelines, database schema version control (Liquibase or equivalent), Dev/Test/Prod promotion.
• Leadership, mentoring, stakeholder management; ability to work independently in Agile.

Nice to have

• Dagster orchestration for ASIS discovery and migration.
• Kafka / eventdriven streaming ingestion.
• Apache Iceberg on S3, Apache Spark/EMR
• Tableau, Metabase servinglayer knowledge.
• Infrastructure as Code (Terraform/CloudFormation).
• Datagovernance tooling (Glue Data Quality, Data Zone, Lake Formation).

Total Overall Experience- 10-15years

Skill AWS Data Engineer

Location Mumbai, Chennai, and Bangalore