Datastage Consultant

Capgemini; Hyderabad, Pune District, Bengaluru; On-site; INR 1,200,000 - 2,300,000; Full time

Job summary

Capgemini in Hyderabad is hiring an experienced DataStage Developer to design, build, and modernize enterprise data pipelines. The role involves sustaining IBM InfoSphere DataStage workloads and migrating them to a cloud-native GCP stack.

You will migrate legacy pipelines to BigQuery and GCP services, design orchestrations with Cloud Composer, and write optimized SQL for data transformations, while collaborating with data architects and QA in an Agile environment.

Qualifications

• 4+ years of hands-on IBM DataStage development (Designer, Director, Administrator)
• Solid SQL and RDBMS experience — Oracle, DB2, Teradata, or SQL Server
• Practical working experience on GCP: BigQuery, Dataflow, Cloud Composer, Dataproc, Pub/Sub
• Strong understanding of data warehousing concepts — dimensional modelling, SCD Types 1/2, star/snowflake schema
• Unix/Linux shell scripting experience
• Experience with job scheduling tools like Control‑M, Autosys, Tivoli

Responsibilities

• Design, develop, and optimize ETL jobs using IBM DataStage (11.x/12.x) parallel jobs and containers
• Migrate legacy DataStage pipelines to GCP-native services (BigQuery, Cloud Storage, Dataflow, Cloud Composer)
• Build and schedule orchestration workflows using Cloud Composer / Apache Airflow
• Write and tune complex SQL for data transformation, reconciliation, and BigQuery performance
• Perform source-to-target mapping, data profiling, and impact analysis for migration workstreams
• Implement error handling, restartability, logging, and data quality checks across pipelines
• Tune DataStage job performance — partitioning, buffer tuning, and configuration files
• Support UAT, production deployments, and post-go-live hypercare; troubleshoot job failures
• Collaborate with data architects, BAs, and QA in an Agile/Scrum delivery model
• Maintain technical documentation, unit test cases, and deployment runbooks

Skills

DataStage

SQL

GCP

Data warehousing

Unix/Linux shell scripting

Job scheduling tools

Tools

IBM InfoSphere DataStage

Job description

We are hiring an experienced DataStage Developer with hands‑on Google Cloud Platform exposure to build and modernize enterprise data pipelines. The role sits within our Data Engineering practice and involves both sustaining existing IBM InfoSphere DataStage workloads and migrating them to a cloud‑native GCP stack.

Key Responsibilities

• Design, develop, and optimize ETL jobs using IBM InfoSphere DataStage (11.x / 12.x) parallel jobs, sequences, and shared containers
• Migrate legacy DataStage pipelines to GCP-native services (BigQuery, Cloud Storage, Dataflow, Cloud Composer)
• Build and schedule orchestration workflows using Cloud Composer / Apache Airflow
• Write and tune complex SQL for data transformation, reconciliation, and BigQuery performance optimization (partitioning, clustering, slot usage)
• Perform source‑to‑target mapping, data profiling, and impact analysis for migration workstreams
• Implement error handling, restartability, logging, and data quality checks across pipelines
• Tune DataStage job performance — partitioning strategies, buffer tuning, parallel configuration files (APT_CONFIG_FILE)
• Support UAT, production deployments, and post‑go‑live hypercare; troubleshoot job failures and SLA breaches
• Collaborate with data architects, BAs, and QA teams in an Agile/Scrum delivery model
• Maintain technical documentation, unit test cases, and deployment runbooks

Must‑Have Skills

• 4+ years of strong hands‑on IBM DataStage development (Designer, Director, Administrator)
• Solid SQL and RDBMS experience — Oracle, DB2, Teradata, or SQL Server
• Practical working experience on GCP: BigQuery + at least one of Dataflow, Cloud Composer, Dataproc, Pub/Sub
• Strong understanding of data warehousing concepts — dimensional modelling, SCD Types 1/2, star/snowflake schema
• Experience with Unix/Linux shell scripting
• Job scheduling tools — Control‑M, Autosys, or Tivoli