Python Software Developer

KRG Technologies; Hyderabad; Hybrid; INR 1,800,000 - 2,800,000; Full time

Job summary

KRG Technologies is seeking a GCP-focused data engineering professional to architect and implement migration and modernization initiatives on Google Cloud Platform.

You will design scalable pipelines, perform data quality checks, and enable metadata governance across BigQuery, Dataproc, Bigtable, and related services. GenAI-enabled data transformations and code generation will be part of the role.

Qualifications

• Proven experience in migrating data and modernizing on-prem or other clouds to GCP.
• Strong SQL and Python for ETL/ELT pipelines and data automation.
• Experience with data quality, governance, metadata management and data lineage.

Responsibilities

• Design and execute migration and modernization strategies to GCP.
• Build and maintain scalable migration pipelines using Python, SQL and Dataproc.
• Implement batch and streaming migrations with Cloud Composer, Kafka and Pub/Sub.
• Enable data governance with Dataplex and Data Catalog.
• Apply GenAI to data discovery, mapping, and transformation processes.

Skills

GCP expertise

Python

SQL

ETL/ELT pipelines

Data governance

Tools

BigQuery

Dataproc

Bigtable

Dataplex

Data Catalog

Cloud Composer

Airflow

Kafka

Pub/Sub

Apigee

REST APIs

Vertex AI

GenAI

Job description

Role & responsibilities

GCP Engineering Skills

• Google Cloud Platform (GCP)
• BigQuery
• Dataproc / Apache Spark
• Bigtable
• Dataplex / Data Catalog
• Python (ETL/ELT pipelines and automation)
• SQL (data transformation, validation, reconciliation)
• Databases:
• MySQL
• Hive
• MongoDB
• Cloud Composer / Apache Airflow
• Kafka and Google Pub/Sub
• Apigee / REST APIs (data extraction and integration)
• GenAI (LLMs, embeddings, prompt engineering) and integration with data platforms
• Experience with Vertex AI, BigQuery ML, or GenAI-enabled GCP services (preferred)

Key Responsibilities

Migration & Modernization

• Design and execute comprehensive migration strategies from on-premises, legacy platforms, or other cloud environments to GCP.
• Migrate structured and semi-structured data to BigQuery, Bigtable, and other GCP storage solutions.
• Build, optimize, and maintain scalable migration pipelines using Python, SQL, and Dataproc (Apache Spark).
• Implement batch and streaming migration patterns using:
• Cloud Composer (Apache Airflow)
• Kafka
• Google Pub/Sub

Data Quality & Governance

• Perform schema mapping, data transformation, reconciliation, and validation to ensure accuracy, consistency, and completeness of migrated datasets.
• Enable metadata management, lineage, and governance using Dataplex and Data Catalog.
• Handle large-volume data transfers using best practices including:
• Parallel loading
• Incremental loads
• CDC (Change Data Capture)

GenAI Adoption

• Leverage GenAI and automation techniques to accelerate:
• Data discovery
• Schema understanding
• Mapping recommendations
• Migration assessments
• Apply GenAI-assisted:
• Data transformation
• Code generation
• Migration pattern optimization
• SQL conversion
• Pipeline scaffolding
• Test case generation

Preferred candidate profile