GCP Data Engineer with big query

System Soft Technologies; Tamil Nadu; On-site; INR 2,000,000 - 4,000,000; Full time

Job summary

System Soft Technologies seeks an experienced Senior Data Engineer to design and modernize big data solutions on the Google Cloud Platform. You will lead large-scale data engineering initiatives, architect data warehouses and data lakes, and ensure production-ready analytics platforms.

You will work with BigQuery, Dataflow, Pub/Sub, Airflow, Terraform and GitHub to implement scalable pipelines and integrate Oracle, Teradata, DB2 and flat files, while tuning ETL performance for reliability and

Qualifications

• 8+ years of data engineering experience.
• 5+ years of cloud experience (GCP preferred) with solutions designed and implemented at production scale.
• Experience with data warehouses, data lakes and analytics platforms on cloud.
• Terraform/GitHub experience.
• Experience integrating data sources like Oracle, Teradata, DB2, BigQuery and flat files.
• Hands-on ETL tuning and debugging.

Responsibilities

• Design transformation and modernization of big data solutions on the GCP cloud.
• Lead large-scale data engineering initiatives and supervise delivery.
• Operationalize data warehouses, data lakes and analytics platforms on cloud.
• Validate, run, schedule and monitor data pipelines; coordinate with team.

Skills

Data engineering

Cloud (GCP)

ETL development

Data warehousing

Performance tuning

Tools

Terraform

GitHub

Airflow

BigQuery

PostgreSQL

Job description

This is onsite role and can sit either in Hyderabad, Chennai, Bengaluru and Nagpur

Job Summary for GCP Data Engineer:

Data & Analytics Product Group under FCSD Tech is looking to for a dynamic and result oriented candidate for the role of Senior Data Engineer within its Data Orchestration product line. The candidate will be responsible for designing the transformation and modernization of big data solutions on the GCP cloud. The candidate must have experience leading large-scale solutions and the operationalization of data warehouses, data lakes, and analytics platforms on a cloud platform, GCP preferred. We seek individuals with a wide range of technology skills in these domains, capable of designing the right solutions using a mix of GCP and third-party technologies for deployment on the GCP cloud.

Job Responsibilities for GCP Data Engineer:

• 8+ years of Data Engineering experience
• 5+ years of Cloud experience (GCP preferred) with solutions designed and
• 5+ implemented at production scale
• Strong understanding of key GCP services, especially those related to data processing [Batch/Real Time] Big Query, Cloud Scheduler, Airflow, Postgres, Data Flow, Pub/Sub, Cloud Logging and Cloud Monitoring Experience with infrastructure as code Terraform/ GitHub Experience in design, development and implementation of data pipelines using Data Warehousing applications Experience in integrating various data sources like Oracle, Teradata, DB2, Big Query & Flat files Hands on experience in performance tuning and debugging ETL jobs Involving in review meetings and coordinating with the team in job designing and fine-tuning the job performance.
• Responsible for validating, run, scheduling and monitoring jobs using data stage director Responsible for debugging Ticketing issues in the production phase