Data Architect- GCP

Quantiphi; Bengaluru; On-site; INR 3,500,000 - 6,000,000; Full time

Job summary

Quantiphi seeks an experienced Architect to lead data engagements on GCP for data lakes, data engineering, migration, data warehouse and BI initiatives. You will interface with IT and business stakeholders to translate data needs into actionable solutions and ensure successful delivery across scope, quality, and customer satisfaction.

You will design distributed, high-volume batch and real-time pipelines, validate data quality, and enable production-ready data assets.

Qualifications

• More than 10 years in technical, solutioning, and analytical roles.
• 5+ years building and managing data lakes, data warehouses, data integration and migration on cloud GCP.
• Ability to translate business requirements into functional/non-functional boundaries (availability, scalability, performance, security, resilience).
• End-to-end data pipelines for varied structured/unstructured data sources.
• Experience with Hadoop, GCP Cloud, and on-premise/ cloud data tooling.

Responsibilities

• Lead multiple data engagements on GCP for data lakes, data engineering, migration, data warehouse, and BI.
• Interface with IT and business stakeholders to understand data requirements.
• Ensure on-time delivery with quality and customer satisfaction.
• Design and develop distributed, high-volume batch, real-time, and event-processing systems.
• Validate data, monitor quality, and ensure production data is accurate and available.
• Collaborate with Pre-Sales on data solutions for RFP/RFI responses.
• Mentor junior talent and contribute to building assets and accelerators.

Skills

Data lakes

Data warehouse

Data integration

Data migration

BI/AI on GCP

BigQuery/Redshift

Spark / PySpark

Looker

Cloud and on-prem DBs

ETL tools

Tools

Spark

PySpark/Scala

Dataflow

DataProc

EMR

Informatica

DataStage

Talend

Looker

BigQuery

Redshift

Job description

Role & Responsibilities:

• More than 10 years of experience in Technical, Solutioning, and Analytical roles.


• 5+ years of experience in building and managing Data Lakes, Data Warehouse, Data Integration, Data Migration and Business Intelligence/Artificial Intelligence solutions on Cloud GCP


• Ability to understand business requirements, translate them into functional and non-functional areas,define non-functional boundaries in terms of Availability, Scalability, Performance, Security, Resilience etc.


• Experience in architecting, designing, and implementing end to end data pipelines and data integration solutions for varied structured and unstructured data sources and targets.


• Experience of having worked in distributed computing and enterprise environments like Hadoop,
  GCP Cloud.


• Well versed with various Data Integration, and ETL technologies on Cloud like Spark, Pyspark/Scala,Dataflow, DataProc, EMR, etc. on various Cloud.


• Experience of having worked with traditional ETL tools like Informatica/DataStage/OWB/Talend, etc.


• Deep knowledge of one or more Cloud and On-Premise Databases like Cloud SQL, Cloud Spanner, Big Table, RDS, Aurora, DynamoDB, Oracle, Teradata, MySQL, DB2, SQL Server, etc.


• Exposure to any of the No-SQL databases like MongodB, CouchDB, Cassandra, Graph dB, etc.

Architect - Data

• Experience in architecting and designing scalable data warehouse solutions on cloud on Big Query or
  Redshift.


• Experience in having worked on one or more data integration, storage, and data pipeline tool sets like S3,


• Preferred experience of having worked on Machine Learning Frameworks like TensorFlow, Pytorch, etc.


• Good understanding of Cloud solutions for Iaas, PaaS, SaaS, Containers and Microservices Architecture
  and Design.


• Ability to compare products and tools across technology stacks on Google, AWS, and Azure Cloud.


• Good understanding of BI Reporting and Dashboarding and one or more tool sets associated with it like Looker.


• Understanding of Security features and Policies in one or more Cloud environments like GCP.


• Experience of having worked in business transformation projects for movement of On-Premise data solutions to Clouds like GCP.

Role:

• Lead multiple data engagements on GCP Cloud for data lakes, data engineering, data migration, data warehouse, and business intelligence.


• Interface with multiple stakeholders within IT and business to understand the data requirements.


• Take complete responsibility for the successful delivery of all allocated projects on the parameters of Schedule, Quality, and Customer Satisfaction.


• Responsible for design and development of distributed, high volume multi-thread batch, real-time, and event processing systems.


• Implement processes and systems to validate data, monitor data quality, ensuring production data is always accurate and available for key stakeholders and business processes that depend on it.


• Work with the Pre-Sales team on RFP, RFIs and help them by creating solutions for data.


• Mentor Young Talent within the Team, Define and track their growth parameters.


• Contribute to building Assets and Accelerators.