Lead Data Engineer 2

9018 AccentureSolutionsPvtLtd. Company; Ernakulam; On-site; INR 1,800,000 - 2,600,000; Full time

Job summary

Accenture is seeking a Lead Data Engineer specializing in GCP and BigQuery with 5+ years of experience to design, develop, and optimize scalable data pipelines for enterprise analytics. The role focuses on cloud-based data platforms, large-scale processing, and ensuring data quality and cost efficiency.

You will work across teams to implement ETL/ELT workflows, build curated data models, and drive adoption of modern GCP data engineering practices.

Qualifications

• 5+ years of data engineering, data warehousing, ETL/ELT, or large-scale processing.
• Proficiency in GCP and BigQuery for enterprise analytics.
• Strong Python and SQL development capabilities.
• Experience with data pipelines, data lakes, and cloud data platforms.

Responsibilities

• Design, build, and maintain scalable data pipelines on GCP using BigQuery, Python, and SQL.
• Optimize pipeline performance, data quality, and cost efficiency.
• Provide technical guidance and enforce engineering best practices.
• Collaborate with architects, data scientists, and stakeholders to translate requirements.

Skills

GCP services

BigQuery

Python

SQL

BI tools

Education

Graduation

Tools

Dataflow

Pub/Sub

Dataproc

Cloud Composer / Airflow

Job description

Job Title - Lead Data Engineer – Specialist - ACS SONG Management Level: Level 9 - Specialist Location: Kochi, Coimbatore, Trivandrum Must have skills: GCP services, BigQuery, Python, and SQL. Good to have skills: BI and Analytical Tool Experience: 5 -8 years of experience is required Educational Qualification: Graduation

Job Summary

We are seeking a Lead Data Engineer specializing in Google Cloud Platform (GCP) and BigQuery with 5+ years of experience in data engineering, cloud-based data platforms, data pipelines, SQL development, and large-scale data processing. The role will focus on designing, developing, and optimizing scalable data pipelines and analytics solutions using BigQuery and other GCP data services.

Roles and Responsibilities

• Design, develop, and maintain scalable data pipelines and data processing solutions using GCP services, BigQuery, Python, and SQL.
• Build reliable and high-performance data transformation and integration workflows supporting reporting, analytics, machine learning, and downstream business applications.
• Drive improvements in data quality, pipeline performance, BigQuery optimization, cost efficiency, monitoring, security, and production stability.
• Provide technical guidance and contribute to the adoption of modern GCP data engineering architecture, standards, and development best practices.
• Design, develop, and maintain batch and real-time data pipelines using BigQuery, Cloud Storage, Dataflow, Pub/Sub, Dataproc, Cloud Composer, Python, and SQL.
• Develop ETL/ELT solutions for ingesting, transforming, and integrating structured, semi-structured, and unstructured data from multiple source systems.
• Design and maintain scalable BigQuery datasets, tables, views, materialized views, partitioning, clustering, and data models for analytics and reporting workloads.
• Optimize BigQuery SQL queries, storage structures, data processing workloads, and pipeline architecture to improve performance and control cloud consumption costs.
• Build and maintain batch and streaming data processing solutions using services such as Dataflow, Pub/Sub, Dataproc, and BigQuery.
• Design curated data layers, reusable datasets, and data models supporting enterprise analytics, BI, machine learning, and downstream applications.
• Implement data validation, reconciliation, data quality checks, error handling, lineage, monitoring, alerting, and operational controls.
• Orchestrate data pipelines and workflows using Cloud Composer / Apache Airflow, Workflows, or equivalent orchestration technologies.
• Collaborate with solution architects, data architects, data scientists, analysts, application teams, and business stakeholders to translate requirements into scalable technical solutions.
• Follow and promote engineering best practices covering Git-based development, automated testing, CI/CD, Infrastructure as Code, documentation, monitoring, security, and production support.

Professional and Technical Skills

• 5+ years of experience in data engineering, data warehousing, ETL/ELT development, SQL development, or large-scale data processing.
• Strong hands-on experience designing and implementing data solutions on Google Cloud Platform (GCP).
• Strong production experience with Google BigQuery for enterprise data warehousing, analytics, and large-scale data processing.
• Experience designing and supporting production-grade batch and/or streaming data pipelines.
• Experience working with large datasets and building scalable, reliable, and high-performance cloud data solutions.
• Strong expertise in Google BigQuery, including SQL development, partitioning, clustering, materialized views, optimization techniques, query execution analysis, and cost management.
• Strong programming experience in Python for data processing, pipeline development, automation, and integration.
• Advanced SQL skills, including complex joins, aggregations, CTEs, window functions, analytical queries, query optimization, and troubleshooting.
• Hands-on experience with GCP data services such as: BigQuery Google Cloud Storage Dataflow Pub/Sub Dataproc Cloud Composer / Apache Airflow
• Good understanding of batch processing, real-time/streaming architecture, ETL/ELT patterns, data lakes, data warehouses, and modern cloud data platforms.
• Experience designing dimensional, analytical, and curated data models supporting reporting, BI, analytics, and machine learning use cases.
• Good understanding of BigQuery performance and cost optimization, including partition pruning, clustering, query optimization, slot utilization, storage optimization, and workload management.
• Experience with Dataform, dbt, or similar SQL-based data transformation frameworks is preferred.
• Experience with Cloud Functions, Cloud Run, Workflows, or other serverless GCP services is an advantage.
• Understanding of GCP security concepts including IAM, service accounts, encryption, Secret Manager, VPC Service Controls, and data access management.
• Familiarity with Git, CI/CD pipelines, Infrastructure as Code such as Terraform, automated testing, logging, monitoring, and deployment practices.
• Experience with Cloud Monitoring, Cloud Logging, or equivalent observability tools for production data pipelines is preferred.
• Exposure to BI and analytics tools such as Looker, Looker Studio, Power BI, or Tableau is an added advantage.
• Strong analytical, troubleshooting, and problem-solving skills with the ability to independently investigate complex data and pipeline issues.
• Strong communication skills with the ability to work effectively with technical teams, architects, business stakeholders, and cross-functional teams.
• Ability to translate business and functional requirements into scalable and maintainable technical data solutions.
• Ability to provide technical guidance, perform code reviews, establish development standards, and support junior engineers.
• Strong ownership mindset with a focus on data quality, scalability, performance, security, cost efficiency, reliability, and timely delivery.
• Ability to work effectively in distributed and agile delivery teams and manage multiple priorities in a fast-paced environment.

Additional Information About Our Company | Accenture

About Accenture Accenture is a leading global professional services company that helps the world’s leading businesses, governments and other organizations build their digital core, optimize their operations, accelerate revenue growth and enhance citizen services—creating tangible value at speed and scale. We are a talent- and innovation-led company with approximately 791,000 people serving clients in more than 120 countries. Technology is at the core of change today, and we are one of the world’s leaders in helping drive that change, with strong ecosystem relationships. We combine our strength in technology and leadership in cloud, data and AI with unmatched industry experience, functional expertise and global delivery capability. Our broad range of services, solutions and assets across Strategy & Consulting, Technology, Operations, Industry X and Song, together with our culture of shared success and commitment to creating 360° value, enable us to help our clients reinvent and build trusted, lasting relationships. We measure our success by the 360° value we create for our clients, each other, our shareholders, partners and communities. Visit us at www.accenture.com

Equal Employment Opportunity Statement

We believe that no one should be discriminated against because of their differences.All employment decisions shall be made without regard to age, race, creed, color, religion, sex, national origin, ancestry, disability status, military veteran status, sexual orientation, gender identity or expression, genetic information, marital status, citizenship status or any other basis as protected by applicable law.Our rich diversity makes us more innovative, more competitive, and more creative, which helps us better serve our clients and our communities.

Bring your incredible skills and join our global team of innovators. We come together from different backgrounds across the world and work with the latest technologies to create value and growth for our clients. With us, you’ll continue to learn and grow so you can advance in your career. Your personal dreams and ambitions are just as important to us; that’s why we offer support any way we can—when you thrive, we all thrive. Explore your next step at Accenture Belong. Grow. Thrive. Join agreat place to work for reinventors who drive meaningful change for our clients, communities, and the world. Wo rld. Explore your next step at Accenture