Senior GCP Data Engineer – Big Data & ETL Expert

Tata Consultancy Services Limited; Charlotte (NC); On-site; USD 100,000 - 120,000; Full time

Benefits offered by this job

Discretionary Annual Incentive

Comprehensive Medical Coverage

Family & Parental Leaves

Auto & Home Insurance

Certification & Training Reimbursement

Vacation & Holidays

401K Plan

Job summary

Tata Consultancy Services Limited in Charlotte, NC is recruiting a GCP Data Engineer with 6+ years of hands-on experience in Big Data and analytics. You will design and implement data pipelines using Python, PySpark, GCP (BigQuery, DataProc, Pub/Sub), and ETL processes.

Strong SQL skills and experience with CI/CD, version control, and Airflow are expected. Location is on-site in Charlotte with competitive salary between $100,000 and $120,000 per year and comprehensive benefits.

Qualifications

• 6+ years of hands-on software development experience with Big Data & Analytics solutions.
• Experience with GCP tools (BigQuery, DataProc, Pub/Sub) and Airflow.
• Strong SQL skills; design/optimize data models for performance and scale.
• Experience with CI/CD pipelines, version control (Git), and scripting.

Responsibilities

• 6+ years of hands-on software development experience with Big Data & Analytics solutions GCP Cloud - Big Query, Airflow, DataProc, PubSub,Hadoop Hive, Spark, Python, shell scripting,
• Strong experience in designing, developing, and optimizing data pipelines for large-scale data processing, transformation, and analysis using Big Data and GCP technologies.
• Proficiency in SQL and database systems, with experience in designing and optimizing data models for performance and scalability.
• Experience of building event processing pipelines with Kafka or GCP PubSub.
• Design and development experience with Airflow, PubSub Kafka, Git, Jenkins is desirable.
• Knowledge of distributed (multi-tiered) systems, algorithms & relational databases.
• Strong Object-Oriented Programming skills and design patterns.
• Experience with CICD pipelines, Automated test frameworks, and source code management tools (XLR, Jenkins, Git).
• Good knowledge and experience with configuration management tools like GitHubAbility to analyze complex data engineering problems, propose effective solutions, and implement them effectively.
• Ability and willingness to learn, adopt and build solutions using the Enterprise frameworks established for Big DataLooks proactively beyond the obvious for continuous improvement opportunities.
• Communicate effectively with product and cross functional team.
• Willingness to learn new technologies and leverage them to their optimal potential.Understanding of various SDLC methodologies, familiarity with Agile & scrum ceremonies.
• Certifications in cloud platform (GCP Professional Data Engineer) is a plus.

Skills

Python

PySpark

GCP

ETL

Big Data / Data Warehousing

Data Analytics

SQL

Git

Job description

Tata Consultancy Services Limited in Charlotte, NC is recruiting a GCP Data Engineer with 6+ years of hands-on experience in Big Data and analytics. You will design and implement data pipelines using Python, PySpark, GCP (BigQuery, DataProc, Pub/Sub), and ETL processes.

Strong SQL skills and experience with CI/CD, version control, and Airflow are expected. Location is on-site in Charlotte with competitive salary between $100,000 and $120,000 per year and comprehensive benefits.