Data Engineer / AI Engineer

Tata Consultancy Services; Bengaluru; On-site; INR 1,800,000 - 3,200,000; Full time

Job summary

Tata Consultancy Services in Bengaluru is seeking an experienced AI / Python / PySpark Developer to build scalable data pipelines, develop AI/ML solutions, and optimize cloud data platforms.

The role requires strong Python and PySpark skills, GCP expertise with BigQuery, and experience with Astronomer (Airflow) to design, implement and orchestrate data workflows for large-scale analytics.

Qualifications

• Strong programming experience in Python and PySpark.
• Experience building scalable data pipelines on cloud platforms.
• Knowledge of GCP services, BigQuery and data warehousing.
• Experience with Astronomer / Apache Airflow for workflow orchestration.
• Understanding of AI/ML concepts and model deployment.

Responsibilities

• Develop and maintain scalable data pipelines using Python and PySpark.
• Design and optimize data solutions on Google Cloud Platform (GCP).
• Work with BigQuery for large-scale data warehousing and analytics.
• Develop and orchestrate workflows using Astronomer (Apache Airflow).
• Build and deploy AI/ML models for business use cases.
• Perform data ingestion, transformation, and processing from multiple sources.
• Monitor pipeline performance and ensure data quality and reliability.
• Collaborate with cross-functional teams to deliver data-driven solutions.

Skills

Python programming

Data pipelines

Cloud data platforms

Airflow orchestration

Tools

Python

PySpark

Google Cloud Platform (GCP)

BigQuery

Astronomer / Airflow

Git

CI/CD practices

Job description

Role & responsibilities

Job Title: AI / Python / PySpark Developer
Experience: 6-8 Years
Location: Bengaluru

Job Description:

We are looking for an experienced AI & Data Engineer with strong expertise in Python, PySpark, GCP, BigQuery, and Astronomer (Airflow). The ideal candidate will be responsible for building scalable data pipelines, developing AI/ML solutions, and optimizing cloud-based data platforms.

Key Responsibilities

• Develop and maintain scalable data pipelines using Python and PySpark.
• Design, implement, and optimize data solutions on Google Cloud Platform (GCP).
• Work with BigQuery for large-scale data warehousing and analytics.
• Develop and orchestrate workflows using Astronomer (Apache Airflow).
• Build and deploy AI/ML models for business use cases.
• Perform data ingestion, transformation, and processing from multiple sources.
• Monitor pipeline performance and ensure data quality and reliability.
• Collaborate with cross-functional teams to deliver data-driven solutions.

Required Skills

• Strong programming experience in Python.
• Hands-on expertise in PySpark and distributed data processing.
• Experience with Google Cloud Platform (GCP) services.
• Strong knowledge of BigQuery.
• Experience with Astronomer / Apache Airflow for workflow orchestration.
• Understanding of AI/ML concepts and model deployment.
• Knowledge of SQL, ETL processes, and cloud-native architectures.
• Experience with Git and CI/CD practices.