Walk-in | Aws Engineer

Tata Consultancy Services; Bengaluru; On-site; INR 2,800,000 - 5,200,000; Full time

Job summary

Tata Consultancy Services in Bengaluru is seeking a Technical Lead with 9+ years of hands-on experience in Python/PySpark, Spark SQL, and Databricks to lead the development of large-scale data pipelines. You will leverage AWS services and Kafka for streaming, and work with Airflow to orchestrate workflows.

The role requires strong SQL/PLSQL skills, data modeling expertise, and a proven ability to collaborate with cross-functional teams in a deadline-driven environment.

Qualifications

• Technical Lead with 9+ years of hands-on Python/PySpark.
• Strong experience in Spark, Databricks, and AWS services.
• 7+ years in SQL/PLSQL and data modeling.
• Kafka and Airflow expertise for streaming pipelines.
• Excellent communication and ability to work with multiple teams.
• Experience with Agile and GitHub.

Responsibilities

• Technical Lead (9+ years) with hands-on Python/PySpark, Spark SQL, Databricks, AWS services, Kafka, Airflow.
• Architect/design/develop data pipelines to meet complex deadlines with minimal supervision.
• End-to-end Databricks architecture, design, development, and implementation.
• Maintain 8+ years of AWS, Databricks, and Airflow expertise.
• Ensure quality, performance, and reliability of data platforms in production.
• Collaborate with stakeholders and vendors; communicate effectively.

Skills

Python

PySpark

Databricks

AWS Cloud

SQL

Apache Kafka

Airflow

Oracle

PL/SQL

Unix Shell

Autosys

PostgreSQL

NoSQL

Data Modeling

Tools

GitHub

Autosys

Unix Shell

Job description

Primary Skill Technical / Functional Skills

Python,PySpark/Spark, Databricks, AWS cloud, SQL, Apache Kafka

Secondary Skills - Technical / Functional Skills

Airflow,Oracle,PLSQL,SQL, Test Driven Design

Generic Managerial Skills (If Any)

NA

Roles & Responsibilities

Technical Lead (overall 9+ years good development experience) with good hands-on experience in Python/PySpark with strong experience in Spark /PySpark, Databricks, AWS services ,Spark SQL for big data processing and,Kafka, Airflow.

• Worked in a complex deadline driven projects with minimal supervision. Ability to architect/design/develop with minimum requirements by effectively.
• 2+ experience on Apache Kafka streaming processing and Python/PySpark.
• End to end understanding of Databricks architecture, design, development and implementation.
• **Min 8+ years of good experience on AWS clouds and Data bricks,Airflow.**
• Excellent work ethics with ability to work independently.
• Hands-on experience in SQL, PL/SQL, Unix Shell Scripting, Autosys.
• Experience with relational databases such as Oracle, PostgreSQL as well as NoSQL databases
• Data integration techniques and tools including APIs and data streaming
• Strong analytical and problem solving skills, with the ability to identify and resolve issues in data pipelines and systems
• Experience with Agile software development methodology and version control systems like GitHub
• Expertise in Data Modeling and DB design with the skills in performance tuning
• Support existing data platforms and products in production for any critical issues and provide resolution fixes
• Have good communication skill, and collaborate well with different teams/stakeholders and vendors.