Senior Software Engineer - Data Engineering

Caterpillar Brazil; Chennai District, Bengaluru; On-site; INR 3,500,000 - 6,000,000; Full time

Job summary

Caterpillar Inc. is seeking a Senior Software Engineer - Data Engineering in India to design and build scalable data pipelines on AWS and Snowflake, using Python and SQL for data processing.

The role includes collaboration with data scientists and business stakeholders to deliver reliable data solutions. The ideal candidate has 8+ years in data engineering, hands-on experience with graph and vector databases, and familiarity with DevOps practices for agile environments.

Qualifications

• 8+ years of data engineering or related roles.
• Hands-on AWS cloud services experience for data and AI workloads.
• Deep Snowflake architecture knowledge and performance tuning.
• Advanced Python and SQL for pipelines and data processing.
• Experience with graph and vector databases (Neo4j/Neptune, Milvus/OpenSearch).
• Version control with Git and Azure DevOps for backlog management.
• Degree in CS/Engineering at bachelor’s or master’s level.

Responsibilities

• Design, develop, and maintain scalable data pipelines on AWS (S3, Glue, Lambda, Redshift, EMR).
• Build and optimize Snowflake data warehousing and data modeling.
• Write efficient Python and SQL for transformations and processing.
• Collaborate with data scientists, analysts and business stakeholders.
• Integrate vector and graph DBs and AI workflows; ensure quality and security.

Skills

Analytical thinking

Problem solving

Communication

Collaboration

Education

Bachelor's or Master's degree in Computer Science or Engineering

Tools

AWS

Snowflake

Python

SQL

Neo4j / Neptune

Milvus / OpenSearch

Git

Azure DevOps

Kafka / Spark Streaming

Job description

## Senior Software Engineer - Data EngineeringApplylocations: Chennai, Tamil Nadu: Bangalore, Karnatakatime type: Full timeposted on: Posted Todayjob requisition id: R0000386238**Career Area:**Technology, Digital and Data**Job Description:****Your Work Shapes the World at Caterpillar Inc.**When you join Caterpillar, you're joining a global team who cares not just about the work we do – but also about each other. We are the makers, problem solvers, and future world builders who are creating stronger, more sustainable communities. We don't just talk about progress and innovation here – we make it happen, with our customers, where we work and live. Together, we are building a better world, so we can all enjoy living in it.**Job Summary**We are looking for a highly motivated and experienced Data Engineer to join our data engineering team. The ideal candidate will have a strong background in building scalable data pipelines using the AWS cloud stack and extensive hands-on experience with Snowflake. Proficiency in Python and SQL, along with graph and vector database technologies, is essential. This role requires strong problem-solving abilities and a proactive mindset to deliver efficient, scalable, and reliable data solutions.**Key Responsibilities*** Design, develop, and maintain scalable data pipelines on AWS using services such as S3, Glue, Lambda, Redshift, and EMR.* Build and optimize data warehousing solutions using Snowflake, including performance tuning and data modeling.* Write efficient and reusable code in Python and SQL for data transformation and processing.* Collaborate with cross-functional teams, including data scientists, analysts, and business stakeholders, to understand data requirements.* Integrate vector databases with LLM-based applications and AI workflows.* Monitor, troubleshoot, and improve pipeline performance and reliability.* Ensure data quality, integrity, and security across all stages of the pipeline.* Participate in code reviews, architecture discussions, and continuous improvement initiatives.**Required Qualifications*** 8+ years of experience in data engineering or related roles.* Strong hands-on experience with AWS cloud services, including data and AI workloads.* Deep understanding of Snowflake architecture, performance tuning, and best practices.* Advanced proficiency in Python and SQL for data pipelines, transformations, and services.* Hands-on experience with graph databases (e.g., Neo4j, Neptune) and vector databases (e.g., Milvus, Amazon OpenSearch).* Experience with version control systems (e.g., Git) and Git workflows.* Experience working with Azure DevOps (AzDO) boards for backlog management in Agile environments.* Excellent analytical and problem-solving skills.* Strong communication and collaboration abilities.* Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.**Nice to Have skills*** Knowledge of the NVIDIA ecosystem and its applications in data and AI.* Exposure to RAPIDS libraries (cuDF, cuML, cuGraph) or CUDA-based tooling for GPU-accelerated data processing, enabling faster transformation and optimization during large-scale ingestion workflows.* Hands-on expertise with vector databases, specifically Milvus, covering schema design, indexing, and optimizing write performance for large-scale embedding ingestion pipelines.* Proficiency in building Knowledge Graph (Neo4J) ingestion pipelines using Graph Databases — including entity extraction, relationship modelling, and populating nodes and attributes.**Preferred Qualifications*** Experience with orchestration tools such as AWS Step Functions.* Familiarity with data governance and compliance practices.* Exposure to real-time data processing frameworks (e.g., Kafka, Spark Streaming).This position requires working onsite five days a week.Relocation is available for this position.**Posting Dates:**August 7, 2026 - August 20, 2026