Specialist Software Engineer - Python + Spark - (26000KL6)

SA Technologies

Bengaluru

On-site

INR 3,500,000 - 7,000,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Great Place to Work
Career Growth Opportunities
Diversity & Inclusion

Job summary

SA Technologies Inc. in Bengaluru seeks a Senior Software Engineer with deep Databricks, AWS, and Python experience to design and lead enterprise-scale AI-enabled data solutions. You will own data pipelines, governance, and observability in a fast-paced, global delivery environment.

The role emphasizes scalable ETL/ELT design, cloud migrations on AWS, and collaboration with data scientists, architects, and product owners. Strong mentoring and leadership are expected.

Qualifications

  • Deep hands-on experience with Databricks (PySpark, SQL, Delta Lake) in large-scale production environments.
  • Proven experience designing and leading end-to-end data pipelines on Databricks.
  • Strong hands-on experience with Delta Live Tables for governed data processing.
  • Advanced experience with Unity Catalog for data governance, access control, lineage, and metadata.
  • Expertise in Spark performance tuning, scalability, and cost optimization.
  • Experience managing Databricks workflows, monitoring, and production operations.
  • Strong knowledge of cloud-based data platforms (AWS preferred).
  • Proven ability to mentor engineers and lead technical delivery.

Responsibilities

  • Design, develop, and maintain scalable data engineering solutions using Databricks, PySpark, SQL, and Delta Lake.
  • Build and optimize batch and real-time data pipelines on AWS using services such as S3, Lambda, Glue, EMR, Redshift, and related cloud-native components.
  • Develop, implement, and support Delta Live Tables (DLT) pipelines for reliable, high-quality, and auditable data processing.
  • Work with Unity Catalog to implement data governance, access control, lineage, and security best practices.
  • Develop and optimize ETL/ELT pipelines with a focus on performance tuning, scalability, reliability, and cost efficiency.
  • Implement infrastructure and platform automation using Terraform and Infrastructure as Code (IaC) principles.
  • Build and maintain CI/CD pipelines using Jenkins, Git, and related DevOps tools to enable automated deployments and testing.
  • Support deployment, monitoring, and maintenance of Databricks workloads across development, testing, and production environments.
  • Collaborate with architects, product owners, data scientists, and business stakeholders to understand requirements and deliver high-quality solutions.
  • Participate in cloud migration, modernization, and data platform transformation initiatives on AWS.
  • Contribute to data quality, observability, logging, monitoring, and alerting frameworks to ensure platform reliability.
  • Work with APIs, data integration frameworks, and external systems to enable seamless data exchange across platforms.
  • Troubleshoot production issues, perform root cause analysis, and drive continuous improvement initiatives.
  • Follow software engineering best practices, participate in code reviews, and ensure adherence to coding, security, and operational standards.
  • Work effectively within Agile teams, contributing to sprint planning, estimation, development, and release activities.
  • Collaborate with global teams and support business‑critical deliveries across different time zones when required.

Skills

Databricks
PySpark
SQL
Delta Lake
AWS
Python
Data governance
CI/CD
Terraform
Jenkins
Git
Data quality

Education

Bachelor's degree in CS/Engineering

Tools

Terraform
Jenkins
Git
S3/AWS services

Job description

Unlock New Career Horizons with SA Technologies
Inc.

At SA Technologies Inc., we believe in turning career
aspirations into reality. As an NSE Listed company ,
recognized as a CMMI Level 5 organization , a Great Place
to Work Certified company , and one of the Best Places for
Women , we are committed to creating a workplace that is
inclusive, excellent, and supportive

Role Summary

Experience: 5–8 Years

  • Design, develop, and maintain scalable data engineering solutions using Databricks, PySpark, SQL, and Delta Lake.
  • Build and optimize batch and real-time data pipelines on AWS using services such as S3, Lambda, Glue, EMR, Redshift, and related cloud-native components.
  • Develop, implement, and support Delta Live Tables (DLT) pipelines for reliable, high-quality, and auditable data processing.
  • Work with Unity Catalog to implement data governance, access control, lineage, and security best practices.
  • Develop and optimize ETL/ELT pipelines with a focus on performance tuning, scalability, reliability, and cost efficiency.
  • Implement infrastructure and platform automation using Terraform and Infrastructure as Code (IaC) principles.
  • Build and maintain CI/CD pipelines using Jenkins, Git, and related DevOps tools to enable automated deployments and testing.
  • Support deployment, monitoring, and maintenance of Databricks workloads across development, testing, and production environments.
  • Collaborate with architects, product owners, data scientists, and business stakeholders to understand requirements and deliver high-quality solutions.
  • Participate in cloud migration, modernization, and data platform transformation initiatives on AWS.
  • Contribute to data quality, observability, logging, monitoring, and alerting frameworks to ensure platform reliability.
  • Work with APIs, data integration frameworks, and external systems to enable seamless data exchange across platforms.
  • Troubleshoot production issues, perform root cause analysis, and drive continuous improvement initiatives.
  • Follow software engineering best practices, participate in code reviews, and ensure adherence to coding, security, and operational standards.
  • Work effectively within Agile teams, contributing to sprint planning, estimation, development, and release activities.
  • Collaborate with global teams and support business‑critical deliveries across different time zones when required.
Profile

Profile Required: Senior‑level Software Engineer with deep Databricks, AWS, and Python data platform expertise, capable of designing and leading enterprise‑scale AI‑enabled data solutions.

Minimum Qualifications
  • Deep hands‑on experience with Databricks (PySpark, SQL, Delta Lake) in large‑scale production environments.
  • Proven experience designing and leading end‑to‑end data pipelines on Databricks.
  • Strong hands‑on experience with Delta Live Tables (DLT) for governed data processing.
  • Advanced experience with Unity Catalog for data governance, access control, lineage, and metadata.
  • Expertise in Spark performance tuning, scalability, and cost optimization.
  • Experience managing Databricks workflows, monitoring, and production operations.
  • Strong knowledge of cloud‑based data platforms (AWS preferred).
  • Proven ability to mentor engineers and lead technical delivery.
Preferred Qualifications
  • Experience building and operating AI agents for workflow automation and decision support.
  • Strong understanding of data observability, platform reliability, and FinOps practices in Databricks.
  • Experience working in regulated or financial services domains.
  • Exposure to enterprise‑scale AI governance, compliance, and audit requirements.
  • Ability to influence architecture decisions and align stakeholders.
  • Excellent communication skills with both technical and non‑technical audiences.
Why SA Technologies Inc.?

Empowering Work Environment: A Great Place to Work, especially celebrated as one of the Best Places for Women.

Excellence and Quality: A CMMI Level 5 Company, ensuring operational and service excellence.

Globally Acclaimed: A Microsoft Gold Partner and Google Partner, with over 20 years of global IT consulting and Development solutions deliver.

Embark on a rewarding journey together with us! At
SA Technologies Inc., your journey begins with a friendly
technical assessment, providing a stage to highlight your unique
skills.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Specialist Software Engineer - Python + Spark - (26000KL6)
Specialist Software Engineer - Python + Spark - (26000KL6)

SA Technologies • Bengaluru South

On-site
INR 1,200,000 - 1,800,000
Great Place to Work
Women-friendly workplace
CMMI Level 5 company
+1
Senior Databricks Data Engineer
Senior Databricks Data Engineer

Saarthee • Bengaluru

On-site
INR 1,200,000 - 2,000,000
Senior Data Engineer
Senior Data Engineer

Wissen • Bengaluru

Hybrid
INR 2,800,000 - 4,800,000
Senior Data Engineer (APAC Region)
Senior Data Engineer (APAC Region)

Anrgi Tech Private Limited • Pune District

On-site
INR 2,000,000 - 2,800,000
Senior Specialist - Data Engineering
Senior Specialist - Data Engineering

Vivantify Technology Solutions India Pvt. Ltd. • Maharashtra

On-site
INR 1,500,000 - 2,100,000
Competitive salary
Growth opportunities
Collaborative environment
+1
Senior Specialist - Data Engineering
Senior Specialist - Data Engineering

Vivantify Technology Solutions India Private Limited • Pune District

On-site
INR 1,800,000 - 3,200,000
Competitive salary and benefits
Collaborative and inclusive team
Growth and development opportunities
Pyspark Engineer
Pyspark Engineer

Citi Group • Chennai

On-site
INR 1,200,000 - 2,400,000
Pyspark developer
Pyspark developer

Aligned Automation • Pune District

On-site
INR 2,200,000 - 3,400,000
Pyspark Engineer
Pyspark Engineer

Citi • India

On-site
INR 900,000 - 1,500,000
Data Engineer
Data Engineer

SA Technologies Inc. • Pune District

On-site
INR 1,400,000 - 2,200,000