Senior Data engineer

Ample Success HR

India

Remote

INR 2,500,000 - 4,000,000

Full time

8 days ago

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Ample Success HR seeks a Senior Data Engineer to design, develop, and optimize enterprise data pipelines using Databricks and Apache Spark for an Insurance/BFSI client.

You will implement robust data warehousing, enable BI consumption via Power BI and Tableau, and collaborate with cross-functional teams to meet business objectives.

Qualifications

  • 5+ years of hands-on data engineering, data integration, ETL/ELT development, or data warehousing.
  • Strong practical experience with Databricks and Apache Spark, including distributed data processing.
  • Advanced proficiency in SQL, including complex queries, query optimization, and database performance tuning.
  • Strong understanding of data warehousing architecture, dimensional modeling, and ETL/ELT frameworks.
  • Experience designing and implementing scalable, high-performance data pipelines.
  • Familiarity with Power BI and Tableau for reporting, visualization, and analytics.
  • Good understanding of relational databases, data transformation, and data integration methodologies.
  • Strong analytical, problem-solving, and troubleshooting skills.
  • Excellent communication skills and the ability to collaborate with technical and business stakeholders.

Responsibilities

  • Design, develop, and maintain scalable end-to-end data pipelines for enterprise data integration and analytics.
  • Build and optimize ETL/ELT workflows using Databricks and Apache Spark.
  • Design and implement robust data warehousing solutions to support enterprise reporting and business intelligence requirements.
  • Integrate data from multiple structured and semi-structured sources while ensuring data quality, consistency, and reliability.
  • Develop complex SQL queries, stored procedures, and data transformation logic.
  • Perform SQL query optimization, database performance tuning, and troubleshooting.
  • Collaborate with data architects, business analysts, and stakeholders to understand data requirements and deliver efficient solutions.
  • Support data modeling, data validation, and data governance best practices.
  • Enable seamless data consumption through BI and reporting tools such as Power BI and Tableau.
  • Monitor, troubleshoot, and enhance existing data pipelines to improve performance and scalability.
  • Work with cross-functional teams to deliver data engineering solutions aligned with insurance business objectives.

Skills

Databricks
Apache Spark
SQL
Data integration
Data warehousing
ETL/ELT
Power BI
Tableau
Dimensional modeling
Data governance

Tools

Delta Lake
PySpark
Python
Azure
AWS
GCP

Job description

Job Description – Senior Data Engineer (Databricks & Apache Spark)Job Title: Senior Data EngineerExperience Required: 5+ YearsIndustry/Domain: Insurance / BFSIWe are seeking a highly skilled and experienced Senior Data Engineer to design, develop, and implement end-to-end data pipelines and enterprise data warehousing solutions for a leading insurance enterprise client. The ideal candidate should have strong expertise in Databricks, Apache Spark, SQL, data integration, and database performance optimization, along with a solid understanding of data warehousing concepts and business intelligence tools.

Key Responsibilities
  • Design, develop, and maintain scalable end-to-end data pipelines for enterprise data integration and analytics.
  • Build and optimize ETL/ELT workflows using Databricks and Apache Spark.
  • Design and implement robust data warehousing solutions to support enterprise reporting and business intelligence requirements.
  • Integrate data from multiple structured and semi-structured sources while ensuring data quality, consistency, and reliability.
  • Develop complex SQL queries, stored procedures, and data transformation logic.
  • Perform SQL query optimization, database performance tuning, and troubleshooting.
  • Collaborate with data architects, business analysts, and stakeholders to understand data requirements and deliver efficient solutions.
  • Support data modeling, data validation, and data governance best practices.
  • Enable seamless data consumption through BI and reporting tools such as Power BI and Tableau.
  • Monitor, troubleshoot, and enhance existing data pipelines to improve performance and scalability.
  • Work with cross-functional teams to deliver data engineering solutions aligned with insurance business objectives.
Required Skills & Qualifications
  • 5+ years of hands-on experience in data engineering, data integration, ETL/ELT development, or data warehousing.
  • Strong practical experience with Databricks and Apache Spark, including distributed data processing.
  • Advanced proficiency in SQL, including complex queries, query optimization, and database performance tuning.
  • Strong understanding of data warehousing architecture, dimensional modeling, and ETL/ELT frameworks.
  • Experience designing and implementing scalable, high-performance data pipelines.
  • Familiarity with Power BI and Tableau for reporting, visualization, and analytics.
  • Good understanding of relational databases, data transformation, and data integration methodologies.
  • Strong analytical, problem-solving, and troubleshooting skills.
  • Excellent communication skills and the ability to collaborate with technical and business stakeholders.
Preferred / Good-to-Have Skills
  • Experience working with insurance or BFSI enterprise clients.
  • Knowledge of insurance-related data, including policy, claims, underwriting, and customer information.
  • Experience with Delta Lake, PySpark, and Python.
  • Familiarity with cloud platforms such as Azure, AWS, or GCP.
  • Understanding of data governance, data security, and regulatory compliance.
  • Experience with workflow orchestration, CI/CD, and automated data pipeline deployment.

Interested in this role?

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Data Engineer Databricks / PySpark
Senior Data Engineer Databricks / PySpark

Wissen Technology Pte Ltd • Bengaluru, Mumbai

On-site
INR 1,500,000 - 2,300,000
Data Engineer
Data Engineer

Ambiquest Hyderabad • Dadri, Hyderabad, Bengaluru

Hybrid
INR 2,800,000 - 4,600,000
Senior Data Engineer
Senior Data Engineer

Amgen Technology • Hyderabad

Hybrid
INR 2,500,000 - 4,000,000
Sr. Data Engineer - Databricks
Sr. Data Engineer - Databricks

Anblicks Inc. • Hyderabad

On-site
INR 3,000,000 - 5,500,000
Senior Data Engineer
Senior Data Engineer

Wissen Technology Pte Ltd • Bengaluru

Hybrid
INR 3,000,000 - 5,400,000
IN_Senior Associate_Databricks Data Engineer_GCC_Advisory_Bangalore
IN_Senior Associate_Databricks Data Engineer_GCC_Advisory_Bangalore

PricewaterhouseCoopers Services LLP • Bengaluru

On-site
INR 1,800,000 - 2,400,000
Databricks - Data Engineer
Databricks - Data Engineer

Tredence Inc. • Bengaluru

On-site
INR 1,500,000 - 2,500,000
Senior Associate - Artificial Intelligence, Machine Learning and Data Science
Senior Associate - Artificial Intelligence, Machine Learning and Data Science

PriceWaterhouseCoopers Pvt Ltd ( PWC ) • Gurugram District

On-site
INR 3,000,000 - 4,200,000
Senior Data Engineer
Senior Data Engineer

EduRun Group • Indore District

On-site
INR 1,500,000 - 2,500,000
Senior Data Engineer (Databricks Lead)
Senior Data Engineer (Databricks Lead)

Prodapt • Bengaluru

On-site
INR 4,000,000 - 7,000,000