Senior Data Engineer

Principal Financial Group; Maharashtra; On-site; INR 1,800,000 - 3,800,000; Full time

Job summary

Principal Financial Group seeks a Senior Data Engineer to design and deploy world-class data solutions for actuarial and enterprise data initiatives on AWS. You will collaborate with actuaries, build scalable data lakes, and craft CI/CD pipelines using GitHub Actions, Docker, AWS CDK, and AWS SAM, while migrating existing data applications to AWS and optimizing performance.

You'll work with PySpark and Python to develop ETL jobs, manage Glue workflows, and implement best practices in data lake

Qualifications

• Hands-on experience with actuarial integrate tools (Milliman) and actuarial SaaS platforms.
• Strong understanding of actuarial concepts, terminology, calculations, and modeling processes.
• 4 to 7 years of hands-on experience in AWS Data Engineering, ETL development, and cloud-based data platform implementations.
• Extensive experience with AWS cloud services, including: AWS Glue, Athena, Lambda, S3, CloudFormation (CFN), AWS CDK, SNS, IAM, RDS, DynamoDB, CloudWatch, Boto3.
• Proficiency in Python and PySpark for large-scale data processing and ETL development.
• Experience developing and managing CI/CD pipelines with GitHub Actions and Docker.
• Practical experience in applying Infrastructure as Code (IaC) through AWS CDK and CloudFormation.
• Strong knowledge of data lake architecture, ETL frameworks, and cloud-based data engineering practices.
• Experience with application deployment using AWS SAM.
• Strong problem-solving, analytical, and troubleshooting skills.

Responsibilities

• Collaborate with actuaries to gather, analyze, and translate actuarial requirements into technical solutions.
• Understand and support actuarial integration tools and related SaaS platforms.
• Compose, build, and maintain scalable data lakes on AWS to support actuarial and enterprise data initiatives.
• Create and maintain CI/CD pipelines using GitHub Actions, Docker, AWS CDK, and AWS SAM.
• Participate in architecture discussions and contribute to crafting new cloud-native solutions while migrating existing data applications to AWS.
• Develop and optimize AWS Glue ETL jobs using PySpark, Boto3, and AWS CDK.
• Build, implement, and maintain ETL frameworks and data processing solutions following established guidelines.
• Develop, test, and deploy ETL scripts using PyTest and containerized environments.
• Improve database performance through query optimization, indexing strategies, partitioning, and data modeling techniques.
• Transition current AWS CloudFormation (CFN) infrastructure to AWS CDK and develop reusable infrastructure-as-code frameworks.

Skills

Python

PySpark

AWS

CI/CD

GitHub Actions

Docker

Data modeling

Tools

AWS Glue

Athena

Lambda

S3

CFN

AWS CDK

SNS

IAM

RDS

DynamoDB

CloudWatch

Boto3

PySpark

Docker

GitHub Actions

AWS SAM

Job description

Responsibilities

At Principal, we don't just offer jobs; we offer careers that propel you forward. As a Senior Data Engineer, you will be at the forefront of crafting and implementing world-class data solutions that drive our actuarial and enterprise data initiatives. This role provides the outstanding opportunity to work with modern technologies and contribute to pioneering projects that have a tangible impact on our business and clients.

• Collaborate with actuaries to gather, analyze, and translate actuarial requirements into technical solutions.


• Understand and support actuarial integration tools and related SaaS platforms.


• Compose, build, and maintain scalable data lakes on AWS to support actuarial and enterprise data initiatives.


• Create and maintain CI/CD pipelines using GitHub Actions, Docker, AWS CDK, and AWS SAM.


• Participate in architecture discussions and contribute to crafting new cloud-native solutions while migrating existing data applications to AWS.


• Develop and optimize AWS Glue ETL jobs using PySpark, Boto3, and AWS CDK.


• Build, implement, and maintain ETL frameworks and data processing solutions following established guidelines.


• Develop, test, and deploy ETL scripts using PyTest and containerized environments.


• Improve database performance through query optimization, indexing strategies, partitioning, and data modeling techniques.


• Transition current AWS CloudFormation (CFN) infrastructure to AWS CDK and develop reusable infrastructure-as-code frameworks.

Qualifications

Must-Have Skills

• Hands-on experience with actuarial integrate tools (Milliman) and actuarial SaaS platforms.


• Strong understanding of actuarial concepts, terminology, calculations, and modeling processes.


• We are looking for candidates with 4 to 7 years of hands-on experience in AWS Data Engineering, ETL development, and cloud-based data platform implementations.


• Extensive experience with AWS cloud services, including:


• AWS Glue, Athena, Lambda, S3, CloudFormation (CFN), AWS CDK, SNS, IAM, RDS, DynamoDB, CloudWatch, Boto3.


• Proficiency in Python and PySpark for large-scale data processing and ETL development.


• Experience developing and managing CI/CD pipelines with GitHub Actions and Docker.


• Practical experience in applying Infrastructure as Code (IaC) through AWS CDK and CloudFormation.


• Strong knowledge of data lake architecture, ETL frameworks, and cloud-based data engineering practices.


• Experience with application deployment using AWS SAM.


• Strong problem-solving, analytical, and troubleshooting skills.

Good-to-Have Skills

• Experience working on data-intensive and large-scale analytics projects.


• Knowledge of insurance, actuarial, or financial services domains.


• Familiarity with data governance, data quality, and data security guidelines.

Join us and be part of a team that values innovation, collaboration, and excellence. Your contributions will help us achieve flawless execution in every project and drive our business forward. Let's compete on a global scale and make a difference together!

Additional Information

Our Engineering Culture

Through our product-driven Agile/Lean DevOps environment, we’ve fostered a culture of innovation and experimentation across our development teams. As a customer-focused organization, we work closely with our end users and product owners to understand and rapidly respond to emerging business needs.

Collaboration is embedded into everything we do – from the products we develop to the quality service we provide. We’re driven by the belief that diversity of thought, background, and perspective is critical to creating the best products and experiences for our customers.

Experience Principal

At Principal, we value connecting on both a personal and professional level. Together, we’re imagining a more purpose-led future for financial services – and that starts with you. Our success depends on the unique experiences, backgrounds, and talents of our employees. And we support our employees the same way we support our customers: with comprehensive, competitive benefit offerings crafted to protect their physical, financial, and social well-being. Check out our careers site to learn more about our purpose, values and benefits.

Principal is an Equal Opportunity Employer

All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or veteran status.