Lead Cloud Data Engineer

Raymond James Financial, Inc.; Memphis (TN); Híbrido; USD 120 000 - 150 000; Tempo integral

Resumo da oferta

Raymond James Financial, Inc. is seeking a data engineering professional to design scalable ETL pipelines using AWS Glue and PySpark. You will integrate data into Redshift, optimize queries, and deploy Terraform-based infrastructure while mentoring junior engineers and collaborating with data scientists.

The role emphasizes security compliance, documentation of workflows, and continuous exploration of AWS services to improve solutions. Hybrid in-office requirements apply.

Qualificações

• Expert level experience with AWS Cloud Services and data engineering best practices.
• Experience with Python or Java and data processing pipelines.
• Familiarity with analytics, data warehouse concepts, and batch/streaming patterns.
• Ability to document coding and architecture standards for teams.

Responsabilidades

• Design and develop scalable ETL pipelines using AWS Glue and PySpark.
• Integrate data from various sources into Amazon Redshift, ensuring data quality and consistency.
• Optimize the performance of data processing jobs and Redshift queries.
• Deploy infrastructure as code (IaC) with Terraform.
• Monitor and troubleshoot data pipelines to ensure reliability and efficiency.
• Collaborate with data scientists, analysts, and stakeholders to understand data requirements.
• Ensure compliance with security policies and regulations.
• Document data workflows and best practices.
• Mentor and guide junior engineers on the team.
• Continuously explore new AWS services and technologies to improve existing solutions.

Conhecimentos

AWS Cloud Services

PySpark

Redshift

S3

Kafka

Glue

AirFlow

SageMaker

EMR

Lake Formation

DynamoDB

Aurora

VPC

IAM

Ferramentas

Terraform

Python

PySpark

AWS Glue

Redshift

S3

AirFlow

Terraform

Descrição da oferta de emprego

This position will follow our hybrid work model, we expect the selected candidate to be in office 2-3 days a week(10-12 days a month ) at the following office location: St. Petersburg, FL/ Memphis, TN/ Denver, CO/Southfield, MI

Responsibilities:

• Design and develop scalable ETL pipelines using AWS Glue and PySpark.
• Integrate data from various sources into Amazon Redshift, ensuring data quality and consistency.
• Optimize the performance of data processing jobs and Redshift queries.
• Deploy infrastructure as code (IaC) with Terraform
• Monitor and troubleshoot data pipelines to ensure reliability and efficiency.
• Collaborate with data scientists, analysts, and other stakeholders to understand data requirements.
• Ensure compliance with security policies and regulations.
• Document data workflows and best practices.
• Mentor and guide junior engineers on the team.
• Continuously explore new AWS services and technologies to improve existing solutions.

Skills:

• Expert level experience with AWS Cloud Services is preferred, knowledge of other Cloud Providers is a plus
• Experience and understanding of AWS Cloud Services components like Redshift, S3, Kafka, Glue, AirFlow, SageMaker, EMR, Athena, Lake Formation, DynamoDB, Aurora, VPC, IAM and many other Cloud Service modules.
• Working experience in Python is preferred, Java and other languages are a plus
• Nice to have Terraform or other Infrastructure as Code experience
• Knowledge and experience with Analytics, Data Warehouse and Big Data concepts across both batch and streaming patterns
• Solid understanding of networking, infrastructure, cost management, SRE concepts
• Creating patterns for adoption from scratch and being able to document and articulate coding and architecture standards for others to follow.
• Performance tuning and optimization of code and infrastructure
• Creating a culture of continuous improvement, performs code reviews and makes recommendations for improvement or streamlining patterns
• Relevant experience in various database platforms, ETL solutions/products, Infrastructure, ETL architecture and On-Prem or Cloud Services.
• Financial Services Industry knowledge is a plus.

Ability to:

• Identify and understand issues, problems and opportunities for automation.
• Good communication and organization skills that clearly convey information and ideas through a variety of media to individuals or groups in a manner that engages the audience and helps them understand and retain the message.
• Use effective approaches for choosing a course of action or developing appropriate solutions; recommend or take action that is consistent with available facts, constraints and probable consequences.
• Demonstrate a satisfactory level of technical and professional skill or knowledge in position-related areas; remains current with developments and trends in areas of expertise.
• Develop and use collaborative relationships to facilitate the accomplishment of work goals.
• Make internal and external clients and their needs a primary focus of actions; develop and sustain productive client relationships.

Certifications

Preferred, not required

• AWS Certified Solutions Architect
• AWS Certified Big Data
• AWS Certified Data Analytics
• AWS Certified Developer
• AWS Certified Machine Learning