Data Engineer Senior Developer, AVP

State Street; Quincy (MA); On-site; USD 90,000 - 157,500; Full time

Benefits offered by this job

401K with company match

Paid time off

Volunteer days

Employee networks

Job summary

State Street is seeking a highly skilled Data Engineer to design, build, and optimize data infrastructure and analytics initiatives. This hands-on technologist will lead data framework development, ensuring data integrity, scalability and security.

In this role, you will work with cloud data lake activities in AWS and Databricks, implement CI/CD pipelines, and guide a global team of engineers. A strong background in Spark, PySpark/Scala, and data modeling is essential for success.

Qualifications

• Bachelor’s/Master's Degree in Computer Science, Engineering or Math.
• 5+ years' experience in professional software design and development.
• Strong analytical skills with ability to absorb and analyze information quickly.
• Proven ability to work with cross-functional processes and timelines.
• Excellent follow-through, attention to detail and time management.
• Adapt and excel at managing multiple tasks in a high-pressure, deadline environment.

Responsibilities

• Collaborate with data scientists, analysts, and product teams to enable analytics and ML use cases.
• Ensure data quality, reliability, security, and governance across data platforms.
• Optimize performance and cost of data workloads in AWS and Databricks.
• Implement CI/CD pipelines and infrastructure as code (e.g., Terraform, CloudFormation).
• Monitor, troubleshoot, and resolve data pipeline and platform issues.
• Document data architecture, pipelines, and operational processes.
• End to end Cloud Data Lake design & development including data ingestion, data modeling and data distribution.
• Build data integrations, hand-offs between on‑prem/cloud‑based systems.
• Design, build, and maintain scalable, reliable data pipelines using AWS services and Databricks.
• Develop and optimize ETL/ELT workflows for batch and near-real-time data processing.
• Implement data solutions using Apache Spark (PySpark/Scala) on Databricks.
• Leverage AWS services such as S3, Glue, Lambda, EMR, Redshift, Athena, and Kinesis.
• Highly motivated, with initiative, prioritization and independent task execution.
• Management oversight of a global team of software development and QA engineers.
• Build strong partnerships with technology & business leaders and peers.
• Work with Product Owner on requirements analysis, planning, and execution.

Skills

Python

Scala/PySpark

SQL/PLSQL

Databricks

AWS

Spark

Terraform

Education

Bachelor’s/Master's Degree in CS/Engineering/Math

Tools

ETL

Cloud platforms

Job description

Seeking a highly skilled data engineer, to design, build, and optimize our data infrastructure and analytics initiatives. As a hands‑on technology lead, you will contribute to the design, development, enhancement & maintenance of data frameworks, ensuring data integrity, scalability and security.

We are looking for a skilled Data Engineer for Cloud Data Lake activities. The candidate should have industry experience (preferably in Financial Services) in supporting enterprise applications and exposure to Cloud based Data engineering platforms with deep understanding of data processing frameworks and solid experience with ETL development using distributed computing frameworks as Apache Spark, Scala, Hadoop, Hive. This hands‑on role will partner with senior‑level development managers, architects and business leadership to develop and execute the technology product roadmap.

What You Will Be Responsible For

• Collaborate with data scientists, analysts, and product teams to enable analytics and ML use cases
• Ensure data quality, reliability, security, and governance across data platforms
• Optimize performance and cost of data workloads in AWS and Databricks
• Implement CI/CD pipelines and infrastructure as code (e.g., Terraform, CloudFormation)
• Monitor, troubleshoot, and resolve data pipeline and platform issues
• Document data architecture, pipelines, and operational processes
• End to end Cloud Data Lake design & development including data ingestion, data modeling and data distribution.
• Build data integrations, hand-offs between on‑prem/cloud‑based systems.
• Design, build, and maintain scalable, reliable data pipelines using AWS services and Databricks
• Develop and optimize ETL/ELT workflows for batch and near‑real‑time data processing
• Implement data solutions using Apache Spark (PySpark/Scala) on Databricks
• Leverage AWS services such as S3, Glue, Lambda, EMR, Redshift, Athena, and Kinesis
• Highly motivated, with the ability to take initiative, set priorities and perform tasks with minimal guidance
• Management oversight of a global team of software development and QA engineers.
• Building strong partnership with technology & business leaders and peers
• Work closely with the Product Owner and others on requirements analysis, planning, project management, and execution.

What We Value

These skills will help you succeed in this role

• Development experience in designing, developing and deploying enterprise applications.
• Strong skill‑set, experience in PL/SQL, Python, Scala/PySpark, PERL/scripting.
• Hands on experience with Cloud platforms, AWS, Databricks.
• Experience in building systems using modern, scalable, resilient, data lakes and architectures.
• Experience in Data Modeling and database design with strong skills in performance tuning.
• Hands on experience in setting up data pipelines for ELT jobs across multiple data sources.
• Experience with Delta Lake, Unity Catalog, and Databricks Workflows
• Strong knowledge of Cloud Security in AWS/Azure, specifically IAM Roles & Polices, Security Groups and Encryption keys.
• Experience with optimizing database performance, scalability, ensuring data security and compliance.
• Experience with event based, micro‑batch and batched high volume and high velocity transaction and data processing system.
• Proficiency with version control(Git), CI/CD and DevOps practices

Exposure to data governance, lineage, and metadata management

Education & Preferred Qualifications

• Bachelor’s/Master's Degree in a Computer Science, Engineering or Math or equivalent experience
• 5+ years' experience in professional software design and development.
• Strong analytical skills, with ability to collect, absorb, and analyze information quickly and independently
• Proven ability to develop and work with cross functional processes and project timelines
• Excellent follow‑through, attention to detail and time management skills
• Adapt and excel at managing multiple simultaneous tasks in a high pressure, deadline environment.

Salary Range

$90,000 - $157,500 Annual

The range quoted above applies to the role in the primary location specified. If the candidate would ultimately work outside of the primary location above, the applicable range could differ.

Employees are eligible to participate in State Street’s comprehensive benefits program, which includes: our retirement savings plan (401K) with company match; insurance coverage including basic life, medical, dental, vision, long‑term disability, and other optional additional coverages; paid‑time off including vacation, sick leave, short‑term disability, and family care responsibilities; access to our Employee Assistance Program; incentive compensation including eligibility for annual performance‑based awards (excluding certain sales roles subject to sales incentive plans); and, eligibility for certain tax advantaged savings plans.

For a full overview, visit https://hrportal.ehr.com/statestreet/Home.

About State Street

Across the globe, institutional investors rely on us to help them manage risk, respond to challenges, and drive performance and profitability. We keep our clients at the heart of everything we do, and smart, engaged employees are essential to our continued success.

We are committed to fostering an environment where every employee feels valued and empowered to reach their full potential. As an essential partner in our shared success, you’ll benefit from inclusive development opportunities, flexible work‑life support, paid volunteer days, and vibrant employee networks that keep you connected to what matters most. Join us in shaping the future.

As an Equal Opportunity Employer, we consider all qualified applicants for all positions without regard to race, creed, color, religion, national origin, ancestry, ethnicity, age, disability, genetic information, sex, sexual orientation, gender identity or expression, citizenship, marital status, domestic partnership or civil union status, familial status, military and veteran status, and other characteristics protected by applicable law.

Discover more information on jobs at StateStreet.com/careers

Read our CEO Statement

Job Application Disclosure

It is unlawful in Massachusetts to require or administer a lie detector test as a condition of employment or continued employment. An employer who violates this law shall be subject to criminal penalties and civil liability.

Job ID: R-794904