Cloud Data Engineer AVP: AWS & Databricks Solutions

State Street; Boston (MA); On-site; USD 90,000 - 158,000; Full time

Job summary

State Street is seeking a Cloud Data Engineer, AVP, in Boston to design, develop, and deliver scalable cloud data solutions using Databricks and AWS. You will collaborate with technology and business partners to modernize data platforms, build reliable data and ML pipelines, and promote secure, cost-efficient engineering practices.

You will work on defining deployment best practices, building reusable data solutions, and migrating data applications to Databricks and AWS.

Qualifications

• Bachelor’s degree in computer science, information systems, or a related field, or equivalent practical experience.
• Five or more years of experience in data engineering, including hands-on work with cloud technologies.
• AWS and/or Databricks certification is preferred but not required.

Responsibilities

• Assess existing application and data infrastructure, identify improvement opportunities, and recommend cloud-based solutions that enhance performance, scalability, and reliability.
• Define and document best practices for application deployment, cloud infrastructure, and production support.
• Build reusable, efficient, and scalable data solutions, and develop cost-effective cloud migration strategies.
• Design, develop, and optimize data engineering and machine learning pipelines in Databricks using AWS services such as Amazon S3, EC2, API services, RDS, Kinesis, Kafka, and Lambda.
• Partner with IT teams and business stakeholders to migrate data engineering and machine learning applications to Databricks and AWS.
• Manage competing priorities and deliver high-quality solutions within time-sensitive project schedules when needed.
• Evaluate, learn, and support vendor products that contribute to the organization’s data and cloud capabilities.
• Quickly learn emerging technologies and develop proofs of concept to evaluate their value, feasibility, and potential for broader adoption.

Skills

Databricks architecture

Databricks Data Engineering

Machine Learning pipelines

Scala

Python

Java

SQL

AWS

GenAI knowledge

CI/CD

Airflow

Docker

Kubernetes

Data engineering

Education

Bachelor’s degree in computer science or related field

Five+ years of data engineering experience

AWS/Databricks certification preferred

Tools

Docker

Kubernetes

Apache Airflow

Delta Live Tables

Job description

State Street is seeking a Cloud Data Engineer, AVP, in Boston to design, develop, and deliver scalable cloud data solutions using Databricks and AWS. You will collaborate with technology and business partners to modernize data platforms, build reliable data and ML pipelines, and promote secure, cost-efficient engineering practices.

You will work on defining deployment best practices, building reusable data solutions, and migrating data applications to Databricks and AWS.