Data Engineer, Specialist

Vanguard; Dallas (TX); Hybrid; USD 110,000 - 150,000; Full time

Benefits offered by this job

Hybrid work model

Job summary

Vanguard is seeking a Data Engineer, Specialist to design, develop, and maintain scalable data pipelines and modern data infrastructure for analytics initiatives. You will build robust ETL processes, manage databases, and optimize cloud data platforms to seamlessly integrate data from multiple sources.

You will collaborate with data scientists and analysts to understand data needs, develop data solutions, and enable self-service analytics across the organization.

Qualifications

• Three+ years of data engineering, programming, database administration, or data management experience.
• Undergraduate degree or equivalent combination of training and experience.
• Strong proficiency in SQL and Python for data manipulation, automation and pipeline development.

Responsibilities

• Develop and maintain scalable ETL processes to extract, transform, and load data from diverse sources.
• Design and optimize database architectures and data pipelines for high performance, availability, and security.
• Integrate data from multiple sources including APIs and cloud/on-prem databases to create unified datasets for analytics.
• Collaborate with data scientists, analysts and stakeholders to understand data needs and enable self-service analytics.
• Develop automated workflows and data processing scripts using Python, Spark, SQL, or other technologies.
• Optimize data storage and retrieval in cloud data warehouses ensuring scalability and cost-efficiency.
• Maintain and improve data quality with validation, anomaly detection, and data cleansing.
• Test code thoroughly and review with end users to ensure clarity and accuracy.
• Provide tool and data support to business users and team members.
• Conduct regression testing for new releases and coordinate with vendors for deployment.

Skills

SQL

Python

Apache Spark

Hadoop

Kafka

AWS

Education

Bachelor's degree

Tools

AWS Glue

S3

Lambda

Databricks

Synapse

Data Factory

Job description

Role Overview

The Data Engineer, Specialist is responsible for designing, developing, and maintaining scalable data pipelines and infrastructure to support analytics and business intelligence initiatives. This role involves building robust ETL (Extract, Transform, Load) processes, managing databases and optimizing cloud-based data platforms to ensure efficient and seamless integration of data from multiple sources.

Responsibilities

• Develop and maintain scalable ETL (Extract, Transform, Load) processes to efficiently extract data from diverse sources, transform it as required and load it into data warehouses or analytical systems.
• Design and optimize database architectures and data pipelines to ensure high performance, availability and security while supporting structured and unstructured data.
• Integrate data from multiple sources, including APIs, third-party services and on-prem/cloud databases to create unified and consistent datasets for analytics and reporting.
• Collaborate with data scientists, analysts and business stakeholders to understand their data needs, develop data solutions and enable self-service analytics.
• Develop automated workflows and data processing scripts using Python, Spark, SQL, or other relevant technologies to streamline data ingestion and transformation.
• Optimize data storage and retrieval strategies in cloud-based data warehouses such as AWS Redshift, Google Big Query, or Azure Synapse, ensuring scalability and cost-efficiency.
• Maintain and improve data quality by implementing validation frameworks, anomaly detection mechanisms and data cleansing processes.
• Thoroughly tests code to ensure accuracy and alignment with its intended purpose. Reviews the final product with end users to confirm clarity and understanding, providing data analysis guidance as needed.
• Offers tool and data support to business users and team members, ensuring seamless functionality and accessibility.
• Conducts regression testing for new software releases, identifying issues and collaborating with vendors to resolve them and successfully deploy the software into production.

Qualifications And Skills

• Minimum of three years data engineering, programming, database administration, or data management experience.
• Undergraduate degree or equivalent combination of training and experience.
• Strong proficiency in SQL, Python for data manipulation, automation and pipeline development.
• Experience working with big data processing frameworks such as Apache Spark, Hadoop, or Kafka.
• Hands-on experience with cloud-based data platforms such as AWS (Glue, S3, Lambda), Data Bricks(Lakehouse, Worklows), or Azure (Synapse, Data Factory).

Sponsorship

Vanguard is not offering visa sponsorship for this position.

About Vanguard

At Vanguard, we don't just have a mission—we're on a mission.

To work for the long-term financial wellbeing of our clients. To lead through product and services that transform our clients' lives. To learn and develop our skills as individuals and as a team. From Malvern to Melbourne, our mission drives us forward and inspires us to be our best.

How We Work

Vanguard has implemented a hybrid working model for the majority of our crew members, designed to capture the benefits of enhanced flexibility while enabling in-person learning, collaboration, and connection. We believe our mission-driven and highly collaborative culture is a critical enabler to support long-term client outcomes and enrich the employee experience.