Senior Data Engineer – Cloud ETL & Data Pipelines

SMX; City of Albany (NY); On-site; USD 96,000 - 159,000; Full time

Job summary

SMX is seeking a Data Engineer to architect, build, test, and maintain data infrastructure for mission partners. You will design scalable ELT/ETL pipelines and deliver production-ready datasets by transforming raw information from multiple sources.

A strong SQL background and hands-on AWS experience, including Glue, Glue Catalog, and Step Functions, are essential. The role requires collaboration with analytics and business teams to improve data quality, reliability, and accessibility for

Qualifications

• Bachelor's degree in Computer Science, Information Technology, Data Engineering, or a related field.
• 3 years of professional experience in a data engineering or backend software engineering role.
• Active Top Secret security clearance.
• Strong proficiency in SQL and data query creation.
• Hands-on experience with AWS services, specifically AWS Glue, Glue Catalog, and Step Functions.
• Functions for data integration and workflow automation.
• Experience with data modeling, data migration, and ETL functions.
• Excellent communication skills, both written and verbal, with the ability to interface with customers.

Responsibilities

• Design, construct, test, and maintain highly scalable data management systems and robust ELT/ETL pipelines.
• Build the infrastructure required for optimal extraction, transformation, and loading of data from various sources using cloud technologies, with a strong focus on AWS Glue, Glue Catalog, and Step Functions.
• Support data migration efforts and ETL functions.
• Optimize database schemas and build systems to collect, validate, and prepare data.
• Combine raw information from different sources to meet business and mission needs.
• Work with analytics and business teams to understand data requirements and deliver production-ready datasets.

Skills

SQL

Data modeling

Data integration

Back-end data engineering

Communication

Education

Bachelor's degree in Computer Science or related field

Tools

AWS Glue

Glue Catalog

Step Functions

Databricks

Job description

SMX is seeking a Data Engineer to architect, build, test, and maintain data infrastructure for mission partners. You will design scalable ELT/ETL pipelines and deliver production-ready datasets by transforming raw information from multiple sources.

A strong SQL background and hands-on AWS experience, including Glue, Glue Catalog, and Step Functions, are essential. The role requires collaboration with analytics and business teams to improve data quality, reliability, and accessibility for