Data Engineer

Cloud Analytics Technologies, LLC; Dallas (TX); In loco; USD 110.000 - 150.000; Tempo pieno

Descrizione del lavoro

Cloud Analytics Technologies, LLC is seeking a Data Engineer to transform complex data into trusted, usable information for program delivery and operational improvements. The role involves designing and building data pipelines, integrating data from multiple sources, and addressing data quality issues in a secure, scalable AWS environment.

The successful candidate will work with large, confidential datasets, support data conversions during system modernization, and help optimize the

Competenze

• Experience building data pipelines and data integration solutions.
• Familiarity with cloud-based data platforms (AWS) and big data tools.
• Ability to transform complex data into usable, policy-relevant analytics.

Mansioni

• Create and maintain optimal data pipeline architecture.
• Design and implement cloud-based data solutions using AWS services (S3, Glue, Redshift, EMR, Athena, Kinesis, Lambda).
• Build and maintain data orchestration workflows using Airflow or similar tools.
• Assemble large, complex datasets meeting business requirements.
• Engineer data environments supporting statistical analysis and ML.
• Collaborate with data scientists and policy analysts to ensure reproducible research.
• Identify and implement internal process improvements for data delivery and scalability.
• Develop infrastructure for efficient ETL from diverse data sources using SQL and big data tech.

Conoscenze

AWS

SQL

Airflow

Big Data

Data Engineering

Strumenti

S3

Glue

Redshift

EMR

Athena

Kinesis

Lambda

Apache Airflow

Descrizione del lavoro

US Citizen, Green Card, H-1B, OPT-EAD, CPT, Other

Work Authorization: US Citizen, Green Card, OPT-EAD, CPT, H-4-EAD, Other

Job Description:

Data Engineer to transform complex data into trusted, usable information to support program delivery and operational improvements. The candidate will understand policy-relevant research questions, evaluate available data sources, and develop analytics products to deliver actionable insights.

Design and build data pipelines, integrate and transform data from multiple sources, identify and address data quality issues, and establish controls to support responsible use of data. This role requires both technical depth and strong analytical judgment, including the ability to work with large, complex, and confidential datasets.

The successful candidate must be self-directed and comfortable supporting the data needs of multiple teams, systems, and products. They must also be comfortable supporting data conversions associated with system modernization and/or replacement efforts. The right candidate will be excited by the prospect of optimizing the Commonwealth's longitudinal data architecture to support its next generation of data initiatives.

Roles and Responsibilities:

• Create and maintain optimal data pipeline architecture.
• Design and implement cloud-based data solutions using AWS services such as S3, Glue, Redshift, EMR, Athena, Kinesis, and Lambda.
• Build and maintain data orchestration workflows using tools such as Apache Airflow, AWS Step Functions, or similar services.
• Assemble large, complex datasets that meet functional and non-functional business area requirements.
• Engineer analytical data environments that support advanced statistical analysis, predictive modeling, machine learning, and evidence-building activities.
• Partner with data scientists, research analysts, and policy analysts to ensure enterprise data assets support reproducible research and statistical modeling.
• Identify, design, and implement internal process improvements, including automating manual processes, optimizing data delivery, and redesigning infrastructure for greater scalability.
• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and big data technologies.

We are an equal opportunity employer. All aspects of employment including the decision to hire, promote, discipline, or discharge, will be based on merit, competence, performance, and business needs. We do not discriminate on the basis of race, color, religion, marital status, age, national origin, ancestry, physical or mental disability, medical condition, pregnancy, genetic information, gender, sexual orientation, gender identity or expression, national origin, citizenship/ immigration status, veteran status, or any other status protected under federal, state, or local law.