Senior Data Engineer, Big Data & Analytics (AWS)

Machine Learning Technologies, LLC; Raleigh (NC); Presencial; USD 120.000 - 170.000; Tempo integral

Resumo da vaga

Machine Learning Technologies, LLC is seeking a Data Engineer with extensive experience in building and optimizing large-scale data pipelines and analytics solutions. The role focuses on designing cloud-based data infrastructure and delivering dashboards for executive and non-technical users.

You will collaborate with data scientists and policy analysts, implement data quality controls, and ensure compliance with governance and security standards while driving scalability across education and

Qualificações

• 12+ years’ experience as a Data Engineer or similar role.
• Advanced SQL knowledge and proficiency with relational databases.
• Experience building and optimizing big data pipelines and datasets.
• Designing dashboards, reports, and analytical models for executive/ non-technical users.
• Experience with student-level higher education data and linkage to other sources.

Responsabilidades

• Create and maintain optimal data pipeline architecture.
• Design and implement cloud-based data solutions using AWS services.
• Build and maintain data orchestrations with Airflow or Step Functions.
• Assemble large, complex data sets meeting business requirements.
• Engineer environments for statistical analysis, predictive modeling, and ML.
• Partner with data scientists and policy analysts to ensure reproducible research.
• Identify internal process improvements for scalability and automation.
• Develop dashboards, reports, and decision-support products.
• Ensure data quality checks and validation across pipelines.
• Develop data tools for analytics and data science teams.

Conhecimentos

Advanced SQL

Relational databases

SQL querying

Big data pipelines

Dashboards & reports

Analytics for exec users

Unstructured data analysis

Data transformation

Data governance & metadata

Cross-functional collaboration

Ferramentas

AWS

Apache Airflow

Redshift

EMR

S3

Glue

Kinesis

Lambda

Athena

Descrição da vaga

Machine Learning Technologies, LLC is seeking a Data Engineer with extensive experience in building and optimizing large-scale data pipelines and analytics solutions. The role focuses on designing cloud-based data infrastructure and delivering dashboards for executive and non-technical users.

You will collaborate with data scientists and policy analysts, implement data quality controls, and ensure compliance with governance and security standards while driving scalability across education and