Senior Data Engineer

S R INTERNATIONAL INC; United States; Remoto; USD 130.000 - 170.000; Tempo pieno

Descrizione del lavoro

Commonwealth of Pennsylvania is seeking a Senior Data Engineer & Analytics Consultant to support the PALDS initiative. The role focuses on transforming complex data into reliable, actionable insights for program delivery and executive decision-making across state government agencies.

You will design and develop data pipelines, integrate data from multiple sources, address data quality issues, and build analytics solutions while handling large, confidential datasets. This is a remote position.

Competenze

• 5+ years of cloud data services experience, especially AWS.
• 5+ years programming in Python/Java/Scala and SQL.
• Strong data modeling, governance, and API integration skills.
• Experience with CI/CD and data pipeline orchestration tools.

Mansioni

• Design, build, and maintain scalable data pipelines in the cloud.
• Develop AWS-based data solutions using S3, Glue, Redshift, EMR, Athena.
• Create and manage data orchestration workflows with Airflow/Azkaban.
• Integrate multi-source data for analytics, dashboards, and models.
• Collaborate with data scientists and policy analysts on analysis.
• Implement ETL, data quality checks, monitoring, and alerting.
• Support data modernization, governance, and security requirements.
• Deliver executive-ready analytics and decision-support outputs.

Conoscenze

AWS

Python

Java/Scala

SQL

Data Modeling

REST APIs

Airflow

Hadoop/Spark

CI/CD

Data Governance

Strumenti

Airflow

Azkaban

Luigi

Informatica

Descrizione del lavoro

Job Title: Senior Data Engineer

Job Code: PA 815653

Client: Commonwealth of Pennsylvania - EISO/PALDS

Location: Harrisburg, Pennsylvania (Remote)

Duration: 11/23/2026 - 12/31/2026

Work Hours: Monday-Friday, 8:00 AM-4:30 PM

Work Arrangement: Remote

Closing Date: 11/10/2026 at 4:00 PM

Interview Process: Two rounds - First round via Teams; second round in-person in Harrisburg for finalists.

Position Overview

The Commonwealth of Pennsylvania is seeking a Senior Data Engineer & Analytics Consultant to support the Pennsylvania Longitudinal Data System (PALDS).

The candidate will transform complex data into reliable, actionable insights to support program delivery, operational improvements, and executive decision-making across state government agencies.

The role involves designing and developing data pipelines, integrating and transforming data from multiple sources, addressing data quality issues, and building analytics solutions. The candidate will work with large, complex, and confidential datasets while supporting data modernization, data conversions, and enterprise data architecture initiatives.

Required Skills

• 5+ years of hands-on experience with cloud-based data services, particularly AWS (S3, EC2, Glue, EMR, RDS, Redshift, Athena, Lambda, DynamoDB).
• 5+ years of programming/scripting experience with Python, R, Java, C++, or Scala.
• 5+ years of experience with relational and NoSQL databases, including Oracle, MS SQL Server, PostgreSQL, and Cassandra.
• 5+ years of experience with Git, version control systems, and CI/CD pipelines.
• 5+ years of experience in data modeling, including dimensional modeling, Data Vault, and Star/Snowflake schemas.
• 5+ years of experience with data cataloging and metadata management tools such as AWS Glue Data Catalog.
• 5+ years of knowledge of data governance, data lineage, and data quality frameworks.
• 5+ years of experience with REST API design and integration patterns.
• 5+ years of experience with big data technologies such as Hadoop, Spark, and Kafka.
• 5+ years of experience with data pipeline and workflow management tools such as Azkaban, Luigi, and Airflow.
• 5+ years of experience with data integration solutions such as Informatica, MuleSoft, Talend, and TIBCO.
• 5+ years of experience with stream-processing technologies such as Storm and Spark Streaming.
• Advanced SQL skills and experience developing and optimizing complex queries.
• Experience building and optimizing large-scale data pipelines and analytical datasets.
• Experience developing dashboards, reports, analytical models, and executive decision-support products.
• Strong analytical, problem-solving, project management, and organizational skills.

Key Responsibilities

• Design, build, and maintain scalable data pipeline architectures.
• Develop cloud-based data solutions using AWS services such as S3, Glue, Redshift, EMR, Athena, Kinesis, and Lambda.
• Build and maintain data orchestration workflows using Apache Airflow, AWS Step Functions, or similar tools.
• Integrate, transform, and process large datasets from multiple sources.
• Develop analytical data environments supporting statistical analysis, predictive modeling, and machine learning.
• Partner with data scientists, research analysts, and policy analysts to support research and statistical modeling.
• Implement ETL processes, data quality checks, validation, monitoring, and alerting frameworks.
• Develop dashboards, reports, analytical models, and decision-support tools.
• Optimize data infrastructure, automate manual processes, and improve scalability.
• Support data modernization, system replacement, and data conversion initiatives.
• Ensure compliance with data governance, privacy, security, and Commonwealth data-sharing requirements.
• Collaborate with executive, data, design, and support teams to address data infrastructure needs.

Primary Skills

Python | Java | AWS | S3 | EC2 | Glue | EMR | RDS | Redshift | Athena | Lambda | DynamoDB | SQL | Oracle | MS SQL Server | PostgreSQL | Cassandra | R | C++ | Scala | Data Modeling | Informatica | MuleSoft | Talend | TIBCO | REST APIs | Git | CI/CD | Hadoop | Spark | Kafka | Airflow | Azkaban | Luigi | Storm | Spark Streaming

This is a remote position.