Data Engineer

S R International Inc; United States; Remote; USD 120,000 - 170,000; Full time

Job summary

Commonwealth of Pennsylvania is seeking an experienced Data Engineer & Insights Analyst to support the PALDS program. You will design and maintain data pipelines, integrate complex datasets, improve data quality, and develop analytics solutions to assist program delivery and executive decisions across state government agencies.

The role requires strong technical expertise in AWS, SQL, Python and big data technologies, data modeling, and data integration, with the ability to handle large,

Qualifications

• 5+ years of experience with AWS cloud services (S3, EC2, Glue, EMR, RDS, Redshift, Athena, Lambda, DynamoDB).
• 5+ years of programming with Python, R, Java, C++, or Scala.
• 5+ years with SQL and relational/NoSQL databases (Oracle, SQL Server, PostgreSQL, Cassandra).
• 5+ years with Git, version control, and CI/CD pipelines.
• Strong data modeling knowledge (dimensional, Data Vault, star/snowflake).
• Experience with AWS Glue Data Catalog and metadata management.
• Understanding of data governance, lineage, quality frameworks.
• Experience designing REST APIs.
• Hands-on with Hadoop, Spark, Kafka.
• Data pipeline orchestration tools like Airflow, Azkaban, Luigi.
• Data integration tools like Informatica, MuleSoft, Talend, or TIBCO.
• Stream processing with Storm or Spark Streaming.
• Advanced SQL with large datasets.

Responsibilities

• Design, build, and maintain scalable data pipeline architectures.
• Develop cloud data solutions using AWS S3, Glue, Redshift, EMR, Athena, Kinesis, and Lambda.
• Create and manage data workflows with Airflow, Step Functions or similar.
• ETL: extract, transform, load data from multiple sources with SQL and big data tech.
• Support analytics including predictive modeling and research.
• Collaborate with data scientists and policy analysts for reproducible research.
• Identify data quality issues and perform root-cause analysis.
• Automate processes to improve data delivery and scalability.
• Develop dashboards, reports, and decision-support tools.
• Implement monitoring, alerts, and logging for pipelines.
• Support data conversions related to modernization projects.
• Ensure governance and security in line with Commonwealth policies.

Skills

AWS cloud

Python

SQL

Big Data

Data modeling

ETL/Data pipelines

Airflow

REST APIs

CI/CD

Git

Spark

Hadoop

Kafka

SQL/NoSQL

DynamoDB

Data governance

Job description

Job Title: Data Engineer & Insights Analyst

Job Code: PA 815653

Client: Commonwealth of Pennsylvania – EISO / PALDS

Location: Remote

Duration: 11/23/2026 – 12/31/2026

Work Hours: Monday–Friday, 8:00 AM–4:30 PM

Work Arrangement: Remote

Closing Date: 11/10/2026 at 4:00 PM

Interview Process: Two rounds – First round via MS Teams; second round in person in Harrisburg, PA for finalists.

Position Overview

The Commonwealth of Pennsylvania is seeking an experienced Data Engineer & Insights Analyst to support the Pennsylvania Longitudinal Data System (PALDS).

The selected candidate will design and maintain data pipelines, integrate and transform complex datasets, improve data quality, and develop analytics solutions to support program delivery, operational improvements, and executive decision-making across state government agencies.

This role requires strong technical expertise in AWS, SQL, Python, big data technologies, data modeling, and data integration, along with the ability to work with large, complex, and confidential datasets. The candidate will also support data conversions related to system modernization and replacement initiatives.

Required Skills

• 5+ years of hands-on experience with AWS cloud services, including S3, EC2, Glue, EMR, RDS, Redshift, Athena, Lambda, and DynamoDB.
• 5+ years of programming/scripting experience with Python, R, Java, C++, or Scala.
• 5+ years of experience with SQL and relational/NoSQL databases, including Oracle, SQL Server, PostgreSQL, and Cassandra.
• 5+ years of experience with Git, version control, and CI/CD pipelines.
• Strong knowledge of data modeling, including dimensional modeling, Data Vault, and star/snowflake schemas.
• Experience with AWS Glue Data Catalog, metadata management, and data cataloging.
• Understanding of data governance, data lineage, and data quality frameworks.
• Experience designing and integrating REST APIs.
• Hands-on experience with big data technologies such as Hadoop, Spark, and Kafka.
• Experience with data pipeline orchestration tools such as Apache Airflow, Azkaban, or Luigi.
• Experience with data integration platforms such as Informatica, MuleSoft, Talend, or TIBCO.
• Experience with stream-processing technologies such as Storm or Spark Streaming.
• Advanced SQL knowledge and experience working with large, complex datasets.
• Strong analytical, problem-solving, project management, and organizational skills.
• Experience supporting cross-functional teams in a dynamic environment.

Key Responsibilities

• Design, build, and maintain scalable data pipeline architectures.
• Develop cloud-based data solutions using AWS services such as S3, Glue, Redshift, EMR, Athena, Kinesis, and Lambda.
• Build and maintain data orchestration workflows using Apache Airflow, AWS Step Functions, or similar tools.
• Extract, transform, integrate, and load data from multiple sources using SQL and big data technologies.
• Develop and optimize data environments supporting statistical analysis, predictive modeling, machine learning, and research.
• Collaborate with data scientists, research analysts, and policy analysts to support reproducible research and data-driven decision-making.
• Identify and resolve data quality issues through validation, monitoring, and root-cause analysis.
• Automate manual processes and improve data delivery, infrastructure scalability, and operational efficiency.
• Develop dashboards, reports, analytical models, and decision-support tools.
• Build monitoring, alerting, and logging frameworks for data pipelines.
• Support data conversions associated with system modernization and replacement projects.
• Ensure compliance with data governance, privacy, security, and Commonwealth policies.
• Collaborate with executive, data, design, and support teams to address technical issues and improve data systems.

Preferred Skills

The job description does not specify a separate preferred-skills list. The technologies and experience listed under Required Qualifications and Required/Desired Skills should be treated as the stated requirements.

Primary Skills

AWS | Python | Java | R | SQL | Oracle | SQL Server | PostgreSQL | Cassandra | Data Modeling | AWS Glue | S3 | EC2 | EMR | RDS | Redshift | Athena | Lambda | DynamoDB | Git | CI/CD | REST APIs | Hadoop | Spark | Kafka | Airflow | Azkaban | Luigi | Informatica | MuleSoft | Talend | TIBCO | Storm | Spark Streaming

This is a remote position.