Remote Data Engineer & Insights Architect

S R International Inc; United States; Remote; USD 120,000 - 170,000; Full time

Job summary

Commonwealth of Pennsylvania is seeking an experienced Data Engineer & Insights Analyst to support the PALDS program. You will design and maintain data pipelines, integrate complex datasets, improve data quality, and develop analytics solutions to assist program delivery and executive decisions across state government agencies.

The role requires strong technical expertise in AWS, SQL, Python and big data technologies, data modeling, and data integration, with the ability to handle large,

Qualifications

• 5+ years of experience with AWS cloud services (S3, EC2, Glue, EMR, RDS, Redshift, Athena, Lambda, DynamoDB).
• 5+ years of programming with Python, R, Java, C++, or Scala.
• 5+ years with SQL and relational/NoSQL databases (Oracle, SQL Server, PostgreSQL, Cassandra).
• 5+ years with Git, version control, and CI/CD pipelines.
• Strong data modeling knowledge (dimensional, Data Vault, star/snowflake).
• Experience with AWS Glue Data Catalog and metadata management.
• Understanding of data governance, lineage, quality frameworks.
• Experience designing REST APIs.
• Hands-on with Hadoop, Spark, Kafka.
• Data pipeline orchestration tools like Airflow, Azkaban, Luigi.
• Data integration tools like Informatica, MuleSoft, Talend, or TIBCO.
• Stream processing with Storm or Spark Streaming.
• Advanced SQL with large datasets.

Responsibilities

• Design, build, and maintain scalable data pipeline architectures.
• Develop cloud data solutions using AWS S3, Glue, Redshift, EMR, Athena, Kinesis, and Lambda.
• Create and manage data workflows with Airflow, Step Functions or similar.
• ETL: extract, transform, load data from multiple sources with SQL and big data tech.
• Support analytics including predictive modeling and research.
• Collaborate with data scientists and policy analysts for reproducible research.
• Identify data quality issues and perform root-cause analysis.
• Automate processes to improve data delivery and scalability.
• Develop dashboards, reports, and decision-support tools.
• Implement monitoring, alerts, and logging for pipelines.
• Support data conversions related to modernization projects.
• Ensure governance and security in line with Commonwealth policies.

Skills

AWS cloud

Python

SQL

Big Data

Data modeling

ETL/Data pipelines

Airflow

REST APIs

CI/CD

Git

Spark

Hadoop

Kafka

SQL/NoSQL

DynamoDB

Data governance

Job description

Commonwealth of Pennsylvania is seeking an experienced Data Engineer & Insights Analyst to support the PALDS program. You will design and maintain data pipelines, integrate complex datasets, improve data quality, and develop analytics solutions to assist program delivery and executive decisions across state government agencies.

The role requires strong technical expertise in AWS, SQL, Python and big data technologies, data modeling, and data integration, with the ability to handle large,