Sr Data Engineer (Data Quality & Data Governance), Malvern, PA, US

Intellibee; Malvern (AR); On-site; USD 140,000 - 180,000; Full time

Benefits offered by this job

Long-Term Stability

Comprehensive Health Coverage

Future Planning

GC Assistance

Job summary

IntelliBee is seeking a Senior Data Engineer focused on Data Quality and Data Governance in Malvern, PA. You will design, build, and scale a production-grade data platform with strong data contracts, automated testing, and continuous monitoring across enterprise systems.

You will implement data quality checks, governance tooling, and robust pipelines on AWS, collaborating with analytics, data science, and product teams to ensure reliable, scalable data products.

Qualifications

• 7+ years hands-on data engineering with focus on data quality and governance.
• Expert in Python, SQL and PySpark.
• Experience in AWS data architectures (S3, Glue, EMR, Redshift, Athena, Lambda).

Responsibilities

• Design, build, and scale data pipelines for quality and governance.
• Implement data contracts and automated data testing across data layers.
• Develop and monitor data quality checks, lineage, and metadata management.
• Collaborate with analytics, data science, and product teams to embed data quality.
• Lead productionization of pipelines with CI/CD and reliable ops practices.

Skills

Python

SQL

PySpark

Tools

Soda

Gable

Great Expectations

Monte Carlo

Airflow

Dagster

Job description

Sr Data Engineer (Data Quality & Data Governance), Malvern, PA, US

Senior Data Engineer (Data Quality and Data Governance)

About the Role

We are seeking a Senior Data Engineer specializing in Data Quality and Data Governance to design, build, and scale our next-generation data platform. In this role, you will lead the transition from ad-hoc data processing to a production-grade, highly reliable data platform.

You will work at the intersection of platform engineering and governance, establishing robust data contracts, automated data testing, and continuous monitoring across our enterprise systems.

Key Responsibilities

• Data Platform Development: Architect, implement, and maintain scalable, production-grade ETL/ELT data pipelines and platforms using Python, PySpark, and SQL on AWS.
• Data Quality & Checks: Design and deploy automated data quality testing, proactive monitoring, and anomaly detection across all data layers to ensure accuracy, completeness, and freshness.
• Data Contracts & Governance: Standardize and enforce data contracts between data producers and consumers to eliminate schema drift and ensure clear operational ownership.
• Governance Tooling Integration: Implement and manage modern data quality and governance frameworks using tools such as Soda, Gable, or similar modern data stack utilities (e.g., Great Expectations, Monte Carlo).
• Productionization & Optimization: Transition legacy or prototype data pipelines into reliable, maintainable, and cost-effective production systems adhering to CI/CD and software engineering best practices.
• Technical Leadership: Partner with cross-functional analytics, data science, and product teams to embed data quality standards into every stage of the software development lifecycle.

Required Qualifications

• Experience: 7+ years of hands-on data engineering experience, with a strong focus on data quality, data governance, and data pipeline reliability.
• Core Tech Stack: Expert-level proficiency in Python, SQL, and PySpark.
• Cloud Infrastructure: Extensive hands-on experience building production data architectures on AWS (e.g., S3, Glue, EMR, Redshift, Athena, Lambda).
• Data Quality Frameworks: Direct experience developing data checks and using data quality/contract tools like Soda, Gable, Great Expectations, or custom automated frameworks.
• Data Modeling & Architecture: Strong knowledge of data warehousing, lakehouse architectures, schema evolution, and data contract implementations.
• Engineering Standards: Experience with software engineering best practices, including Git, CI/CD pipelines, containerization (Docker), and orchestration tools (e.g., Apache Airflow, Prefect, or Dagster).

Preferred Qualifications

• Experience with automated data lineage and metadata management platforms.
• Familiarity with streaming architecture technologies (e.g., Apache Kafka, AWS Kinesis).
• Background in establishing SLAs, SLOs, and error budgets specifically tailored for data quality metrics.

Benefits at IntelliBee

• Long-Term Stability: Join us on a multi-year opportunities with room to grow.
• Comprehensive Health Coverage: Access quality healthcare benefits to keep you and your family well.
• Future Planning: Enroll in our 401(k) program and invest in your financial security.
• GC Assistance: We support immediate Green Card processing, if required.