AWS Data Engineer

Incedo Inc.; Austin (TX); Hybrid; USD 120,000 - 160,000; Full time

Job summary

Incedo Inc. in Austin, TX is seeking an experienced Data Engineer to design and build production-grade data pipelines across ingestion, transformation, storage, and consumption layers in a hybrid work environment.

You will own end-to-end data architectures, implement robust data quality checks, and optimize PySpark/SQL workloads on AWS-native stacks (S3, Glue, Athena) using Airflow or Step Functions, ensuring scalable, reliable data flows for business insights.

Qualifications

• 8+ years of hands-on data engineering experience building production data pipelines.
• End-to-end design and ownership of data pipelines across ingestion, transformation, storage and consumption.
• Experience with AWS-native data stack including S3, Glue (PySpark), Athena, and orchestration tools such as Airflow or Step Functions.
• Experience building automated data quality checks into pipelines: validation rules, reconciliation logic, anomaly detection.
• Strong understanding of data modeling and storage design including partitioning strategies and handling of small files.
• Hands-on data modeling expertise (dimensional modeling, normalization, slowly changing dimensions) for new datasets.
• Ownership of data quality frameworks with validation rules, thresholds and exception handling.
• Experience defining testing standards for data pipelines and driving adoption.
• Ability to make architectural trade-offs between Glue, Lambda, EMR or container-based approaches.
• Solid PySpark and SQL skills with performance tuning in distributed environments.
• Experience building reliable pipelines with error handling, retries, and monitoring.
• Experience reconciling data across sources and targets to secure production sign-off.

Responsibilities

• Design and own end-to-end data pipelines across ingestion, transformation, storage and consumption.
• Implement data quality checks and monitoring for pipelines.
• Optimize PySpark/SQL workloads on AWS-native stacks for performance and reliability.
• Make architectural decisions and trade-offs based on use case and scale.
• Reconcile data across source and target systems and secure production sign-off.

Skills

Data engineering

End-to-end pipelines

Data quality

Data modeling

Performance tuning

Production-grade pipelines

Monitoring & troubleshooting

Architectural decision-making

Data reconciliation

Data pipeline ownership

Tools

S3

Glue (PySpark)

Athena

Airflow

Step Functions

PySpark

SQL

EMR

Lambda

Job description

Incedo Inc. is a high-growth Digital, Data and AI Transformation Specialist firm headquartered in New Jersey. We

are a long-term strategy execution partner for Fortune 500 enterprises, operating at the intersection of business

and technology across Banking & Payments, Wealth Management, Telecom, Hi-Tech, and Life Sciences.

We are building Incedo 4.0 - an AI-native, execution-focused, founder-led organization designed for scale, speed,

and long-term impact.

Incedo delivers ROI from AI @ Scale through the “Power of 3”:

Engineering & Operations excellence.

Location : Austin TX (Hybrid)

• 8+ years of hands-on data engineering experience building production data pipelines, not just supporting or maintaining them.
• Proven experience designing and owning end-to-end data pipelines across ingestion, transformation, storage, and consumption layers.
• Strong experience with AWS-native data stack including S3, Glue (PySpark), Athena, and orchestration tools such as Airflow or Step Functions.
• Experience building automated data quality checks into pipelines, such as validation rules, reconciliation logic, and anomaly detection.
• Strong understanding of data modeling and storage design including partitioning strategies, file sizing, and handling of small files.
• Deep, hands-on data modeling expertise, with solid grounding in data modeling concepts (dimensional modeling, normalization, slowly changing dimensions, fact/dimension design) and the ability to independently apply them to design data models and schemas for new datasets, including resolving grain or structural differences between source and target systems.
• Direct, end-to-end ownership experience with data quality frameworks, defining validation rules, thresholds, and exception handling, not just operating within a framework someone else built.
• Demonstrated experience defining testing standards for data pipelines and leading their adoption across a team.
• Experience making architectural decisions and trade-offs, such as choosing between Glue, Lambda, EMR or container-based approaches based on use case.
• Solid experience with PySpark and SQL including performance tuning and optimization in distributed processing environments.
• Experience building reliable, production-grade pipelines including error handling, retries, and monitoring.
• Experience reconciling and validating data across source and target systems to identify discrepancies and drive sign-off before production release.