Senior Data Architect - AWS Lakehouse & Real-Time Pipelines

Coforge; Fort Mill (SC); On-site; USD 120,000 - 160,000; Full time

Job summary

Coforge seeks a Data Architect with AWS Data platform experience in Fort Mill, SC. You will design scalable enterprise data platforms, build data lake/lakehouse solutions, and develop batch and streaming data pipelines using AWS services and Spark-based technologies.

This role requires leadership in architecture reviews and collaboration with governance, security, and analytics teams. You will implement data quality controls, metadata management, and governance while mentoring data engineers to

Qualifications

• Design scalable enterprise data platforms on AWS.
• Architect data lakehouse solutions with S3, Glue, and Lake Formation.
• Develop batch, streaming, and CDC data pipelines.

Responsibilities

• Design, develop, and support scalable enterprise data platforms using AWS cloud-native services and modern data engineering technologies.
• Architect Data Lake and Lakehouse solutions using Amazon S3, AWS Glue, Lake Formation, Athena, Redshift, Apache Iceberg, Delta Lake, and Parquet.
• Develop scalable batch, incremental, and Change Data Capture pipelines for ingesting and processing data from databases, APIs, files, enterprise applications, and streaming platforms.
• Build and optimize ETL and ELT workflows using AWS Glue, Apache Spark, PySpark, Python, SQL, Lambda, and distributed data-processing frameworks.
• Design reusable data ingestion, transformation, validation, reconciliation, exception-handling, and error-recovery frameworks.
• Implement real-time and near-real-time data ingestion and processing solutions using Kafka, Amazon MSK, Amazon Kinesis, Lambda, SNS, SQS, and EventBridge.
• Develop event-driven solutions supporting integration between business applications, operational platforms, analytical systems, and downstream data consumers.
• Implement workflow orchestration using Apache Airflow, Amazon MWAA, AWS Step Functions, and event-based scheduling mechanisms.
• Design and maintain metadata management, schema management, data cataloging, data lineage, and data discovery capabilities using AWS Glue Data Catalog and Lake Formation.
• Implement data partitioning, compaction, retention, lifecycle management, and storage optimization strategies to improve performance and cost efficiency.
• Establish data quality controls for completeness, accuracy, consistency, integrity, uniqueness, and source-to-target reconciliation.
• Implement secure and governed data-access models using IAM, KMS, S3 policies, Lake Formation permissions, encryption, and fine-grained access controls.
• Collaborate with Business, Data Governance, Security, Analytics, Infrastructure, and Architecture teams to deliver trusted and reusable enterprise data products.
• Tune data pipelines, Spark workloads, data-storage layouts, and analytical queries for scalability, reliability, and performance.
• Lead architecture reviews, technical design discussions, coding standards, platform modernization, and engineering best-practice initiatives.
• Provide technical leadership and mentoring to data engineers while ensuring alignment with enterprise architecture, security, governance, and delivery standards.

Skills

AWS Data Platform

Data Architecture

Data Lake / Lakehouse

ETL/ELT Design

Spark / PySpark

SQL

Kafka / Streaming

Tools

AWS Glue

S3

Lake Formation

Athena

Redshift

Apache Iceberg

Delta Lake

Parquet

Apache Spark

PySpark

Kafka

Kinesis

Airflow

MWAA

Lambda

Job description

Coforge seeks a Data Architect with AWS Data platform experience in Fort Mill, SC. You will design scalable enterprise data platforms, build data lake/lakehouse solutions, and develop batch and streaming data pipelines using AWS services and Spark-based technologies.

This role requires leadership in architecture reviews and collaboration with governance, security, and analytics teams. You will implement data quality controls, metadata management, and governance while mentoring data engineers to