A distinguished technology firm in Dallas, TX is seeking a skilled professional to design and implement AWS-based data Lakehouse architecture. The role involves building automated ETL/ELT pipelines, managing RESTful APIs for data access, and ensuring data quality and compliance. Candidates should be proficient in AWS services like S3 and Lambda. This position is a great opportunity for individuals looking to work in a cutting-edge data environment.
Qualifications
Experience designing AWS-based data lakehouse architectures.
Responsibilities
Design and implement AWS-based data Lakehouse architecture using Apache Iceberg.
Build and automate ETL/ELT pipelines for large-scale data ingestion.
Develop and manage secure RESTful APIs for data access.
Manage and optimize Iceberg tables (ACID, schema evolution, time travel, performance tuning).
Utilize AWS services such as S3, EMR, Lake Formation, Lambda, API Gateway, IAM.
Ensure data quality, security, and compliance.
Collaborate with cross-functional teams and provide technical support.
Skills
AWS
Data Lakehouse
ETL pipelines
RESTful APIs
Iceberg management
Data security
Compliance
Collaboration
Tools
Apache Iceberg
S3
EMR
Lake Formation
Lambda
API Gateway
IAM
Job description
Visa: U.S. Citizens or Green Card Holders only (due to government contract requirements)
Key Responsibilities
Design and implement AWS-based data Lakehouse architecture using Apache Iceberg
Build and automate ETL/ELT pipelines for large-scale data ingestion
Develop and manage secure RESTful APIs for data access
Manage and optimize Iceberg tables (ACID, schema evolution, time travel, performance tuning)
Utilize AWS services such as S3, EMR, Lake Formation, Lambda, API Gateway, IAM
Ensure data quality, security, and compliance
Collaborate with cross-functional teams and provide technical support