Senior Data Lake Engineer

PETADATA; Dallas (TX); Remote; USD 140,000 - 190,000; Full time

Benefits offered by this job

Professional work environment

Opportunities for growth

Job summary

A leading technology company is seeking a Senior Data Lake Engineer with over 15 years of experience in data engineering. The ideal candidate will have strong skills in AWS services such as Lake Formation, Glue, and Lambda, focusing on building scalable data lake solutions. Responsibilities include designing data lakes, developing data pipelines, and leading cross-functional projects. Candidates should be proficient in Python and experienced with AI-assisted development tools. This position offers remote work options and is critical to supporting advanced analytics and machine learning initiatives.

Qualifications

• 15+ years in data engineering roles, with 5+ years focused on AWS-native data lake development.
• Deep expertise in AWS Lake Formation, Glue, Lambda, and DynamoDB.
• Proficient in Python for serverless applications.
• Experience with AI-assisted development tools (Amazon Q, GitHub Copilot, AWS CodeWhisperer).
• Security practices in AWS including IAM, encryption, and compliance standards.

Responsibilities

• Design, build, and optimize scalable, secure data lakes using AWS Lake Formation.
• Build and deploy AWS Lambda functions using Python for data processing.
• Develop and maintain robust data pipelines using AWS Glue.
• AI Tools for Development: Leverage AI-powered coding tools (such as Amazon Q, GitHub Copilot, or similar) to increase development speed, code quality, and automation.
• Database Integration: Design and implement integrations between the Data Lake and DynamoDB, optimizing for performance, scale, and consistency.
• Security & Compliance: Implement fine-grained access control using Lake Formation, IAM policies, encryption, and data masking techniques to meet enterprise and compliance standards (e.g., GDPR, HIPAA).
• Monitoring & Optimization: Implement logging, monitoring, and performance tuning for Glue jobs, Lambda functions, and data workflows.
• Collaboration & Leadership: Collaborate with cross-functional teams including data science, analytics, DevOps, and product teams. Provide mentorship and technical leadership to junior engineers.

Skills

Data Engineering

AWS Lake Formation

Python

Serverless Development

AI Coding Tools

Data Security

Collaboration

Architecture leadership

Education

Bachelor's or Master's degree in Computer Science or Engineering

Master’s degree in Computer Science or related field

Tools

AWS Glue

DynamoDB

AWS Lambda

AI Tools

CloudFormation

Airflow

Job description

Overview

Experience: 15+ Years

PETADATA is seeking a seasoned Senior Data Lake Engineer with over 15 years of experience in data engineering and a strong focus on building and managing AWS-native Data Lake solutions. The ideal candidate will have deep expertise with AWS Lake Formation, serverless data processing using Lambda and Python, and experience with AI-assisted development tools such as Amazon Q.

Responsibilities

• Data Lake Architecture: Design, build, and optimize scalable, secure data lakes using AWS Lake Formation and best practices for data governance, cataloging, and access control.
• Serverless Development: Build and deploy AWS Lambda functions using Python for real-time data processing, automation, and event-driven workflows.
• ETL / ELT Pipelines: Develop and maintain robust data pipelines using AWS Glue, integrating data from various structured and unstructured sources.
• AI Tools for Development: Leverage AI-powered coding tools (such as Amazon Q, GitHub Copilot, or similar) to increase development speed, code quality, and automation.
• Database Integration: Design and implement integrations between the Data Lake and DynamoDB, optimizing for performance, scale, and consistency.
• Security & Compliance: Implement fine-grained access control using Lake Formation, IAM policies, encryption, and data masking techniques to meet enterprise and compliance standards (e.g., GDPR, HIPAA).
• Monitoring & Optimization: Implement logging, monitoring, and performance tuning for Glue jobs, Lambda functions, and data workflows.
• Collaboration & Leadership: Collaborate with cross-functional teams including data science, analytics, DevOps, and product teams. Provide mentorship and technical leadership to junior engineers.

Position Details

Position: Senior Data Lake Engineer

Location: Dallas, TX (Remote)

Work Type: C2C

Experience: 15+ Years

Note: This role requires strong hands-on skills in AWS Glue, DynamoDB, and building secure, scalable, and automated data platforms that support advanced analytics and machine learning use cases.

Roles & Responsibilities (duplicate sections consolidated)

• Data Lake Architecture: Design, build, and optimize scalable, secure data lakes using AWS Lake Formation and best practices for data governance, cataloging, and access control.
• Serverless Development: Build and deploy AWS Lambda functions using Python for real-time data processing, automation, and event-driven workflows.
• ETL / ELT Pipelines: Develop and maintain robust data pipelines using AWS Glue, integrating data from various structured and unstructured sources.
• AI Tools for Development: Leverage AI-powered coding tools (such as Amazon Q, GitHub Copilot, or similar) to increase development speed, code quality, and automation.
• Database Integration: Design and implement integrations between the Data Lake and DynamoDB, optimizing for performance, scale, and consistency.
• Security & Compliance: Implement fine-grained access control using Lake Formation, IAM policies, encryption, and data masking techniques to meet enterprise and compliance standards (e.g., GDPR, HIPAA).
• Monitoring & Optimization: Implement logging, monitoring, and performance tuning for Glue jobs, Lambda functions, and data workflows.
• Collaboration & Leadership: Collaborate with cross-functional teams including data science, analytics, DevOps, and product teams. Provide mentorship and technical leadership to junior engineers.

Required Skills

• Experience: 15+ years in data engineering roles, with 5+ years focused on AWS-native data lake development.
• Cloud Expertise: Deep, hands-on expertise in AWS Lake Formation, Glue, Lambda, and DynamoDB.
• Programming: Proficient in Python, especially for serverless and data processing applications.
• AI Coding Tools: Experience using AI-assisted development tools (e.g., Amazon Q, GitHub Copilot, AWS CodeWhisperer).
• Security: Strong knowledge of data security practices in AWS, including IAM, encryption, and compliance standards.
• Orchestration & Automation: Experience with workflow orchestration tools such as Step Functions, Airflow, or custom AWS-based solutions.
• Soft Skills: Strong communication, problem-solving, and collaboration skills. Able to lead discussions on architecture and best practices.

Preferred Skills

• AWS Certifications (e.g., AWS Certified Data Analytics, AWS Certified Solutions Architect)
• Experience with Athena, Redshift, or other analytics services in the AWS ecosystem
• Exposure to DevOps practices and tools like Terraform, CloudFormation, or CDK
• Familiarity with data cataloging and metadata management tools

Education

Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.

We offer a professional work environment and provide every opportunity for growth in the Information technology world.

Note

Candidates are required to attend Phone/video calls and in-person interviews. After the Selection, the candidate (He/She) should undergo all background checks on Education and Experience.

Application

Please email your resume to [e-mail]. After carefully reviewing your experience and skills, one of our HR team members will contact you on the next steps.