Data Engineer: Build Scalable Data Lakes & ML Pipelines

Wishalicious; United States; Remote; USD 120,000 - 160,000; Full time

Job summary

Wishalicious is seeking a Data Engineer to design, build, and maintain large-scale data systems including warehouses, lakes, and pipelines. The role collaborates with data scientists to deploy ML models and data products while ensuring data quality and security.

You will develop architectures that meet business needs, document architectures, and optimize performance across cloud platforms like AWS and Azure.

Qualifications

• Design, build, and maintain large-scale data systems including data warehouses, data lakes, and data pipelines.
• Develop and implement data architectures that meet business needs.
• Collaborate with data scientists to deploy machine learning models and data products.
• Ensure data quality, security, and compliance with standards.

Responsibilities

• Design, build, and maintain large-scale data systems, including data warehouses, data lakes, and data pipelines
• Develop and implement data architectures that meet the needs of our business stakeholders
• Collaborate with data scientists and analysts to develop and deploy machine learning models and data products
• Ensure data quality, security, and compliance with standards
• Develop and maintain technical documentation of data systems and architectures
• Troubleshoot and resolve data-related issues and optimize system performance
• Develop and implement automated testing and deployment scripts to ensure smooth and efficient delivery of data solutions
• Collaborate with cross-functional teams to identify and prioritize data requirements and develop solutions to meet business needs

Skills

Data engineering

Python

Terraform

data architecture

data modelling

data warehousing

data pipeline tools

data security

AI/ML technologies

AWS

Azure

Git

Tableau

Cognos Analytics

NIST 800-171/SOX

Education

Bachelor

Tools

Apache Spark

Apache Kafka

AWS Kinesis

Git

Tableau

Cognos Analytics

Virtualization/Cloud

Job description

Wishalicious is seeking a Data Engineer to design, build, and maintain large-scale data systems including warehouses, lakes, and pipelines. The role collaborates with data scientists to deploy ML models and data products while ensuring data quality and security.

You will develop architectures that meet business needs, document architectures, and optimize performance across cloud platforms like AWS and Azure.