Big Data Engineer - Remote

YO AI Labs; New York (NY); Remote; USD 90.000 - 179.000; Teilzeit

Zusammenfassung

YO AI Labs is seeking a Big Data Engineer contractor to design and maintain scalable data pipelines and architectures, enabling robust data solutions for AI training and analytics. You will work remotely with cross-functional teams, applying Python and big data technologies to ensure data quality, governance, and reliable performance across distributed databases and storage systems.

This role requires strong communication skills and the ability to work autonomously in a fast-paced, startup-like

Qualifikationen

• Proven expertise in big data engineering with hands-on experience building and maintaining large-scale data pipelines.
• Advanced proficiency in Python for data processing, automation, and integration.
• Deep understanding of relational and NoSQL databases, including optimization and management techniques.
• Experience with distributed data processing frameworks (e.g., Hadoop, Spark, Flink).
• Strong foundation in data modeling, ETL processes, and data warehousing principles.
• Excellent written and verbal communication skills, with the ability to convey technical ideas clearly to both technical and non-technical stakeholders.
• Detail-oriented, proactive, and self-motivated, thriving in remote and autonomous work environments.

Aufgaben

• Design, build, and maintain scalable big data pipelines and architectures to support robust data solutions.
• Collaborate with cross-functional teams to understand and deliver on data requirements and business objectives.
• Implement data integration, transformation, and processing solutions using Python and relevant big data technologies.
• Develop, manage, and optimize distributed databases and storage systems for efficiency and reliability.
• Monitor, troubleshoot, and enhance data systems to ensure high availability and performance.
• Enforce data quality, security, and governance standards across all solutions.
• Document solutions and communicate complex technical concepts effectively, both in writing and verbally.

Kenntnisse

Big data pipelines

Python

Databases

Distributed data processing

Data modeling

ETL

Communication

Tools

Hadoop

Spark

Flink

SQL/NoSQL

Cloud platforms (AWS/GCP/Azure)

Jobbeschreibung

Job Title: Big Data Engineer

Job Type: Contractor

Location: Remote

Job Summary: In this role, you'll apply your expertise to help train next-generation AI systems. Your work will shape how models learn, reason, and perform through high-quality, real-world input.

Key Responsibilities:

• Design, build, and maintain scalable big data pipelines and architectures to support robust data solutions.
• Collaborate with cross-functional teams to understand and deliver on data requirements and business objectives.
• Implement data integration, transformation, and processing solutions using Python and relevant big data technologies.
• Develop, manage, and optimize distributed databases and storage systems for efficiency and reliability.
• Monitor, troubleshoot, and enhance data systems to ensure high availability and performance.
• Enforce data quality, security, and governance standards across all solutions.
• Document solutions and communicate complex technical concepts effectively, both in writing and verbally.

Required Skills and Qualifications:

• Proven expertise in big data engineering with hands-on experience building and maintaining large-scale data pipelines.
• Advanced proficiency in Python for data processing, automation, and integration.
• Deep understanding of relational and NoSQL databases, including optimization and management techniques.
• Experience with distributed data processing frameworks (e.g., Hadoop, Spark, Flink).
• Strong foundation in data modeling, ETL processes, and data warehousing principles.
• Excellent written and verbal communication skills, with the ability to convey technical ideas clearly to both technical and non-technical stakeholders.
• Detail-oriented, proactive, and self-motivated, thriving in remote and autonomous work environments.

Preferred Qualifications:

• Prior experience in fast-paced or startup-like environments supporting global teams.
• Expertise with cloud-based big data platforms (e.g., AWS, GCP, Azure).
• Familiarity with machine learning operations and data science workflows is a plus.