Senior Data Engineer

Scubyt Inc; Sunnyvale (CA); On-site; USD 180,000 - 240,000; Full time

Job summary

Scubyt Inc in Sunnyvale, CA is seeking a highly experienced Senior Data Engineer to design, build, and scale data infrastructure powering our AI/ML platforms, analytics, and intelligent applications.

The role requires 10+ years of Data Engineering, 3+ years Snowflake, and strong expertise in Spark, Airflow, Python/Scala, SQL, AWS, and modern data architectures. You will collaborate with AI/ML engineers, data scientists, and product teams in a fast-paced startup.

Qualifications

• 10+ years of professional experience in Data Engineering.
• 3+ years of hands-on experience with Snowflake.
• Strong experience designing and developing enterprise-scale data pipelines and data platforms.
• Strong hands-on experience with Apache Spark.
• Strong hands-on experience with Apache Airflow or equivalent orchestration technologies.
• Advanced SQL skills and strong programming experience with Python and/or Scala.
• Strong understanding of ETL/ELT, data modeling, data integration, and data architecture.
• Experience working with both structured and semi-structured data.
• Strong experience with AWS cloud services, particularly Amazon S3.
• Experience designing data solutions for large-scale and high-volume workloads.
• Experience with Git, version control, and CI/CD pipelines.
• Strong understanding of data quality, monitoring, and pipeline reliability.

Responsibilities

• Design, develop, and maintain scalable data platforms supporting AI/ML and data-intensive applications.
• Build high-performance ETL/ELT pipelines for large volumes of structured and semi-structured data.
• Develop and optimize data processing solutions using Snowflake, Apache Spark, and Apache Airflow.
• Build reliable data ingestion and transformation pipelines from multiple sources.
• Design data models and architectures optimized for analytics, machine learning, and AI workloads.
• Develop cloud-based data solutions using AWS and Amazon S3.
• Optimize Snowflake databases, SQL queries, data models, and data pipelines for performance and scalability.
• Implement data orchestration, scheduling, monitoring, and workflow automation using Apache Airflow.
• Work closely with AI/ML engineers and data scientists to provide high-quality datasets for model development and production workloads.
• Support real-time and batch data processing requirements as needed.
• Implement data quality, validation, monitoring, governance, and observability practices.
• Build and maintain CI/CD pipelines for data engineering applications and infrastructure.
• Use Git and modern software engineering practices for development and deployment.
• Troubleshoot complex data pipeline and production issues.
• Contribute to the evolution of the company's data architecture as the AI platform scales.
• Collaborate with engineering, product, AI/ML, and business teams to translate requirements into scalable technical solutions.
• Work effectively in a fast-paced startup environment, taking ownership of projects from design through production.

Skills

10+ years of data engineering

Snowflake

Apache Spark

Apache Airflow

Python/Scala

SQL

AWS

Education

Bachelor's degree

Tools

Snowflake

Apache Spark

Apache Airflow

Python

Scala

SQL

AWS

Amazon S3

Job description

We are looking for a highly experienced Senior Data Engineer to join our growing AI technology team. In this role, you will design, build, and scale the data infrastructure that powers our AI/ML platforms, products, analytics, and intelligent applications.

The ideal candidate brings 10+ years of Data Engineering experience, including 3+ years of hands-on Snowflake experience, along with strong expertise in Apache Spark, Apache Airflow, Python/Scala, SQL, AWS, and modern data architecture.

You will work closely with AI/ML engineers, software engineers, data scientists, architects, and product teams to build reliable and scalable data systems in a fast-paced startup environment.

Key Responsibilities

• Design, develop, and maintain scalable data platforms supporting AI/ML and data-intensive applications.
• Build high-performance ETL/ELT pipelines for large volumes of structured and semi-structured data.
• Develop and optimize data processing solutions using Snowflake, Apache Spark, and Apache Airflow.
• Build reliable data ingestion and transformation pipelines from multiple sources.
• Design data models and architectures optimized for analytics, machine learning, and AI workloads.
• Develop cloud-based data solutions using AWS and Amazon S3.
• Optimize Snowflake databases, SQL queries, data models, and data pipelines for performance and scalability.
• Implement data orchestration, scheduling, monitoring, and workflow automation using Apache Airflow.
• Work closely with AI/ML engineers and data scientists to provide high-quality datasets for model development, training, evaluation, and production workloads.
• Support real-time and batch data processing requirements as needed.
• Implement data quality, validation, monitoring, governance, and observability practices.
• Build and maintain CI/CD pipelines for data engineering applications and infrastructure.
• Use Git and modern software engineering practices for development and deployment.
• Troubleshoot complex data pipeline and production issues.
• Contribute to the evolution of the company's data architecture as the AI platform scales.
• Collaborate with engineering, product, AI/ML, and business teams to translate requirements into scalable technical solutions.
• Work effectively in a fast-paced startup environment, taking ownership of projects from design through production.

Required Qualifications

• 10+ years of professional experience in Data Engineering.
• 3+ years of hands-on experience with Snowflake.
• Strong experience designing and developing enterprise-scale data pipelines and data platforms.
• Strong hands-on experience with Apache Spark.
• Strong hands-on experience with Apache Airflow or equivalent orchestration technologies.
• Advanced SQL skills and strong programming experience with Python and/or Scala.
• Strong understanding of ETL/ELT, data modeling, data integration, and data architecture.
• Experience working with both structured and semi-structured data.
• Strong experience with AWS cloud services, particularly Amazon S3.
• Experience designing data solutions for large-scale and high-volume workloads.
• Experience with Git, version control, and CI/CD pipelines.
• Strong understanding of data quality, monitoring, and pipeline reliability.
• Excellent problem-solving and analytical skills.
• Strong communication and collaboration skills.
• Ability to work independently and take ownership in a rapidly changing startup environment.
• Bachelor's degree in computer science, Engineering, Information Technology, or a related field.
• Must be authorized to work for any U.S. company without requiring visa sponsorship or transfer now or in the future.

Preferred Qualifications

• Experience working in an AI/ML, SaaS, cloud technology, or high-growth startup environment.
• Experience building data infrastructure supporting machine learning or generative AI applications.
• Experience with large-scale distributed data processing.
• Experience with Apache Kafka or other streaming technologies.
• Experience with modern data lake/data warehouse architectures.
• Experience with Snowflake performance optimization and cost management.
• Experience working closely with Data Scientists, ML Engineers, and AI Engineers.
• Experience building highly scalable data platforms from an early-stage environment.

What You'll Bring

• Strong ownership and a startup mindset.
• Ability to solve complex data problems with practical and scalable solutions.
• Passion for building infrastructure that enables AI and machine learning innovation.
• Ability to work across engineering, AI/ML, product, and business teams.
• Comfortable working in a rapidly evolving environment where priorities can change quickly.
• Strong focus on reliability, scalability, performance, and engineering quality.