Senior Data Engineer - Scalable Pipelines & Cloud Analytics

Initvalue; Fremont (CA); Hybrid; USD 130,000 - 190,000; Full time

Job summary

Initvalue is seeking a Data Engineer to design, develop, and maintain scalable data pipelines and platforms that empower analytics, BI, and ML workloads. You will collaborate with data scientists, analysts, and engineers to ensure data reliability, accessibility, and security, while optimizing data models, warehouses, and lakes across cloud-based environments.

The role requires hands-on experience with Spark, Airflow, and cloud services (AWS/Azure/GCP), strong SQL, Python, and a bias for quality

Qualifications

• 5+ years in Data Engineering or related role.
• Strong Python and SQL skills.
• Hands-on experience developing ETL/ELT pipelines.
• Strong understanding of relational and non-relational databases.
• Experience with data warehousing and data lake architectures.
• Experience with distributed data processing technologies such as Apache Spark.
• Experience with at least one major cloud platform (AWS, Azure, or GCP).
• Experience with workflow orchestration tools such as Apache Airflow or similar.
• Strong data modeling, data quality, and data integration knowledge.
• Experience with Git, CI/CD, and software engineering best practices.

Responsibilities

• Design, develop, and maintain scalable data pipelines and ETL/ELT workflows.
• Build and optimize data ingestion, transformation, and processing systems.
• Develop efficient data models for analytics and reporting.
• Work with structured and unstructured data from multiple sources.
• Design and maintain data warehouses, data lakes, and modern data platforms.
• Optimize SQL queries and data-processing jobs for performance and scalability.
• Implement data quality, validation, monitoring, and governance processes.
• Integrate data from APIs, databases, applications, and third-party systems.
• Work with cloud-based data services and distributed processing technologies.
• Collaborate with data scientists and ML engineers to support AI/ML data requirements.
• Ensure data pipelines are scalable, secure, reliable, and maintainable.
• Troubleshoot data-related issues and improve pipeline reliability.
• Contribute to data architecture and technical design decisions.
• Mentor junior data engineers and contribute to engineering best practices.

Skills

Python

SQL

ETL/ELT

Data warehousing

Data lakes

Apache Spark

Cloud platforms

Airflow

Data modeling

Git & CI/CD

Tools

Apache Airflow

Git

CI/CD

Apache Spark

Job description

Initvalue is seeking a Data Engineer to design, develop, and maintain scalable data pipelines and platforms that empower analytics, BI, and ML workloads. You will collaborate with data scientists, analysts, and engineers to ensure data reliability, accessibility, and security, while optimizing data models, warehouses, and lakes across cloud-based environments.

The role requires hands-on experience with Spark, Airflow, and cloud services (AWS/Azure/GCP), strong SQL, Python, and a bias for quality