Senior Data Engineer

Norwest Venture Partners; Enterprise (OR); Remote; USD 100,000 - 140,000; Full time

Job summary

A market research firm is seeking a Senior Data Engineer to lead the design and optimization of data pipelines. The ideal candidate will have 6+ years of experience, strong skills in Spark, SQL, and Databricks, and excellent communication abilities. This position offers the opportunity to work in a remote setting and significantly impact data initiatives within the organization.

Qualifications

• 6+ years of experience as a Data Engineer or in similar technical roles.
• Strong experience with data pipeline construction and optimization.
• Excellent verbal and written communication skills.

Responsibilities

• Own end-to-end data pipelines and the data platform environment.
• Collaborate with stakeholders to design robust data solutions.
• Maintain high standards for data modeling, governance, and CI/CD practices.

Skills

Spark

SQL

PySpark

Delta

Databricks

Education

Bachelor's or Master's degree in Computer Science, STEM, or related technical field

Job description

Overview

YipitData is the leading market research and analytics firm for the disruptive economy. This is a remote opportunity based in India with standard IST working hours and occasional late-day overlap with US and LatAm teams.

About YipitData

YipitData is the leading market research firm for the disruptive economy.

Role

We are seeking a highly skilled Senior Data Engineer to join our dynamic Data Engineering team. The ideal candidate possesses 6-8 years of data engineering experience and has a solid understanding of Spark and SQL, with data pipeline experience. Hired individuals will help build out our data engineering team to support strategic pipelines and optimize for reliability, efficiency, and performance. The Data Engineering team builds and maintains core pipelines and tools that power our products, collaborating with analysts, data engineers, and stakeholders to design, debug, and optimize high-performance data pipelines in Databricks, while mentoring teammates and expanding the team\u2019s capabilities.

• Report directly to the Senior Manager of Data Engineering, with hands-on training on cutting-edge data tools and techniques.
• Own the design, build, and continuous improvement of end-to-end data pipelines and the surrounding platform.
• Define and evangelize technical standards (coding, data-modeling, monitoring) for the team.
• Mentor and pair-program with other engineers; provide architectural and code reviews.
• Lead deep-dive performance-tuning initiatives (Spark UI analysis, cluster configuration, shuffle reduction).
• Evaluate and introduce new Databricks / AWS capabilities that improve reliability or efficiency.
• Translate ambiguous business questions into scalable, production-ready data solutions with clear SLAs.
• Produce high-quality documentation, architecture diagrams, and internal training sessions.

Responsibilities

• Own end-to-end data pipelines and the data platform environment.
• Collaborate with analysts, data engineers, and stakeholders to design robust data solutions.
• Maintain high standards for data modeling, governance, and CI/CD practices.
• Develop and optimize pipelines in Databricks, PySpark, Delta, and related technologies.

Qualifications

• Bachelor’s or Master’s degree in Computer Science, STEM, or related technical field.
• 6+ years of experience as a Data Engineer or in similar technical roles.
• Strong experience with PySpark, Delta, Databricks, Spark, and SQL.
• Ability to balance long-term architecture thinking with hands-on execution under tight timelines.
• Excellent verbal and written communication skills and ability to collaborate with senior stakeholders.
• Ability to set best practices for data modelling, governance, and CI/CD.

We are committed to equal employment opportunity regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, marital status, disability, gender, gender identity or expression, or veteran status. We are proud to be an equal-opportunity employer.