Data Pipeline Engineer

TurnKey Tech Staffing; Teletrabalho; BRL 120.000 - 160.000; Tempo integral

Resumo da oferta

A dynamic tech staffing agency is seeking a skilled Data Engineer to support their ETL Engineering team. The role demands 4+ years of software or data engineering experience and expert knowledge in Spark and SQL. Responsibilities include building data pipelines and collaborating with stakeholders. Candidates should be available for US East Coast working hours and have a strong understanding of data challenges and technical solutions. This is a significant opportunity in a fast-paced environment.

Qualificações

• 4+ years of experience as a Data Engineer, Data Analyst, or in another technical function.
• Strong understanding of large-scale datasets and pipeline building.
• Excellent verbal and written communication skills.

Responsabilidades

• Build end-to-end data pipelines.
• Create documentation and architecture diagrams.
• Collaborate with stakeholders to incorporate business logic.

Conhecimentos

Pyspark

SQL

Data pipeline development

Data modeling

Airflow

Formação académica

Bachelor’s or Master’s degree in Computer Science or related field

Ferramentas

Databricks

Airflow

Descrição da oferta de emprego

About YipitData

YipitData is the leading market research and analytics firm for the disruptive economy and recently raised up to $475M from The Carlyle Group at a valuation over $1B.

We analyze billions of alternative data points every day to provide accurate, detailed insights on ridesharing, e-commerce marketplaces, payments, and more. Our on-demand insights team uses proprietary technology to identify, license, clean, and analyze the data many of the world’s largest investment funds and corporations depend on.

For three years and counting, we have been recognized as one of Inc’s Best Workplaces. We are a fast-growing technology company backed by The Carlyle Group and Norwest Venture Partners. Our offices are located in NYC, Austin, Miami, Denver, Mountain View, Seattle, Hong Kong, Shanghai, Beijing, Guangzhou, and Singapore. We cultivate a people-centric culture focused on mastery, ownership, and transparency.

Why You Should Apply NOW

• You’ll be working with many strategic leaders within the company.
• The ETL Engineering team is growing rapidly.
• You’ll be challenged with a lot of big data problems.

About The Role

We are seeking a highly skilled Data Engineer to join our dynamic ETL Engineering team. The ideal candidate possesses 4+ years of software engineering experience and/or data engineering experience. An excellent candidate should have a solid understanding of Spark, SQL, and data pipeline experience. This individual will play a crucial role in supporting our strategic pipelines and optimize for reliability, efficiency, and performance.

Additionally, ETL Engineering serves as the gold standard for all other YipitData analyst teams, building and maintaining the core pipelines that power our products. This high-impact, high-visibility team is instrumental to the success of our rapidly growing business. This is a unique opportunity to join a fast paced strategic team and help solve some of our biggest data challenges.

We expect US East Coast work hours

• Candidates should be available and willing to work during East Coast business hours (approximately 10/11am – 6/7pm EST).

As Our Data Engineer You Will

• Build end-to-end data pipelines.
• Help with setting best practices for our data modeling and pipeline builds.
• Create documentation, architecture diagrams, and other training materials for analysts.
• Become an expert at solving complex data pipeline issues using Pyspark and SQL.
• Collaborate with stakeholders to incorporate business logic into our central pipelines.
• Deeply learn Databricks, Airflow, and other ETL toolings.

You Are Likely To Succeed If

• You hold a Bachelor’s or Master’s degree in Computer Science, STEM, or a related technical discipline.
• You have 4+ years of experience as a Data Engineer, Data Analyst, or in other technical function.
• You are excited about solving data challenges and learning new skills.
• You have some understanding of working with data or building data pipelines.
• You are comfortable working with large-scale datasets using PySpark or Pandas.
• You understand business needs and the rationale behind data transformations to ensure alignment with organizational goals and data strategy.
• You are eager to constantly learn new technologies.
• You are a self-starter who enjoys working collaboratively with stakeholders.
• You have exceptional verbal and written communication skills.
• Nice to have: Experience with Airflow, dbt, or equivalent.

We are committed to equal employment opportunity regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, marital status, disability, gender, gender identity or expression, or veteran status. We are proud to be an equal-opportunity employer.