Senior Data Engineer

TheCorporate; Oakland (CA); On-site; USD 100,000 - 140,000; Full time

Job summary

A tech company in California seeks a Data Engineer to design and develop robust data models and pipelines. The candidate must have a Bachelor's degree in Computer Science and 5+ years of experience in building data pipelines using Python and Pyspark. The role involves collaborating with analysts for high-quality data solutions and ensuring data integrity across workflows. Familiarity with tools like Tableau and Git is preferred. A strong attention to detail, problem-solving skills, and knowledge of Typescript or Javascript are also essential.

Qualifications

• 5+ years of experience as a data engineer required.
• Hands-on experience with building data pipelines essential.
• Experience with Typescript or Javascript preferred.

Responsibilities

• Design and develop data models and data pipelines.
• Implement and maintain ETL processes.
• Collaborate with analysts to deliver data solutions.

Skills

SQL

Python

Pyspark

Data pipelines

Problem-solving

Education

Bachelor's Degree in Computer Science or related field

Tools

Tableau

Power BI

Git

Palantir Foundry

Job description

Responsibilities

• Work closely with Subject Matter Experts (SMEs) to design and develop the ontology (data model) and data pipelines.
• Design, implement, and maintain robust data pipelines and ETL processes.
• Develop and optimize cloud-based data storage and processing solutions.
• Collaborate with data analysts and data scientists to understand data requirements and deliver high-quality data solutions.
• Ensure data integrity and security across all data workflows and storage solutions.
• Monitor and troubleshoot data pipelines, addressing any issues promptly to ensure the smooth flow of data.
• End to End data pipeline development – responsible for using Python and Pyspark to perform ETL and derive new datasets necessary for business applications.

Minimum Qualifications

• Bachelor's Degree in Computer Science, Engineering, or related field or related experience.
• 5+ years of experience as a data engineer.
• Hands‑on experience with building data pipelines.
• Strong SQL skills and experience working with large datasets and complex data structures.
• Proficient in Python, and Pyspark.
• Experience in Typescript (preferred) or Javascript.
• Excellent problem‑solving and analytical skills with a strong attention to detail.

Preferred Qualifications

• Experience with Palantir Foundry application development and data visualization tools.
• Knowledge with commercial visualization tools such as Tableau or Power BI.
• Experience in Typescript (preferred) or Javascript.
• Databases – familiarity with common relational database models and proprietary instantiations, such as SAP, Salesforce, etc.
• Git – knowledge of version control / collaboration workflows and best practices.

Agile – familiarity with agile and iterative working methodology and rapid user feedback gathering concepts.

UX design – knowledge of best practices and applications.

Skills: pipelines, python, etl, data, cloud, design.