Data Engineer

Ethereum Technologies; San Jose (CA); On-site; USD 130,000 - 200,000; Full time

Job summary

Ethereum Technologies is seeking a data engineer to design, develop, and maintain scalable data pipelines and ETL workflows. You will work with Python, SQL, PySpark, and cloud tools like Azure Data Factory, Databricks, and Snowflake to integrate data across systems and deliver analytics-ready datasets.

The role emphasizes collaboration with data analysts, data scientists, and business teams, while participating in Agile ceremonies and CI/CD practices to ensure reliable data delivery.

Qualifications

• Bachelor's degree required in CS/IT or related field.
• Experience building data pipelines and ETL/ELT processes.
• Proficient in Python, SQL, and PySpark.
• Experience with cloud platforms and data warehousing.
• Strong data modeling and data integration understanding.
• Excellent analytical, problem-solving, debugging, and communication skills.

Responsibilities

• Designed, developed, and maintained scalable data pipelines to extract, transform, and load data from multiple sources.
• Developed ETL/ELT workflows using Python, SQL, PySpark, and Azure Data Factory.
• Built and optimized data processing solutions using Apache Spark and Databricks.
• Integrated data from multiple enterprise systems using REST APIs and database connectors.
• Monitored data pipelines, identified failures, and resolved performance and data processing issues.
• Implemented incremental data loading, partitioning, and optimization techniques to improve pipeline performance.
• Worked with Snowflake for data loading, transformation, query optimization, and warehouse management.
• Used Git and Azure DevOps for source control, CI/CD, and deployment management.
• Collaborated with data analysts, data scientists, and business teams to understand data requirements.
• Participated in Agile/Scrum ceremonies, technical discussions, code reviews, and production support.

Skills

Analytical thinking

Problem-solving

Communication skills

Collaboration

Education

Bachelor's degree in Computer Science, Information Technology, or related field

Tools

Python

SQL

PySpark

Azure Data Factory

Databricks

Snowflake

Git

Azure DevOps

Job description

US Citizen, Green Card, H-1B, OPT-EAD, CPT, Other

Roles and Responsibilities

Designed, developed, and maintained scalable data pipelines to extract, transform, and load data from multiple sources.

Developed ETL/ELT workflows using Python, SQL, PySpark, and Azure Data Factory.

Built and optimized data processing solutions using Apache Spark and Databricks.

Worked with structured and unstructured data from relational databases, APIs, flat files, and cloud storage.

Designed and implemented data integration solutions to support business intelligence and analytics requirements.

Developed complex SQL queries, stored procedures, views, and functions to transform and process large datasets.

Implemented data validation, data cleansing, data transformation, and data quality checks to ensure data accuracy.

Worked with Azure Data Lake Storage and cloud-based data platforms to store and manage large volumes of data.

Created and maintained data warehouses, data marts, and dimensional data models.

Integrated data from multiple enterprise systems using REST APIs and database connectors.

Monitored data pipelines, identified failures, and resolved performance and data processing issues.

Implemented incremental data loading, partitioning, and optimization techniques to improve pipeline performance.

Worked with Snowflake for data loading, transformation, query optimization, and warehouse management.

Implemented data security, access controls, and governance practices to protect sensitive business information.

Used Git and Azure DevOps for source control, CI/CD, and deployment management.

Collaborated with data analysts, data scientists, and business teams to understand data requirements.

Participated in Agile/Scrum ceremonies, technical discussions, code reviews, and production support.

Technical Skills

Programming: Python, SQL, PySpark

Big Data: Apache Spark, Hadoop, Databricks

Data Modeling: Star Schema, Snowflake Schema, Dimensional Modeling

Reporting: Power BI

Methodologies: Agile, Scrum, SDLC

Qualifications

Bachelor's degree in Computer Science, Information Technology, or a related field.

Experience in data engineering, ETL development, and data pipeline implementation.

Strong programming knowledge in Python, SQL, and PySpark.

Experience working with cloud platforms and data warehousing technologies.

Understanding of data modeling, database concepts, and data integration.

Strong analytical, problem-solving, debugging, and communication skills.

We are an equal opportunity employer. All aspects of employment including the decision to hire, promote, discipline, or discharge, will be based on merit, competence, performance, and business needs. We do not discriminate on the basis of race, color, religion, marital status, age, national origin, ancestry, physical or mental disability, medical condition, pregnancy, genetic information, gender, sexual orientation, gender identity or expression, national origin, citizenship/ immigration status, veteran status, or any other status protected under federal, state, or local law.