Data Engineer

TechDigital Group; Santa Clara (CA); On-site; USD 90,000 - 150,000; Full time

Job summary

An innovative firm is looking for a talented Data Engineer to enhance its data infrastructure. In this exciting role, you will be responsible for developing and optimizing ETL pipelines, managing data integrations, and ensuring data quality. Collaborating with cross-functional teams, you will help drive analytics and reporting efforts by building robust data solutions. If you have a passion for data and a strong background in engineering, this is the perfect opportunity to make a significant impact in a dynamic environment.

Qualifications

• 10+ years of experience in data engineering or related roles.
• Proficiency in SQL and ETL tools with big data technologies.

Responsibilities

• Design and develop ETL pipelines for efficient data flow.
• Architect and maintain data warehousing solutions.

Skills

Data Engineering

ETL Processes

Big Data Technologies

Cloud Data Platforms

SQL

Data Modeling

Data Governance

Collaboration

Education

Bachelor's degree in Computer Science

Engineering degree

Tools

Apache Airflow

Talend

PostgreSQL

MySQL

Redshift

Hadoop

Spark

AWS

Azure

GCP

Job description

We are seeking a highly skilled Data Engineer to join our team, focusing on building, maintaining, and optimizing our data infrastructure. As a Data Engineer, you will develop data pipelines, manage data integrations, and support data architecture to enable efficient analytics and reporting. This role requires expertise in ETL processes, big data technologies, and cloud data platforms.

Key Responsibilities:

1. Data Pipeline Development: Design, develop, and optimize ETL pipelines to ensure reliable and efficient data flow across systems.
2. Data Warehousing: Architect, build, and maintain data warehouse solutions to support analytical and reporting needs.
3. Data Quality & Integrity: Implement data quality checks and cleaning processes to ensure accuracy, consistency, and reliability of data.
4. Collaboration: Work closely with data analysts, data scientists, and cross-functional teams to understand data needs and deliver solutions.
5. Optimization: Improve data infrastructure and optimize performance for better data processing and querying capabilities.
6. Documentation: Create and maintain documentation of data architecture, pipelines, and workflows for internal use and compliance.

Qualifications:

1. Bachelor's degree in Computer Science, Engineering, or a related field.
2. 10+ years of experience in data engineering, data warehousing, or related roles.
3. Proficiency in SQL and experience with database management systems (e.g., PostgreSQL, MySQL, Redshift).
4. Experience with ETL tools (e.g., Apache Airflow, Talend) and big data technologies (e.g., Hadoop, Spark).
5. Hands-on experience with cloud platforms (e.g., AWS, Azure, GCP) and cloud data storage solutions.
6. Programming skills in Python, Java, or Scala.
7. Strong understanding of data modeling, data architecture, and data governance principles.

Preferred Skills:

1. Familiarity with data visualization tools (e.g., Tableau, Power BI).
2. Knowledge of NoSQL databases (e.g., MongoDB, Cassandra).