Data Engineer

Vein Standard Metals

United States

À distance

USD 110 000 - 160 000

Plein temps

14 jours+

Une candidature complète en une minute

Un CV et une lettre de motivation personnalisés, prêts à envoyer.

Passez les filtres ATS

Résumé du poste

Vein Standard Metals is seeking a Data Engineer to design, implement, and maintain scalable data pipelines and systems that support data processing, analysis, and visualization. You will collaborate with cross-functional teams to understand data needs, optimize workflows, and ensure data quality and reliability.

Key responsibilities include building ETL pipelines, modeling data, integrating data from databases and APIs, enforcing data quality checks, optimizing performance, and documenting

Qualifications

  • Bachelor's degree in CS, engineering, or related field.
  • 4+ years experience as a Data Engineer or similar role.
  • Proficiency in Python, Java, or Scala and experience with Spark, Flink, or Hadoop.
  • Solid knowledge of SQL and NoSQL databases (PostgreSQL, MySQL, MongoDB, Cassandra).

Responsabilités

  • Data Pipeline Development: Design, build, and optimize ETL data pipelines from multiple sources into data warehouses or data lakes.
  • Data Modeling: Develop and maintain data models and schemas for analytics and ML.
  • Data Integration: Integrate data from databases, APIs, streaming platforms, and third-party systems.
  • Data Quality Assurance: Implement data quality checks and monitoring for reliability.
  • Performance Optimization: Optimize processing and query performance via indexing, partitioning, and caching.
  • Infrastructure Management: Manage databases, data warehouses, ETL pipelines, and data processing clusters.
  • Collaboration and Documentation: Document pipelines, workflows, and configurations for knowledge sharing.

Connaissances

Python
Java
Scala

Formation

Bachelor's degree in Computer Science, Engineering, or related field
Master's degree preferred

Outils

Apache Spark
Apache Flink
Hadoop
AWS
Azure
GCP
PostgreSQL
MySQL
MongoDB
BigQuery
Redshift

Description du poste

Job Description: As a Data Engineer, you will be responsible for designing, implementing, and maintaining scalable data pipelines and systems to support data processing, analysis, and visualization. You will collaborate closely with cross-functional teams to understand data requirements, optimize data workflows, and ensure data quality and reliability.

Responsibilities
  1. Data Pipeline Development: Design, build, and optimize data pipelines to extract, transform, and load (ETL) data from various sources into data warehouses or data lakes, ensuring scalability, reliability, and efficiency.
  2. Data Modeling: Develop and maintain data models and schemas to support business analytics, reporting, and machine learning applications, ensuring data integrity, consistency, and performance.
  3. Data Integration: Integrate data from disparate sources, including databases, APIs, streaming platforms, and third-party systems, to provide unified views of data for analysis and reporting purposes.
  4. Data Quality Assurance: Implement data quality checks and monitoring processes to identify and rectify data anomalies, inconsistencies, and errors, ensuring high data quality and reliability.
  5. Performance Optimization: Optimize data processing and query performance through indexing, partitioning, and caching techniques, leveraging distributed computing frameworks and database technologies.
  6. Infrastructure Management: Manage and maintain data infrastructure components, including databases, data warehouses, ETL pipelines, and data processing clusters, ensuring availability, scalability, and security.
  7. Collaboration and Documentation: Collaborate with data analysts, data scientists, and other stakeholders to understand data requirements and deliver solutions that meet business needs. Document data pipelines, workflows, and system configurations for knowledge sharing and future reference.
Requirements
  • Bachelor's degree in Computer Science, Engineering, or related field. Master's degree preferred.
  • Proven experience 4 years as a Data Engineer or similar role, with a strong track record of designing and implementing data pipelines and systems.
  • Proficiency in programming languages such as Python, Java, or Scala, and experience with data processing frameworks such as Apache Spark, Apache Flink, or Hadoop.
  • Solid understanding of database technologies, including SQL and NoSQL databases (e.g., PostgreSQL, MySQL, MongoDB, Cassandra).
  • Familiarity with cloud platforms such as AWS, Azure, or GCP, and experience with cloud-based data services (e.g., AWS Redshift, Azure SQL Data Warehouse, Google BigQuery).
  • Strong analytical and problem-solving skills, with the ability to troubleshoot complex data issues and optimize data workflows for performance and scalability.
  • Excellent communication and collaboration skills, with the ability to work effectively in cross-functional teams and communicate technical concepts to non-technical stakeholders.
Obtenez votre examen gratuit et confidentiel de votre CV.

ou faites glisser et déposez votre fichier ici.

Similar jobs

Postes similaires à comparer

Data Engineer
Data Engineer

TechDigital Group • Santa Clara (CA)

Sur place
USD 90 000 - 150 000
Data Engineer
Data Engineer

Aptdata Solutions Inc. • Farmington Hills (MI)

Sur place
USD 90 000 - 115 000
Data Engineer
Data Engineer

TechDigital Group • Burbank (CA)

Sur place
USD 120 000 - 150 000
Data Engineer
Data Engineer

EdgeQ Technologies Private Limited • Dallas (TX)

Sur place
USD 100 000 - 160 000
Data Engineer
Data Engineer

VITAL SAÚDE &SEGURANÇA DO TRABALHO • États-Unis

Hybride
USD 90 000 - 120 000
Competitive salary
Performance incentives
Professional development
+1
Data Engineer
Data Engineer

Compunnel Inc. • Westlake (OH)

Sur place
USD 90 000 - 150 000
Data Engineer
Data Engineer

Compunnel Inc. • Orlando (FL)

Sur place
USD 85 000 - 110 000
Data Engineer
Data Engineer

La Concorde Marocaine • États-Unis

Hybride
USD 110 000 - 150 000
Competitive salary
Performance-based incentives
Professional development
+1
Data Engineer
Data Engineer

Compunnel Inc. • Columbus (OH)

Sur place
USD 85 000 - 115 000
Data Engineer
Data Engineer

Advanced Tech Placement • Atlanta (GA)

Sur place
USD 120 000 - 150 000