Jr. Data Engineer

Unrivaled Metal Buildings; New York (NY); On-site; USD 70,000 - 90,000; Full time

Job summary

A leading company in metal structures based in New York is seeking a Junior Data Engineer to develop scalable data processing pipelines. The successful candidate will collaborate with a data team to enhance data sources and ensure data integrity. Required qualifications include a BA or BS in Computer Science or related fields, along with experience in Apache Spark, SQL, and ETL processes. Knowledge of Informatica is a plus. This position requires US citizenship.

Qualifications

• Bachelor's degree in a relevant field is required.
• Experience with ETL processes using Apache Spark or Databricks is important.
• Familiarity with SQL or Spark SQL is needed.

Responsibilities

• Develop and maintain scalable automated data pipelines.
• Construct cloud data solutions based on best practices.
• Support regular and ad hoc data querying and analysis.

Skills

Apache Spark

ETL pipelines

SQL

Automated testing

Dimensional modeling

Education

BA or BS in Computer Science, Information Systems, or related discipline

Tools

Databricks

Spark SQL

Informatica

Job description

Overview

We seek a Junior Data Engineer with a passion for developing highly scalable data processing pipelines. The engineer will collaborate with a data team to add new and diverse data sources to our warehouse, continuously enhance our pipeline to ensure high availability of data to our customers, and ensure the integrity of the data delivered to our customers.

Responsibilities

Develop, construct, and maintain scalable automated data pipelines.

Construct cloud data solutions and provide domain‑specific insight regarding storage, big data platform services, serverless architecture, RDBMS, DW/DM, NoSQL databases, and security.

Deploy, automate, maintain, and manage a cloud‑based production system in order to assure the system's availability, performance, scalability, and security.

Implement data management best practices, including master data, reference data, metadata, data quality, and data lineage.

To comprehend customer behavior, support regular and ad hoc data querying and analysis.

Qualifications

BA or BS in Computer Science, Information Systems, or a related discipline.

Experience constructing and deploying ETL pipelines utilizing Apache Spark and/or Databricks.

Experience with either SQL or Spark SQL.

Knowledge of Informatica is an advantage.

Dimensional model expertise in data warehouses.

Experience working with Star Schema data importing.

Expertise and familiarity with automated testing.

Knowledge and experience with Spark Notebooks deployment.

Clearance Requirement

Citizen of the United States with the ability to obtain and maintain US Suitability.

Equal Opportunity Employer

unrivaled metal buildings is an equal opportunity employer who recognizes and appreciates the value diversity brings to the workplace. All qualified applicants will be considered for employment regardless of race, color, religion, gender, national origin, disability, veteran status, or any other legally protected characteristic.