Senior Data Engineer - Cloud Data Platform (AWS, Spark)

Valiant Technologies LLC; Harrisburg (Dauphin County); In loco; USD 95.000 - 150.000; Tempo pieno

Descrizione del lavoro

Valiant Technologies LLC is seeking a data engineering professional to build and optimize cloud-based data services and pipelines in a fast-paced environment. You will work with AWS, big data tools, and ETL platforms to design scalable data architectures.

The role requires strong SQL/NoSQL proficiency, data governance awareness, and experience with REST APIs and workflow schedulers. This position favors candidates with experience in Python/Java/C++ and data integration tooling.

Competenze

• Hands-on experience with cloud-based data services (AWS) and data integration.
• Proficiency in object-oriented scripting languages (Python, Java, C++, Scala).
• Strong SQL and NoSQL database knowledge (Oracle, MS SQL Server, Postgres, Cassandra).
• Experience with version control and CI/CD pipelines (Git).
• Knowledge of data modeling (dimensional, data vault, star/snowflake).
• Familiarity with AWS Glue Data Catalog and data governance concepts.
• Understanding of REST API design and integration patterns.
• Experience with big data tools (Hadoop, Spark, Kafka).
• Experience with data pipelines/workflows (Airflow, Azkaban, Luigi).
• Experience with ETL tooling (Informatica, MuleSoft, Talend, TIBCO).
• Experience with stream processing (Storm, Spark-Streaming).

Mansioni

• Design and implement cloud-based data services and pipelines.
• Develop and optimize data models and metadata management.
• Collaborate on REST API integration and data governance processes.
• Work with big data tools (Hadoop, Spark, Kafka) to process streams.
• Manage data integration via ETL tools (Informatica, Talend, MuleSoft, Tibco).
• Build and maintain data pipelines using Airflow or similar schedulers.

Conoscenze

AWS

Python

Java

C++

Scala

SQL

NoSQL

Git

CI/CD

Data modeling

Data governance

REST APIs

Big data

Airflow

Informatica

MuleSoft

Talend

TIBCO

Hadoop

Spark

Kafka

Storm

Spark-Streaming

Strumenti

AWS Glue Data Catalog

Azure/AWS Data Services

Git

CI/CD pipelines

Descrizione del lavoro

Valiant Technologies LLC is seeking a data engineering professional to build and optimize cloud-based data services and pipelines in a fast-paced environment. You will work with AWS, big data tools, and ETL platforms to design scalable data architectures.

The role requires strong SQL/NoSQL proficiency, data governance awareness, and experience with REST APIs and workflow schedulers. This position favors candidates with experience in Python/Java/C++ and data integration tooling.