Senior Data Engineer

Valiant Technologies LLC; Harrisburg (Dauphin County); On-site; USD 95,000 - 150,000; Full time

Job summary

Valiant Technologies LLC is seeking a data engineering professional to build and optimize cloud-based data services and pipelines in a fast-paced environment. You will work with AWS, big data tools, and ETL platforms to design scalable data architectures.

The role requires strong SQL/NoSQL proficiency, data governance awareness, and experience with REST APIs and workflow schedulers. This position favors candidates with experience in Python/Java/C++ and data integration tooling.

Qualifications

• Hands-on experience with cloud-based data services (AWS) and data integration.
• Proficiency in object-oriented scripting languages (Python, Java, C++, Scala).
• Strong SQL and NoSQL database knowledge (Oracle, MS SQL Server, Postgres, Cassandra).
• Experience with version control and CI/CD pipelines (Git).
• Knowledge of data modeling (dimensional, data vault, star/snowflake).
• Familiarity with AWS Glue Data Catalog and data governance concepts.
• Understanding of REST API design and integration patterns.
• Experience with big data tools (Hadoop, Spark, Kafka).
• Experience with data pipelines/workflows (Airflow, Azkaban, Luigi).
• Experience with ETL tooling (Informatica, MuleSoft, Talend, TIBCO).
• Experience with stream processing (Storm, Spark-Streaming).

Responsibilities

• Design and implement cloud-based data services and pipelines.
• Develop and optimize data models and metadata management.
• Collaborate on REST API integration and data governance processes.
• Work with big data tools (Hadoop, Spark, Kafka) to process streams.
• Manage data integration via ETL tools (Informatica, Talend, MuleSoft, Tibco).
• Build and maintain data pipelines using Airflow or similar schedulers.

Skills

AWS

Python

Java

C++

Scala

SQL

NoSQL

Git

CI/CD

Data modeling

Data governance

REST APIs

Big data

Airflow

Informatica

MuleSoft

Talend

TIBCO

Hadoop

Spark

Kafka

Storm

Spark-Streaming

Tools

AWS Glue Data Catalog

Azure/AWS Data Services

Git

CI/CD pipelines

Job description

US Citizen, Green Card, H-1B, OPT-EAD, CPT, Other

Work Authorization: US Citizen, Green Card, OPT-EAD, CPT, H-1B, H4-EAD, Other

Job Description:

• Hands-on experience with cloud-based data services such as AWS (e.g. S3, EC2, Glue, EMR, RDS, Redshift, Athena, Lambda, DynamoDB)
• Proficiency in object-oriented/object function scripting languages: e.g., Python, R, Java, C++, Scala
• Proficiency in relational SQL and NoSQL databases, including Oracle, MS SQL Server, Postgres, Cassandra.
• Experience with version control systems such as Git and CI/CD pipelines
• Knowledge of data modeling techniques including dimensional modeling, data vault, and star/snowflake schemas.
• Familiarity with data cataloging and metadata management tools such as AWS Glue Data Catalog.
• Understanding of data governance, data lineage, and data quality frameworks.
• Knowledge of REST API design and integration patterns.
• Experience with big data tools: e.g. Hadoop, Spark, Kafka.
• Experience with data pipeline and workflow management tools: e.g. Azkaban, Luigi, Airflow.
• Experience with data integration services solutions from vendors such as Informatica, MuleSoft, Talend, and TIBCO.
• Experience with stream-processing systems: e.g. Storm, Spark-Streaming.

We are an equal opportunity employer. All aspects of employment including the decision to hire, promote, discipline, or discharge, will be based on merit, competence, performance, and business needs. We do not discriminate on the basis of race, color, religion, marital status, age, national origin, ancestry, physical or mental disability, medical condition, pregnancy, genetic information, gender, sexual orientation, gender identity or expression, national origin, citizenship/ immigration status, veteran status, or any other status protected under federal, state, or local law.