Data Engineer

Realign LLC

Charlotte (NC)

On-site

USD 101,000 - 137,000

Full time

6 days ago

Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Realign LLC in Charlotte, NC is seeking a Data Engineer to design and implement complex data solutions with a focus on big data processing and cyber security domain knowledge. This onsite role requires close collaboration with business partners, the Product Owner, and the development team to deliver data-driven outcomes.

You will mentor junior engineers, build scalable data pipelines, and participate in governance, deployment, and CI/CD activities, ensuring data quality and timely insights for

Qualifications

Minimum 8+ years of hands-on data engineer and/or software development experience.
Experience with big data processing systems such as Hadoop and Cloudera.
Programming with Python, including modular Python programming.
Strong SQL and UNIX experience.

Responsibilities

Ensure the design and engineering approach for complex data solutions remains consistent across multiple flows and systems.
Perform research, design, and prototyping activities, including evaluating new toolsets.
Build processes for data transformation, data structures, metadata, data quality controls, dependency tracking, and workload management.
Define and build data pipelines that support faster, better, data-informed business decisions.
Mentor more junior Data Engineers and coach team members for delivery and release CI-CD events, including branching timelines, pull requests, issue triage, merge or conflict resolution, and release notes.

Skills

Python

SQL

UNIX

Hadoop

Cloudera

Hive

Spark

PySpark

AWS

Azure

Bitbucket

CI/CD

Data security

SDLC

Tools

Bitbucket

Job description

The Data Engineer role at Realign LLC is a full-time, permanent position based in Charlotte, NC (onsite). The work focuses on complex data solution engineering with a strong emphasis on big data processing and cyber security domain knowledge, including related vulnerability data management.

Role Overview

In this role, you will participate as a team member under the supervision of a more senior domain expert. Primary interactions include business partners and stakeholders, Product Owner, Scrum Master, development team, feature lead, architect, domain community of practice, and data scientists.

Objective of role: knowledge in the cyber security domain.

Key Responsibilities

Ensure the design and engineering approach for complex data solutions remains consistent across multiple flows and systems.
Perform research, design, and prototyping activities, including evaluating new toolsets.
Utilize multiple architectural components to design and develop client requirements.
Maintain, improve, clean, and manipulate data for operational and/or analytics data systems.
Assemble large, complex datasets aligned to functional and non-functional requirements.
Build processes for data transformation, data structures, metadata, data quality controls, dependency tracking, and workload management.
Define and build data pipelines that support faster, better, data-informed business decisions.
Contribute to integration, regression, and performance test suites; analyze reports, identify test issues or errors, and triage root causes.
Document and communicate deployment, maintenance, support, and business functionality requirements.
Follow team delivery and release processes, including code deployment and release cadence.
Identify gaps in data management standards adherence and collaborate with partners to create plans to close gaps.
Lead concept and experimentation testing and synthesize results to validate and improve analytical solutions.
Define key performance indicators and internal controls.
Mentor more junior Data Engineers and coach team members for delivery and release CI-CD events, including branching timelines, pull requests, issue triage, merge or conflict resolution, and release notes.

Required Skills and Experience

Minimum 8+ years of hands-on data engineer and/or software development experience, including at least 5 years onsite experience.
Experience with big data processing systems such as Hadoop and Cloudera.
Programming experience with Python, including modular Python programming.
Strong SQL and UNIX experience.
Experience with Hadoop components such as Hive, Spark, and PySpark (listed as Pispark).
Proven technology leadership, guiding junior engineers toward project completion and deliveries.
Cyber security domain experience, preferably vulnerability data management.
Ability to switch between technology leadership and hands-on coding when needed.
Experience with cloud services and technologies such as AWS and Azure.
Experience with data stores including Hive, HBase, and Elastic.
Experience with Bitbucket for code versioning.
Experience with CI/CD pipelines.
Good knowledge of data engineering principles and the SDLC process.
Ability to detect early and articulate requirement challenges and gaps with business partners.