Data Engineer

iVoyant; Atlanta (GA); On-site; USD 100,000 - 130,000; Full time

Job summary

A leading technology company in Atlanta is looking for experienced Data Engineers to join its digital team. The role focuses on optimizing data engineering processes, ensuring data quality, and implementing ETL solutions using tools like Informatica and SQL. Candidates should have significant experience (7+ years) in data engineering and analysis. A Bachelor’s degree in a relevant field is required, and skills in data modeling and leading teams are crucial. This position offers opportunities for contributing to complex data architectures.

Qualifications

• 7+ years of experience in data engineering.
• 7+ years of experience in data analysis using SQL and PowerBI.
• 4+ years of experience in data modeling and working with business.
• 3+ years of team leadership in technical solutions.

Responsibilities

• Take ownership of assigned Functional Data Domain.
• Work with Functional team to understand data requirements.
• Analyze and profile data to fit business needs.
• Design Data Engineering Pipelines and Quality validation rules.
• Work with Developers for technical implementation of Data Pipelines.
• Support maintenance and performance analysis along data pipeline.

Skills

SQL

Data Integration

Critical Thinking

Cross-functional Teamwork

Problem Solving

ETL Process

Data Pipelines

Education

Bachelor’s Degree in Computer Science, Engineering, or Mathematics

Tools

SQL

Informatica BDM

PowerBI

Hadoop

Python

Job description

iVoyant is looking for multiple Data Engineers to join our client Digital Team. The Data Engineers will work with the client functional SMEs within the Digital and the Business team to understand the requirements and prepare technical design. They will ensure the availability and validity of the data to ensure that it fits with the requested business rules. The Data Engineers will create and maintain the Source-to-Target data mappings, Logical and Physical Data Model and, business and technical test cases to ensure data quality. In addition, they will also assist a team of developersin developing the data pipeline (using Informatica BDM or Python) to ensure ongoing data collection, consolidation, and management.

The Data Engineers will aid in the optimization of operations by manipulating and aggregating the disparate data sources into a format that is easily digestible by both data scientists and statistically adept colleagues. Their core responsibility will be to combine large volumes of disparate complex data, conduct quality checks on the data, manipulate the data and ensure continuous access to a clean format of the operational data for data scientists and other stakeholders.

Responsibilities

• Take ownership of the Functional Data Domain assigned to the Data Engineer. Gather knowledge of the business, functions, and their data analysis needs along with business’s relationship with the Data.
• Work with Functional team to understand requirements, data sources and business rules.
• Identify, Analyze and Profile the data across various source systems to understand and ensure that data fits the business requirements. Raise any data anomalies and work with the functional users to close any anomalies or data gaps.
• Identify relationships across multiple source systems; and create reference data with the help of data stewards.
• Design the Dimensional Data Model, Data Engineering Pipelines, Data Quality and Validation rules based on the business requirements and rules that would include any error handing, exception management and data quality routines to expose the anomalies in the data.
• Finalize the technical Source-to-Target mapping to develop the data pipelines from source systems to the dimensional model.
• Work with the team of Developer(s) for the technical implementation of the Data Pipelines. Ensure that the Unit and Integration tests complete successfully. Drive the QA/UAT with the Functional users and ensure a deployment to higher environments.
• Designs and jointly develops the data architecture with data architect and ensures security and maintenance
• Identifies gaps and implements solutions for data security, quality and automation of processes
• Supports maintenance, bug fixing and performance analysis along data pipeline
• Diagnoses existing architecture and data maturity and identifies gaps
• Gather requirements, assess gaps and build roadmaps and architectures to help the analytics driven organization achieve its goals

Qualifications

• 7+ years of experience in data engineering.
• 7+ years of working on data analysis using SQL, PowerBI and other equivalent tools
• 4+ years of experience in data modeling, data analysis and working with business.
• 3+ years of experience in leading a team of developers and drive technical solutions.

Required Skills

• Knowledgeof SQL, databases and tables structure to query the data from various places, integrate and present the results to the business
• Knowledge of ETL, data pipelines using Informatica (DEI/BDM) or equivalent technologies.
• Strong critical thinking and problem solving skills
• Success at working on cross-functional teams to meet a common goal
• Self-starter with a high sense of urgency

Other Skills

• Knowledge of Big Data platforms (e.g. Hadoop, Map/Reduce, Spark, HBase, HDInsight, Data Bricks, Hive) and with programming languages like UNIX shell scripting, Python etc.
• Knowledge of reporting and BI packages e.g., PowerBI, Tableau, SAP BO etc.

Education

Bachelor’s Degree in Computer Science, Engineering, and/or background in Mathematic and Statistics; Master’s or other advanced degree a plus