Senior Data Engineer

JBHired; United States; Remote; USD 120,000 - 190,000; Full time

Job summary

JBHired seeks a Senior Data Engineer to build the architecture and implementation of the data platform, pipelines and infrastructure. You will work with analytics teams, data scientists and other data warehouse engineers to understand requirements, analyze performance and troubleshoot issues.

Responsibilities include designing scalable ETL packages, data modeling, and enabling migrations across databases. Strong SQL, Shell scripting and governance expertise are required, with exposure to AWS,

Qualifications

• 5+ years of experience in data engineering.
• Experience with enterprise data warehouses and data pipelines.
• Expert in ETL technologies, SQL development and Shell scripting.
• Experience building data models and managing data quality.
• Knowledge of data governance and agile development practices.
• University degree in a quantitative subject.

Responsibilities

• Design and build the architecture and implementation of the data platform, data pipelines and infrastructure.
• Collaborate with data analytics teams, data scientists and other data warehouse engineers to meet requirements and troubleshoot issues.
• Lead development of scalable ETL packages and data models including facts, aggregates and dimensions; enable data migrations across databases.
• Ensure data governance and data quality across the Data and Analytics department.
• Work with the entire Data and Analytics team to support data-centric needs.

Skills

ETL technologies

SQL development

Shell scripting

Data modelling

Data governance

Agile development

Cloud computing

Education

Bachelor's degree in a quantitative subject

Tools

MySQL

Microsoft SQL

Oracle

Hadoop

Spark

NoSQL databases

AWS

SSIS

SSRS

SSAS

Power BI

Job description

The Senior Data Engineer will build the architecture and implementation of the data platform, data pipeline and infrastructure.

• Responsible for the development, maintenance, improvement, cleaning, and manipulation of data in the context of the data platform.
• Work with data analytics teams, data scientists, and other data warehouse engineers in order to understand and aid in the implementation of enterprise data warehouse/enterprise data platform requirements, analyze performance, and troubleshoot any existing issues.
• Have to be an expert in ETL technologies, SQL development, and Shell Scripting, further spearheading the data and analytics database design (Logical Models and Physical Data Models), creation of master data and maintaining data flow activities.
• Lead as a pioneer in a data-driven organization and this role is instrumental in making this happen.

Key Results Area:

• Design and create enterprise data platform systems optimized for performance, implementing schema changes, and maintaining data architecture standards across all of the business functions
• Additionally tasked with designing and developing scalable ETL packages from the business source systems and the development of ETL routines/jobs in order to populate databases from sources and also to create FACTs, Aggregates and dimensions
• In this capacity, the Data Engineer is also responsible for enabling and running data migrations across different databases and servers in the ETL data pipelines. For example, data migration from MySQL/Oracle to SQL servers. She/He defines and implements data stores based on the system requirements and consumer requirements
• Strive to ensure proper data governance and quality across the Data and Analytics department and the business as a whole
• Work collaboratively with the entire Data and Analytics team, providing support to the entire department for its data-centric needs.

Competencies & Behaviors:

• Technical: Keeps up to date with the latest trends in data and data engineering (Both on-premises and Cloud Practices)
• Passion for data: Belief that data and insight can grow and transform an organization
• Partnership: Successful professional approaches to collaboration with all stakeholders and teams
• Communication: Creating and promoting an enabling environment for open communication; Constructively challenging those with power and authority
• Governance: Work well with the Management, regardless of its composition; contribute to Management; Adhere to clear lines of responsibility and accountability
• Management: Create a positive and productive work environment, Model proper staff behaviour and promote inclusive practices; create a sense of shared responsibility/credit for accomplishments and shared responsibility for challenges or failures; Lead an efficient and effective organizational operation according to best practices
• Decision-making: Delegate appropriate decisions and responsibilities; Make clear and timely decisions; Fair and transparent decision-making
• Organizational Development: Create a work environment in which learning is; create an emotionally intelligent organization and staff competence; Build and promote effective teams, creating an environment of creativity and innovation

Job Requirements:

• 5+ Years experience in data engineering
• Experience with high data volumes, both digital and transactional
• Experience in building and planning a new data infrastructure for a sizable organization
• Must have hands-on experience in full project development lifecycle related to Data Warehouse and engineering
• Understanding Agile development will be an added advantage
• University degree in a quantitative subject
• Language Requirement: Thai, English (written and verbal)

Technology and Methodology:

• Essential: Strong Relational Database experience (MySQL, Microsoft SQL and Oracle)
• Essential: Experience in analytical and quantitative data pipeline creation and data processing with one or more big-data components such as Hadoop, Spark, NoSQL databases
• Essential: Familiarity in building, configuring and monitoring data platforms using highly scalable distributed computing solutions in AWS or equivalent
• Knowledge of data warehousing principles. (Data Wrangling, Data Profiling, Integration, etc.)
• Data modelling skills (Entity Relationships, Logical Data Model, Physical Data Model, Dimensional Modeling, etc.)
• Expert-level hands-on experience in ETL and Reporting technologies like SSIS, SSRS, SSAS and Power BI 3-dimensional cube design
• Ability to develop and update detailed technical documentation
• Awareness of IoT platforms/devices used in FinTech and Digital Money/Payments