Senior Product Software Data Engineer

Wolters Kluwer; United States; Remote; USD 80,000 - 120,000; Full time

Job summary

An established industry player is seeking a skilled Data Engineer to design and develop high-performance data pipelines. This role involves collaborating with business users and data scientists to implement machine learning models and enhance data governance. The ideal candidate will have extensive experience in data pipeline development, particularly with Azure and big data technologies. Join a dynamic team that values innovation and operational excellence, where your contributions will directly impact the efficiency and effectiveness of data management processes.

Qualifications

• 8+ years of overall experience with 5+ years in data pipelines.
• Experience with Azure data pipeline development and big data technologies.

Responsibilities

• Design and develop optimized data pipelines for real-time data movement.
• Collaborate with team members to review use cases and improve data governance.

Skills

Data Pipeline Development

Scala

Spark

Azure Databricks

SQL Server

T-SQL

Machine Learning

Big Data Technologies

Data Governance

Agile/Scrum

Education

Bachelor's degree in Computer Science

Tools

Azure Data Factory

Azure Data Lake

Power BI

MS BI Stack (SSIS/SSRS/SSAS)

Hadoop

Kafka

Job description

Job Description Summary

Under moderate supervision, designs, develops, tests, debugs and implements moderately complex operating systems components, software tools, and utilities required for the operation, maintenance, and control of computer systems. Makes moderately complex modifications to existing software to fit specialized needs and configurations, and maintains program libraries and technical documentation. Assists in planning, scheduling and implementation. Initiates corrective action to stay on schedule.

ESSENTIAL DUTIES AND RESPONSIBILITIES

· Participate and collaborate with team members to review the use cases for data pipelines.

· Design and develop highly optimized and reliable enterprise level data pipelines to move data to data warehouse in real-time.

· Review and analyze existing business process, identify any gaps, and recommend improvements.

· Design and develop modern real-time solutions with Scala, Spark, Azure Databricks.

· Partner with business users and data scientists to build and deploy machine learning models.

· Improve data governance and operational excellence.

· Develop data expertise and take ownership of data quality for the newly developed pipelines.

· Adhere to company coding standards and best practices.

· Conduct code reviews and create technical solution document.

MINIMUM QUALIFICATIONS

· 8+ overall experience

· 5+ years of building data pipelines for structured and un-structured data.

· 2+ years of experience of azure data pipeline development.

· 3 + years of experience with Hadoop, Azure Databricks, Stream Analytics, Eventhub, Kafka.

· 3+ years of experience of relational and dimensional data modeling with SQL Server and data warehouse.

· 3+ years of experience with SQL Server, T-SQL, stored procedure development and database optimization.

· 3+ years of experience with MS BI stack (SSIS/SSRS/SSAS, Power BI).

· 2 + years of experience with Azure data platform (Azure data factory, Azure data lake, Azure SQL DB, Azure DW/Synapse).

· 2 + years of experience with Azure ML, Python, Scala

· 3+ years of experience with big data technologies.

· 2 + years of experience with Power BI, Azure Analysis Services

· Experience with working with business stakeholders, requirements & use case analysis.

· Strong communication and collaboration skills with creative problem-solving skills.

PREFERRED QUALIFICATIONS

· Bachelor's degree in Computer Science or equivalent work experience.

· Experience with Agile/Scrum methodology.

· Experience with tax and accounting domain a plus.

· Azure Data Engineer certification a plus.

Applicants may be required to appear onsite at a Wolters Kluwer office as part of the recruitment process.