Data Engineer

Institute for Experiential AI at Northeastern University; Portland (ME); On-site; USD 80,000 - 110,000; Full time

Benefits offered by this job

Medical benefits

Tuition assistance

Paid time off

Retirement benefits

Wellness plans

Job summary

The Institute for Experiential AI at Northeastern University is seeking a skilled IT professional to build and enhance data ingestion and transformation components. This role involves utilizing strong programming skills in Java and Scala along with expertise in Big Data technologies to support various AI projects.

Ideal candidates will have a Bachelor’s degree in Computer Science and 5+ years of relevant experience. This position provides opportunities for collaboration across academia and industry, along with a comprehensive benefits package, including medical, tuition assistance, and paid time off.

Qualifications

• 5 or more years of professional experience in IT.
• Experience in cloud architecture, particularly AWS, is preferred.

Responsibilities

• Build, maintain, and improve data pipelines and infrastructure.
• Support data scientists and end users.
• Support practicum courses and academic programs.

Skills

Java/Scala development

Python programming

ETL pipelines

Big Data pipeline processing

Cloud platforms (AWS, Azure, Google Cloud)

Containerization (Docker, Podman)

Machine learning frameworks (SKLearn, AutoML)

Deep learning frameworks (Pytorch)

Data governance best practices

Version control best practices

Relational and non-relational databases

Education

Bachelor’s degree in Computer Science or Engineering

Job description

Summary

In this role, you will be responsible for building out key components of data ingestion and transformation, using a combination of strong Java/Scala development skills as well as Big Data processing technologies. As a member of the team, you will be expected to take ownership of individual platform components and help set the vision and architecture for it. In the process, you will identify the requirements of new features, and propose design and drive the solution.

Minimum Requirements

Education & Experience: Bachelor’s degree in Computer Science, Engineering, or similar disciplines and 5 or more years of professional experience, or can convincingly demonstrate this level of skill.

Knowledge and Skills

• Expertise writing clear, maintainable, production-level code in Python.
• Experience with cloud platforms such as Azure, AWS, or Google Cloud.
• Experience with containerized application development with Docker or Podman.
• Expertise on ETL pipelines, data migration, data cleaning.
• Experience with big data pipelines (Apache Spark, Kafka, AirFlow) is a plus.
• Experience with deep learning frameworks, such as Pytorch, is a plus.
• Experience with machine learning frameworks such as SKLearn, AutoML is a plus.
• Experience with vectors, matrix, dataframe libraries such as Numpy, Pandas is a plus.
• Expertise on best practices for data governance, curation, validation, integrity.
• Expertise on version control best practices.
• Experience with relational and non-relational databases.
• Experience preparing well-documented, reproducible results for external stakeholders.

Preferred Experience

• Expertise in setting up cloud architecture, particularly in AWS.
• Experience working on a collaborative codebase with data science and engineering.
• Data science, analytics, or machine learning.
• Experience within a particular domain such as healthcare, logistics, genomics.
• Open source ecosystem contributions or expertise.

Values & Abilities

• Aptitude to independently learn new technologies, prototype and propose software design and solutions.
• Ability to communicate effectively across academia and industry.
• Team-player who can collaborate effectively across many teams within the University.
• Open-minded and assertive when collaborating and working within our team and with other groups within Northeastern University.
• Entrepreneurial mind‑set with an ability to navigate complex structures and processes.

Key Responsibilities & Accountabilities

40% - Build, maintain and improve data pipelines, infrastructure and tools to support Institute projects and operations in data science and AI.

10% - Support data scientists and other end users such as faculty and postdocs.

20% - Support delivery of practicum courses and other academic programs as subject matter expert.

30% - Participate in delivery of projects at the AI Solutions Factory to solve AI problems of industry partners.

Position Type

Information Technology.

Additional Information

Northeastern University considers factors such as candidate work experience, education and skills when extending an offer.

Northeastern has a comprehensive benefits package for benefit eligible employees. This includes medical, vision, dental, paid time off, tuition assistance, wellness and life, retirement- as well as commuting and transportation. For more information, visit https://hr.northeastern.edu/benefits/.