Data Engineer

American Bureau of Shipping; Knoxville (AL); On-site; USD 110,000 - 160,000; Full time

Job summary

American Bureau of Shipping is seeking a Data Engineer to design, build, and maintain scalable data pipelines and lakehouse architectures to support analytics, reporting, and AI/ML workloads.

You will develop ETL/ELT processes across diverse data sources, ensure data quality, and support governance with lineage and cataloging practices. Strong collaboration with data scientists and engineers is essential.

Qualifications

• 8+ years of experience in data engineering with cloud pipelines.
• Degree in a technical field; Databricks certification preferred.

Responsibilities

• Design, build, and maintain scalable data pipelines and lakehouse/data warehouse structures.
• Develop ETL/ELT workflows ingesting diverse data sources.
• Prepare datasets for feature engineering, model training, and AI initiatives.
• Implement data quality controls, validation, monitoring, and observability.
• Manage data integration across internal and external systems.

Skills

Python

SQL

Databricks

Cloud platforms

Data modeling

ETL/ELT

Spark

Docker

Kubernetes

Data governance

Data quality

Streaming data

Education

Bachelor's degree in a technical field

Tools

Databricks

Snowflake

Redshift

Docker

Kubernetes

AWS

Azure

Google Cloud

Job description

The Data Engineer designs, builds, and maintains scalable data architecture, pipelines, and platforms that support analytics, reporting, and AI/ML solutions across the organization. This role is responsible for preparing, integrating, governing, and optimizing data from multiple sources to ensure reliable, accessible, and high-quality data assets for business and technical use. The Data Engineer works closely with data scientists, software engineers, analysts, and business stakeholders to deliver cloud-based data solutions that support operational goals, innovation, and emerging AI capabilities.

What You Will Do:

• Design, develop, and maintain scalable data pipelines, data models, and lakehouse or data warehouse structures to support analytics, reporting, and AI/ML workloads.
• Build, test, and optimize ETL/ELT workflows that ingest, transform, and deliver data from multiple structured, semi-structured, and unstructured sources.
• Support AI/ML initiatives by preparing and managing datasets for feature engineering, model training, inference, and other AI-enabled applications.
• Implement and maintain data quality controls, validation processes, monitoring, and observability practices to improve reliability, accuracy, and performance of data pipelines.
• Manage data integration processes across internal and external systems to ensure timely, secure, and efficient data availability.
• Support data governance practices, including lineage, cataloging, access management, and documentation, to promote trusted and well-controlled data assets.
• Develop and deploy cloud-based data engineering solutions using modern platforms, distributed processing tools, and containerized environments where applicable.
• Build and maintain batch and near real-time data pipelines to meet business, reporting, and application requirements.
• Collaborate with cross-functional teams to gather requirements, translate business needs into technical solutions, and support enterprise data initiatives.
• Troubleshoot, enhance, and continuously improve data infrastructure, workflows, and related tools to increase scalability, efficiency, and business value.
• Remain current with developments in data engineering, cloud platforms, and AI-enabling technologies and apply practical improvements where beneficial.
• Comply with applicable ABS Health, Safety, Quality, and Environmental Management System requirements and other internal policies and procedures.

What You Will Need:

Education and Experience

• Degree in a technical field or equivalent combination of education and experience
• 8+ years of relevant experience in data engineering or a closely related discipline, including significant experience building and maintaining cloud-based data pipelines and architectures.
• Databricks certification (preferred)

Knowledge, Skills, and Abilities

• Strong knowledge of data engineering concepts, including data modeling, data integration, ETL/ELT design, and pipeline orchestration.
• Strong programming skills in Python and SQL, with the ability to build, test, and maintain production-grade data workflows.
• Experience working with cloud-based data platforms such as AWS, Azure, or Google Cloud.
• Experience designing, implementing, and supporting data warehouse or lakehouse architectures such as Databricks, Snowflake, Redshift, or similar technologies.
• Knowledge of distributed data processing frameworks and tools, such as Spark or equivalent technologies.
• Ability to work with structured, semi-structured, and unstructured data from multiple data sources.
• Experience implementing data quality checks, validation frameworks, monitoring, and performance optimization practices.
• Knowledge of data governance, lineage, metadata management, and cataloging concepts and tools.
• Familiarity with data infrastructure requirements that support AI/ML workflows, including feature engineering, data preparation, and embedding pipelines.
• Experience with real-time or streaming data technologies such as Kafka, Kinesis, or similar tools is preferred.
• Familiarity with containerization and deployment tools such as Docker, Kubernetes, or similar technologies is preferred.
• Ability to analyze technical requirements, solve complex data problems, and deliver practical solutions in a fast-paced environment.
• Strong collaboration and communication skills, with the ability to work effectively across technical and business teams.
• Ability to manage multiple priorities, meet deadlines, and maintain a high standard of quality and accuracy.
• Working knowledge of the ABS Health, Safety, Quality, and Environmental Management System.

Reporting Relationships:

Reports directly to the Chief Data Scientist in Global Technology or other management or executive level position.

Working Conditions

Work is primarily sedentary; exerting up to 10 pounds of force occasionally and/or a negligible amount of force frequently or constantly to lift, carry, push, pull or otherwise move object.