Data Engineer

Enterprise Engineering Inc. (EEI); St. Louis (MO); Hybrid; USD 120,000 - 150,000; Full time

Job summary

Enterprise Engineering Inc. (EEI) in St. Louis is seeking an experienced Data Engineer to design, build, and optimize data pipelines across on-prem and cloud environments.

The role emphasizes scalable, secure solutions using Spark, Java/Scala, and Hadoop ecosystems. This 12-month hybrid position involves collaboration with architects, analysts, and product owners, mentoring juniors, and leading end-to-end development from requirements through production support.

Qualifications

• 8+ years delivering enterprise data solutions.
• Experience migrating workloads to cloud.
• Strong SQL and data modeling skills.
• Proficient in Spark, Hadoop ecosystem.
• Experience with Agile delivery.

Responsibilities

• Design, test, and deploy scalable data pipelines using Spark, Java/Scala.
• Build batch and real-time processing for petabyte-scale workloads.
• Develop reusable components and enterprise-quality frameworks.
• Migrate ETL workloads from on-prem to cloud-native.
• Implement data lineage, cataloguing, governance and observability.
• Lead end-to-end development incl. requirement analysis, deployment, and support.
• Mentor junior engineers and conduct code reviews.
• Collaborate with architects, product owners, and stakeholders.
• Triage production incidents and perform RCA.
• Adhere to engineering, security, QA standards.

Skills

Spark

Java/Scala

Hadoop

Airflow/NiFi

AWS

SQL/NoSQL

Data pipelines

Tools

S3

EMR

Glue

Cloud Storage

Databricks

Job description

Contract & Work Arrangement

12 Months (Hybrid role 3 days a week)

Location & Relocation

St. Louis, MO. Non-local candidate is fine but 100% open to relocate

Key Responsibilities

• Design, develop, test, and deploy high-quality, secure, scalable, and resilient data pipelines using Apache Spark, Java/Scala across Hadoop and cloud-native object storage platforms.


• Build and maintain batch and near real-time data processing frameworks capable of supporting petabyte-scale workloads.


• Develop reusable engineering components and frameworks that accelerate data product delivery while maintaining enterprise standards.


• Design and implement a \"build once, run anywhere\" architecture supporting seamless deployment across on-premises and public cloud environments without code changes.


• Implement data lineage, metadata management, data cataloguing, data quality controls, and observability capabilities across the data ecosystem.


• Collaborate with architects and platform teams to establish scalable design patterns and engineering best practices.


• Contribute to migration initiatives moving legacy ETL and data warehouse workloads from on-premises environments to cloud-native architectures.


• Leverage cloud services such as Amazon S3, EMR, Glue, and related data services to improve scalability, reliability, and operational efficiency.


• Drive adoption of modern Lakehouse and distributed compute architectures.


• Lead end-to-end development activities including requirement analysis, solution design, coding, testing, deployment, and production support.


• Mentor and guide junior engineers through code reviews, technical coaching, and engineering best practices.


• Partner with product owners, analysts, architects, and business stakeholders to deliver high-quality solutions within committed timelines.


• Troubleshoot complex production incidents and perform root cause analysis to identify and implement long-term remediation strategies.


• Ensure compliance with Client's engineering, security, quality assurance, and operational governance standards.


• Continuously identify opportunities to improve performance, automation, monitoring, and process efficiency.


• Evaluate emerging data technologies and conduct proof-of-concept (POC) initiatives to determine their applicability within Client's data ecosystem.


• Contribute to engineering innovation and continuous improvement initiatives across the organisation.

Qualifications

• \" 8-10 yrs. experience years of experience delivering enterprise-scale Data Warehouse, Data Lake, or Data Lakehouse solutions.\"


• \" Proven experience implementing multiple end-to-end data engineering projects within large-scale distributed computing environments.\"


• \" Hands-on experience migrating ETL and analytics workloads from on-premises platforms to cloud-native environments.\"


• \" Strong development experience using: Apache Spark, Scala or Java, Hadoop ecosystem technologies, Object Storage platforms\"


• \" Experience building orchestration and workflow solutions using: Apache Airflow/ Apache NiFi and Similar enterprise scheduling frameworks\"


• \" Strong SQL expertise and experience with Relational and NoSQL database technologies: Oracle/ SQL Server, Cassandra, Dynamo DB etc.\"


• \" Working knowledge of cloud platforms, preferably AWS: Amazon S3, EMR, AWS Glue, Cloud-native data services\"


• \" Strong analytical and problem-solving capabilities.\"


• \" Experience operating within Agile delivery environments.\"


• \" Excellent written and verbal communication skills.\"


• \" Proven ability to collaborate within geographically distributed and matrix-based teams.\"


• \" Self-starter with strong ownership, accountability, and execution focus.\"


• \" Ability to learn emerging technologies quickly and apply them effectively to business challenges.\"