Data Engineer

Integrated Resources, Inc ( IRI ); Richmond (VA); Presencial; USD 68 000 - 97 000; Tempo integral

Resumo da oferta

Integrated Resources, Inc (IRI) is seeking an experienced Data Engineer in the Richmond, VA area. The role focuses on designing, building, and optimizing scalable data pipelines, including ELT/ETL, data warehousing, and real-time streaming solutions.

Strong SQL, Python, and scripting skills are essential. The candidate will work hands-on while guiding architecture and data governance, ensuring secure, observable, and high-performance data flows within enterprise environments.

Qualificações

• Must have 5 to 7+ years of Data Engineering experience.
• Strong experience with Oracle Exadata, Snowflake or similar cloud/on prem data warehouses.
• Hands on experience with enterprise ETL/ELT platforms (e.g., Talend, dbt, Informatica).
• Deep understanding of data warehousing architecture and dimensional modeling.
• Experience designing and supporting large scale, production data pipelines.
• Strong scripting experience (Python, shell).
• Experience with DataOps practices, CI/CD, and observability.

Responsabilidades

• Build end-to-end data pipelines and ETL/ELT solutions for analytics, reporting, and AI/ML use cases.
• Develop scalable batch and streaming data workflows with hands-on coding and troubleshooting.
• Optimize data transformations and performance across warehouses and pipelines.
• Prototype and integrate tooling to meet project requirements.
• Lead performance tuning efforts for pipelines and data stores.
• Ensure pipelines are observable, error-tolerant, and production-ready.

Conhecimentos

Advanced SQL

ELT patterns

Python

Shell scripting

Data warehousing concepts

Performance tuning

Cloud data platforms

Formação académica

Bachelor's degree or higher

Ferramentas

Oracle Exadata

Snowflake

Talend

dbt

Informatica

Denodo

Composite

Dremio

Starburst

Kafka

Kinesis

Spark Streaming

Descrição da oferta de emprego

Job Title: Data Engineer (Candidate should be residing in VA)

Job Duration: 12+ Months (Possibility of extension)

Pay rate: $60/hr. on w2

Note: Candidate should willing to work on w2

Job Description:

Job Summary:

• The Senior Data Engineer is a hands-on expert and technical leader, actively engaged in designing, building, and optimizing scalable, reliable data pipelines at an enterprise level. This role not only guides architectural decisions but also directly implements advanced ELT solutions, troubleshoots complex data challenges, and ensures best practices through practical, high-impact contributions.
• This role combines deep hands-on expertise with technical ownership, mentoring, and architectural alignment. The Senior Data Engineer drives and implements data engineering best practices, ensures high standards for quality and security, and partners with architecture

Key Responsibilities:

• Build end-to-end data pipelines and ETL/ELT solutions to support analytics, reporting, and AI/ML use cases, ensuring solutions are robust and production-ready through practical implementation.
• Apply scalable patterns for batch and incremental processing by developing, testing, and deploying data workflows, focusing on hands-on coding and troubleshooting.
• Review and implement data modeling, transformation logic, and performance strategies, using deep technical expertise to optimize and validate solutions.
• Evaluate, select, and integrate tooling, frameworks, and platform capabilities by actively prototyping and configuring systems to meet project requirements.
• Build up complex, high-volume data pipelines using SQL-centric ETL/ELT patterns.
• Design and implement scalable streaming pipelines to process real-time data, ensuring low latency and reliable delivery for analytics and operational use cases.
• Lead performance tuning efforts across pipelines, warehouses, and workloads.
• Ensure data pipelines are resilient, observable, and production ready.
• Implement enterprise-grade error handling, restart ability, and monitoring.
• Build and maintain scalable, low-latency streaming data pipelines using technologies such as Kafka, Kinesis, or Spark Streaming
• Perform on-the-fly data cleaning, validation, and enrichment before data reaches its final destination.
• Uses strategies such as Indexing and partitioning to fine tune the data warehouse and big data environments to improve the query response time and scalability
• Implement standards for data quality checks, validation, and reconciliation.
• Ensure pipelines meet security, access control, and governance requirements.
• Partner with governance & DataOps teams on metadata, lineage, and auditability.
• Apply consistent naming conventions, documentation, and coding standards.
• Improve operational monitoring, alerting, and incident response processes.
• Proactively identify reliability, performance, and cost optimization opportunities.
• Support and guide production troubleshooting and root cause analysis.
• Investigating data quality incidents and identifying design/coding GAPs
• Participate in design and code reviews to enforce quality and best practices.
• Partner with various infrastructure teams, application teams, and architects to generate process designs and complex transformations to various data elements to provide the Business with insights into their business processes.
• Translate ambiguous requirements into well designed technical solutions.
• Work in complex multi-platform environments on multiple project assignments.

Required Skills & Experience:

• Advanced expertise in SQL, ELT patterns, and performance tuning.
• Strong experience with Oracle Exadata, Snowflake or similar cloud/on prem data warehouses.
• Hands on experience with enterprise ETL/ELT platforms (e.g., Talend, dbt, Informatica).
• Deep understanding of data warehousing architecture and dimensional modeling.
• Experience designing and supporting large scale, production data pipelines.
• Strong scripting experience (Python, shell).
• Experience with data virtualization tools (e.g., Denodo, Composite, dremio, Starburst).
• Experience with DataOps practices, CI/CD, and observability.
• Required 5 to 7+ years of Data Engineering experience.
• ETL Development and Process Support, may require weekend/off business hours work.

Key Behavioural Expectations ("How We Work"):

• Delivering at Pace: Acts with urgency to meet deadlines while maintaining quality, prioritizing workload effectively under pressure.
• Collaborative Teamwork: Actively supports colleagues, shares knowledge freely, and treats others with dignity and respect.
• Effective Communication: Tailors communication style to the audience, ensuring clarity and transparency in both successes and challenges.
• Ownership and Adaptability: Takes responsibility for outcomes and welcomes feedback for improvement.
• Ability to work independently
• Achievement orientation
• Flexibility

Preferred / Nice to Have:

• Experience supporting AI/ML or advanced analytics pipelines.
• Cloud platform experience (AWS, Azure, or GCP).
• Prior experience influencing enterprise data standards or reference architecture.
• Experience optimizing cost and performance in cloud data warehouses.
• Hands-on experience with Cribl, Apache Kafka, Kafka Connect, Spark Streaming, or Apache Flink

Required Years of Experience:

• MUST have 5 to 7+ years of Data Engineering experience.

Education:

• Education: Bachelors or higher required

Are there any specific companies/industries you’d like to see in the candidate’s experience?

• High Preference for candidates that have previously worked with a large-scale commercial utilities team but will review candidates who have a background with large scale capital projects for companies

Preferred Interview Process Overview (High level):