Senior Big Data Engineer: ETL/Python/SQL Expert

Omniscius Consulting; Washington (District of Columbia); On-site; USD 120,000 - 180,000; Full time

Job summary

Omniscius Consulting is seeking an experienced Big Data Engineer to design, develop, and support large-scale data pipelines and transformations. The ideal candidate will bring strong SQL, Python, ETL/ELT skills and the ability to communicate complex concepts to clients and cross-functional teams.

Java experience is helpful but not central; responsibilities include building ETL pipelines, writing SQL, Python data transforms, XML parsing, Linux scripting, and collaborating with security,

Qualifications

• Bachelor’s degree or equivalent experience.
• 5+ years in data engineering, ETL/ELT, or related data integration.
• Strong hands-on SQL with complex queries, performance tuning, and data modeling.
• Experience designing, building, optimizing, and supporting large-scale data pipelines.
• Strong Python for data transformation, integration, automation, or pipeline operations.
• Experience parsing, transforming, and validating XML data.
• Experience in Linux and shell environments, including scripting and cron.
• Experience implementing data validation, error handling, monitoring, and production troubleshooting.
• Excellent written and verbal communication skills and ability to explain technical concepts to clients and stakeholders.
• U.S. citizenship and ability to undergo a background check.

Responsibilities

• Design, build, maintain, and optimize ETL/ELT pipelines for large-scale data.
• Write and optimize complex SQL queries, stored procedures, data models, and validation processes.
• Develop data transformation, integration, and automation using Python.
• Use Java as needed to support existing applications and integrations.
• Parse, transform, and validate XML-based data formats.
• Work in Linux and shell environments for scripting, cron jobs, and troubleshooting.
• Implement data validation, error handling, monitoring, and traceability controls.
• Troubleshoot pipeline failures and data discrepancies across environments.
• Collaborate with client, security, infrastructure, data engineering, and delivery teams.
• Communicate issues and risks clearly to technical and nontechnical stakeholders.
• Participate in Agile planning, status reporting, documentation, and cross-team coordination.

Skills

SQL

Python

ETL/ELT

Data pipelines

Communication

Linux

Shell scripting

Agile

Problem solving

Education

Bachelor’s degree or equivalent

Relevant professional experience

Tools

AWS

S3

Glue

Lambda

Redshift

Oracle

PostgreSQL

Power BI

MicroStrategy

Job description

Omniscius Consulting is seeking an experienced Big Data Engineer to design, develop, and support large-scale data pipelines and transformations. The ideal candidate will bring strong SQL, Python, ETL/ELT skills and the ability to communicate complex concepts to clients and cross-functional teams.

Java experience is helpful but not central; responsibilities include building ETL pipelines, writing SQL, Python data transforms, XML parsing, Linux scripting, and collaborating with security,