Omm2971J

Omm IT Solutions; McLean (VA); On-site; USD 120,000 - 180,000; Full time

Job summary

Omm IT Solutions is seeking a Senior Data Engineer to design and scale our cloud data platform. This on-site role in Mc Lean, VA focuses on PySpark pipelines, Snowflake data models, and AWS-based workloads.

The ideal candidate brings 6+ years of enterprise data engineering experience, strong SQL and Python skills, and hands-on work with Snowflake, Iceberg, and dynamic tables to enable near real-time analytics. Collaboration with cross-functional teams is essential.

Qualifications

• 6+ years of hands-on data engineering experience in large-scale enterprise environments.
• Deep proficiency in building distributed data processing pipelines with PySpark and handling S3 Parquet/Delta files.
• Strong SQL & Python skills for automation, scripting, and orchestration utilities.
• Experience with Snowflake, Iceberg tables, Dynamic Tables, and Materialized Views.

Responsibilities

• Design and build scalable batch and streaming data pipelines using PySpark for large-scale data processing.
• Migrate legacy ETL workloads to PySpark and Snowflake cloud pipelines.
• Write production-grade PySpark code to read from S3, perform transformations, and process massive datasets.
• Design deduplication strategies for high-volume datasets using PySpark.
• Build and manage Snowflake warehouses, schemas, and data models for analytics.

Skills

Advanced SQL & Python

PySpark expertise

Tools

PySpark

Snowflake

AWS

SnowSQL

Snowpipe

Iceberg tables

Dynamic Tables

Materialized Views

Airflow

Job description

Omm IT Solutions | Permanent W-2 Employee / Corp2Corp Contractor

Mc Lean, United States

• Job Type Permanent W-2 Employee / Corp2Corp Contractor
• Compensation Negotiable
• Country United States
• Work Authorization USC/GC/H1B/H4-EAD

Job Description

PLEASE NOTE:

• IT IS 100 % ON SITE POSITION IN Mc lean VA

KEY REQUIRED SKILLS:

DESCRIPITION:

We are seeking a hands-on, delivery-focused Senior Data Engineer to help build and scale our cloud data platform. In this role, you will design and develop modern data pipelines using PySpark, Snowflake, and AWS to optimize cloud data workloads. The ideal candidate combines strong engineering fundamentals with cloud-native data expertise and is capable of translating complex business needs into robust, performant, and well-documented data solutions. Experience within Fannie Mae, Freddie Mac, or equivalent GSE/mortgage enterprise environments is highly valued.

RESPONSIBILITES:

• Scalable Architecture: Design and build scalable batch and streaming data pipelines using PySpark for large-scale data processing.
• Modernization: Migrate legacy, on-premises ETL workloads (e.g., IBM DataStage, Informatica) to high-performing PySpark and Snowflake cloud pipelines.
• Data Transformation: Write production-grade PySpark code to read from Amazon S3 (Parquet/Delta files), execute complex transformations, and process massive datasets efficiently.
• Deduplication: Design and implement robust deduplication strategies for high-volume datasets using PySpark. Platform Engineering: Build and manage Snowflake warehouses, schemas, and data models optimized for enterprise analytics and business intelligence reporting.
• Iceberg Tables: Design and implement Apache Iceberg tables in Snowflake to support open lakehouse architectures and data interoperability. Incremental Processing: Build and maintain Snowflake Dynamic Tables and Materialized Views to enable near real-time analytics and query acceleration.
• PySpark Tuning: Optimize distributed Spark jobs by leveraging partitioning, caching, broadcast joins, and shuffle optimization. Snowflake Optimization: Tune Snowflake workloads using clustering keys, micro-partition pruning, query profiling, precise warehouse sizing, and strategic result caching.
• Cost Management: Continuously monitor and optimize Spark jobs, Snowflake queries, and AWS infrastructure to balance speed and cloud expenditure.
• Data Quality: Implement data validation, lineage tracking, and monitoring solutions across all pipeline stages to ensure high data integrity. Cross-Functional Collaboration: Partner closely with data architects, business analysts, Technical Program Managers (TPMs), and corporate stakeholders to deliver dependable data products.
• Technical Documentation: Author comprehensive technical designs, data schemas, and operational runbooks to ensure every pipeline is maintainable and audit-ready.

Requirements

REQUIRED QUALIFICATION:

• Experience:6+ years of hands-on data engineering experience in large-scale enterprise environments.
• PySpark Expertise:Deep proficiency in building distributed data processing pipelines, handling S3 Parquet/Delta files, and implementing complex transformations and deduplication logic.
• Snowflake Proficiency:Strong hands-on experience with SnowSQL, Snowpipe, Streams, Tasks, and Role-Based Access Control (RBAC). Proven track record establishing Iceberg tables, Dynamic Tables, and Materialized Views.
• AWS Cloud Ecosystem:Robust working knowledge of AWS services, including S3, Glue, EMR, Lambda, IAM, Step Functions, CloudWatch, and Redshift.
• Advanced SQL & Python:Mastery of advanced SQL techniques (window functions, CTEs, complex joins) alongside strong Python programming skills for automation, scripting, and orchestration utilities.
• Orchestration & Architecture:Solid understanding of data warehousing, ELT/ETL patterns, data lakes, and lakehouse architectures using tools like Airflow or AWS Step Functions.
• Communication:Strong verbal and written communication skills with the ability to articulate technical decisions clearly to both technical peers and business leaders.

PREFERRED QUALIFICATION:

• Industry Experience:Prior experience working within heavily regulated environments such as financial services, mortgage banking, or GSE programs (Fannie Mae / Freddie Mac).
• ETL Migration:Hands-on experience with legacy ETL frameworks (e.g., IBM DataStage) to support modernization initiatives.
• DevOps & CI/CD:Familiarity with continuous integration and continuous deployment pipelines for data infrastructure (Git, Jenkins, GitHub Actions, Terraform).
• Data Quality Frameworks:Exposure to automated data quality and validation frameworks (e.g., Great Expectations, dbt testing suites).
• Streaming Analytics:Knowledge of real-time streaming platforms like Apache Kafka or AWS Kinesis.
• Professional Certifications:AWS Certified Data Analytics, AWS Certified Solutions Architect, or SnowPro Core/Advanced certifications.

Education/Work Experience

Experience:6+ years of hands-on data engineering experience in large-scale enterprise environments.