Data Engineer

Valiant Technologies LLC; Tampa (FL); On-site; USD 89,000 - 126,000; Full time

Job summary

Valiant Technologies LLC is seeking a senior Data Engineer in Tampa, FL to architect and build scalable data pipelines on the Databricks lakehouse platform. You will leverage Spark, Delta Lake and SQL to model data, design secure multi-account AWS solutions, and plan migrations from legacy Hadoop/Hive systems.

Responsibilities include choosing batch vs. streaming, ensuring disaster recovery and high availability, and producing clear architecture documentation for stakeholders.

Qualifications

• 12+ years in data engineering or architecture with 5+ years on Databricks lakehouse.
• Strong Spark, Delta Lake and SQL skills with data modeling knowledge.
• AWS networking, IAM, encryption and multi-account design.
• Secure design for PII/NPI with Unity Catalog access models and lineage.
• Migration planning from Hadoop/Hive/warehouse environments.
• Ability to design for both batch and streaming (Kafka, Structured Streaming).
• Disaster recovery and high-availability design know-how.
• FinOps and Databricks cost and capacity modeling with business case.

Responsibilities

• Lead data engineering architecture and implementation on Databricks lakehouse.
• Design secure, scalable data pipelines with batch and streaming support.
• Model data using medallion architecture and ensure data quality.
• Collaborate with stakeholders; manage vendors and document architecture.

Skills

Databricks lakehouse

Apache Spark

SQL

Data modeling (medallion)

AWS architecture

Security design (PII/NPI)

Streaming (Kafka)

Disaster recovery

Cost modeling (FinOps)

Stakeholder communication

Tools

Databricks lakehouse

Spark

SQL

Job description

US Citizen, Green Card, H-1B, OPT-EAD, CPT, Other

Job Title: Data Engineer
Location: Tampa, FL
Duration: 12 Months + Extension
Bill Rate: $78/hour
Job Type: W-2, 1099, C2C
Client: To Be Discussed Later
Work Authorization: US-Citizen, H-1B, OPT-EAD, GC-EADJob Description :

Must-Have Skills (candidate must meet all)

• Experience:12+ years in data engineering or architecture, with 5+ years on Databricks lakehouse platform.
• Data platform fundamentals:Strong Spark, Delta Lake and SQL, plus data modeling (medallion, dimensional).
• AWS architecture:Networking, IAM, encryption and multi-account design on AWS.
• Secure design:Platform design for PII/NPI, including Unity Catalog access models and lineage.
• Migration:Migration planning from legacy Hadoop/Hive/warehouse systems.
• Streaming and batch design:Ability to choose between batch and streaming (Kafka, Structured Streaming) and design for both.
• Resilience:Disaster recovery and high-availability design.
• Cost modeling:FinOps and Databricks cost and capacity modeling, including a business case.

Communication:Stakeholder management, vendor management and clear architecture documentation

We are an equal opportunity employer. All aspects of employment including the decision to hire, promote, discipline, or discharge, will be based on merit, competence, performance, and business needs. We do not discriminate on the basis of race, color, religion, marital status, age, national origin, ancestry, physical or mental disability, medical condition, pregnancy, genetic information, gender, sexual orientation, gender identity or expression, national origin, citizenship/ immigration status, veteran status, or any other status protected under federal, state, or local law.