Big Data Engineer - Intermediate- NYC

American Cybersystems, Inc.; New York (NY); On-site; USD 91,000 - 129,000; Full time

Benefits offered by this job

Medical & Pharmacy

Dental & Vision

401(k)

HSA

FSA

Life Insurance

Disability

Job summary

Innova Solutions is seeking a Big Data Engineer - Intermediate in NYC for a contract-to-hire role. You will architect and oversee complex data pipelines, guide data models, and build cloud-native solutions across multiple business functions.

The role emphasizes hands-on AWS-based data pipelines, data quality, and scalable architectures with CI/CD, security, and operational resilience in a large corporate setting.

Qualifications

• Formal training or certification on software engineering concepts and 5+ years applied experience.
• Extensive experience in managing the full lifecycle of data, from collection and storage to analysis and reporting.
• Proficiency in one or more large-scale data processing distributions such as Java or Python Spark along with knowledge on Data Pipeline, Data Modeling, Data warehouse, Data Migration.

Responsibilities

• Architect and oversee the design of complex data solutions meeting diverse business needs.
• Guide evolutions of data models to support new use cases and tech advances.
• Build and manage cloud-native data pipelines in AWS.
• Develop end-to-end analytical systems and translate business issues into requirements.

Skills

Python

Spark

Java

AWS

Data Modeling

CI/CD

Data Quality

Tools

S3

Glue

EMR

Kubernetes

Job description

We're Looking for a Big Data Engineer - Intermediate- NYC Position type: Contract to Hire Duration: 6+ Months

As a core technical contributor, you are responsible for maintaining critical data pipelines and architectures across multiple technical areas within various business functions in support of the firm's business objectives.

Job responsibilities

• Architect and oversee the design of complex data solutions that meet diverse business needs and customer requirements.
• Guide the evolution of logical and physical data models to support emerging business use cases and technological advancements.
• Build and manage end-to-end cloud-native data pipelines in AWS, leveraging your hands-on expertise with AWS components.
• Build analytical systems from the ground up, providing architectural direction, translating business issues into specific requirements, and identifying appropriate data to support solutions.
• Work across the Service Delivery Lifecycle on engineering major/minor enhancements and ongoing maintenance of existing applications.
• Conduct feasibility studies, capacity planning, and process redesign/reengineering of complex integration solutions.
• Help others build code to extract raw data, coach the team on techniques to validate its quality, and apply your deep data knowledge to ensure the correct data is ingested across the pipeline.
• Guide the development of data tools used to transform, manage, and access data, and advise the team on writing and validating code to test the storage and availability of data platforms for resilience.
• Oversee the implementation of performance monitoring protocols across data pipelines, coaching the team on building visualizations and aggregations to monitor pipeline health.
• Coach others on implementing solutions and self-healing processes that minimize points of failure across multiple product features.

Required qualifications, capabilities, and skills

• Formal training or certification on software engineering concepts and 5+ years applied experience.
• Extensive experience in managing the full lifecycle of data, from collection and storage to analysis and reporting.
• Proficiency in one or more large-scale data processing distributions such as Java or Python Spark along with knowledge on Data Pipeline (DPL), Data Modeling, Data warehouse, Data Migration and so-on.
• Hands-on practical experience in system design, application development, testing, and operational stability.
• Experience in developing, debugging, and maintaining code in a large corporate environment with one or more modern programming languages and database querying languages.
• Solid understanding of agile methodologies such as CI/CD, Application Resiliency, and Security.
• Strong organizational, problem-solving, and critical thinking skills; strong documentation skills.
• Proficiency in Python or Java and proficiency in Spark.
• Hands-on experience on AWS services and its components along with good understanding on Kubernetes.
• AWS Data Services: Proficiency in Lake formation, Glue ETL (or) EMR, S3, Glue Catalog, Athena, Kinesis (or) MSK, Airflow (or) Lambda + Step Functions + Event Bridge.
• Data De/Serialization: Expertise in at least 2 of the formats: Parquet, Iceberg, AVRO, JSON‑L.
• AWS Data Security: Good Understanding of security concepts such as: Lake formation, IAM, Service roles, Encryption, KMS, Secrets Manager.

Preferred qualifications, capabilities, and skills

• Familiarity with modern front-end technologies.
• Experience designing and building REST API services using Java.
• Exposure to cloud technologies - knowledge on Hybrid cloud architectures.
• DevOps: Linux Scripting, Jenkins, Git, CI/CD, JIRA, TDD skills.
• Proficient in Big Data application development skills as well as multiple design techniques.
• Working proficiency in Big Data development toolset to design, develop, test, deploy, maintain and improve software.
• Strong understanding of Agile methodologies with ability to work in at least one of the common frameworks.
• Strong understanding of techniques such as Continuous Integration, Continuous Delivery, Test Driven Development, Cloud Development, application resiliency and security.
• Proficiency in one or more general-purpose programming languages.
• Working proficiency in a portion of software engineering disciplines and demonstrates understanding of overall software skills including business analysis, development, testing, deployment, maintenance and improvement of software.

PAY RANGE AND BENEFITS

• Pay Range*: $80/Hr. *Pay range offered to a successful candidate will be based on several factors, including the candidate's education, work experience, work location, specific job duties, certifications, etc.
• Benefits: Innova Solutions offers benefits( based on eligibility) that include the following: Medical & pharmacy coverage, Dental/vision insurance, 401(k), Health saving account (HSA) and Flexible spending account (FSA), Life Insurance, Pet Insurance, Short term and Long term Disability, Accident & Critical illness coverage, Pre-paid legal & ID theft protection, Sick time, and other types of paid leaves (as required by law), Employee Assistance Program (EAP).

ABOUT INNOVA SOLUTIONS

Founded in 1998 and headquartered in Atlanta, Georgia, Innova Solutions employs approximately 50,000 professionals worldwide and reports an annual revenue approaching $3 Billion. Through our global delivery centers across North America, Asia, and Europe, we deliver strategic technology and business transformation solutions to our clients, enabling them to operate as leaders within their fields.

Recent Recognitions:

• Named One of America's Best Employers for New Grads by Forbes (2024)
• Named One of the Best Companies to Work for by U.S. News & World Report (Private Companies List, 2024-2025)
• One of the Largest IT Staffing Firms in the US - Ranked #3 by Staffing Industry Analysts (SIA, 2024)
• One of the Largest Staffing Firms in the US - Ranked #13 by Staffing Industry Analysts (SIA, 2024; includes Innova Solutions, Volt, & HireGenics)
• Named One of the Top Ten Private Companies in Atlanta by the Atlanta Business Chronicle (2023)
• One of the Largest Certified Minority Business Enterprises (MBEs) in the United States (NMSDC, 2024)
• AWS Advanced Tier Services Partner with 100+ certifications

Website: https://www.innovasolutions.com/

Innova Solutions is an Equal Opportunity Employer and prohibits any kind of unlawful discrimination and harassment. Innova Solutions is committed to the principle of equal employment opportunity for all employees and to providing employees with a work environment free of discrimination and harassment on the basis of race, color, religion or belief, national origin, citizenship, social or ethnic origin, sex, age, physical or mental disability, veteran status, marital status, domestic partner status, sexual orientation, or any other status protected by the statutes, rules, and regulations in the locations where it operates.

If you are an individual with a disability and need a reasonable accommodation to assist with your job search or application for employment, please contact us at [email protected] or [phone]. Please indicate the specifics of the assistance needed.

Innova Solutions encourages all interested and qualified candidates to apply for employment opportunities.

Innova Solutions (HireGenics/Volt) does not discriminate against applicants based on citizenship status, immigration status, or national origin, in accordance with 8 U.S.C. § 1324b.

The company will consider for employment qualified applicants with arrest and conviction records in a manner that complies with the San Francisco Fair Chance Ordinance, the Los Angeles Fair Chance Initiative for Hiring Ordinance, and other applicable laws.

By applying for this job, you agree to receive calls, AI-generated calls, text messages, or emails from Innova Solutions and its affiliates, and contracted partners. Frequency varies for text messages. Message and data rates may apply. Carriers are not liable for delayed or undelivered messages. You can reply STOP to cancel and HELP for help. You can access our privacy policy at Privacy Policy - innovasolutions.

Thank You! Team Innova Solutions?