Senior Data Engineer - Spark, Emr & Ai Pipelines (Remote)
Agileengine
Stand out for this role — generate a tailored resume and cover letter in about a minute.
Ventajas ofrecidas por este puesto de trabajo
Descripción de la vacante
Darwoft is seeking a Senior Data Engineer focused on data ingestion, integrations, and collectors to serve enterprise customers. You will design, implement, and optimize ingestion pipelines, connectors, and APIs across AWS, Azure, and GCP environments, working closely with customers to meet security and performance requirements.
You will combine data engineering, software engineering, solution architecture, cloud infrastructure, and consulting to deliver scalable data solutions.
Formación
- 10+ years of professional software/data/ platform engineering or related experience.
- Strong proficiency in Python and Go and REST APIs.
- Experience building data ingestion pipelines and connectors.
- Experience with SQL and NoSQL databases and cloud platforms (AWS/Azure/GCP).
- Docker and Kubernetes proficiency; CI/CD practices.
Responsabilidades
- Design, develop, and maintain production-grade data collectors, connectors, and ingestion services.
- Integrate enterprise data sources including identity providers, cloud platforms, and security systems.
- Collaborate with customers to understand environments and constraints, delivering scalable ingestion solutions.
- Onboard customers, design architectures, and deploy production solutions.
- Optimize ingestion performance, reliability, and security.
Conocimientos
Herramientas
Descripción del empleo
Coderio designs and delivers scalable digital solutions for global companies. We combine strong technical expertise with a product mindset to lead complex software initiatives end-to-end. We work with international clients, value autonomy and clear communication, and build long-term partnerships through technical excellence.
More information:
This is a high-impact technical role. We are looking for a motivated and adaptive DataOps Engineer, Technical Referent to join our fast-paced environment. You will operate at the intersection of data engineering, cloud infrastructure, and DevOps, where a high tolerance for ambiguity and a passion for constant learning are essential. Your primary mission involves interacting with customers to understand their needs, extract requirements, and both design and propose scalable solutions. You will be responsible for tackling the challenge of working within a team to develop solutions, as well as executing hands-on technical work to ensure robust data operations.
Automation & Integration: Develop Python components, Terraform modules, and pipelines to integrate and automate various tools and processes.
API & System Orchestration: Promote the use of APIs from distinct systems to extract and update data, trigger and monitor processes, and help tie infrastructures together.
Cloud Reliability & Operations: Deploy, maintain, and oversee cloud infrastructure to ensure it runs with the performance and reliability our customers expect.
Data Modeling & Documentation: Help create Data Models, technical documents, and best practices for our users.
DataOps Best Practices & Innovation: Develop policies, processes, and best practices regarding DevOps and DataOps. You will also help identify opportunities, generate innovative solutions, and improve existing product features.
Development & Scripting: Strong proficiency in Python and bash, and fluency in SQL as a second language.
Data Ecosystems & Pipelines: Deep familiarity with databases, data warehouses, and data lakes, as well as being adept at data pipelines and process improvement.
Cloud & Tooling Mastery: Strong knowledge of AWS environments, including EC2, Lambda, IAM, SQS, RDS, Kinesis, Glue, and EKS. Must also be familiar with modern development tools like Terraform, GitHub, Docker, and dbt.
Mindset & Work Style: Ability to thrive in an environment with constant and quick iterations, consistently seeking out creative solutions to challenging problems. Must have strong attention to detail.
Autonomy & Collaboration: Ability to work collaboratively while also being capable of owning a project with little guidance. Must be comfortable working in a diverse environment and non-hierarchical organizations.
Customer-Facing Skills: Ability to interact directly with customers to understand needs, extract requirements, and propose technical solutions.
AWS associate or professional-level certifications, such as Solutions Architect, Developer, or DevOps Engineer.
High-impact, strategic role driving modern data infrastructure and automation.
Collaborative international environment with senior technical leadership.
High autonomy to design, propose, and implement cutting-edge DataOps solutions.
Long-term engagement with competitive compensation and growth opportunities.
At Coderio, we value talent and technical excellence. We are passionate about modern engineering, high autonomy, and building solutions that make a real difference. We offer an inclusive and challenging environment where innovation is encouraged and technical leadership is valued. If you are eager to design and deploy next-generation data solutions that elevate engineering teams,
BigQuery
- Location: LATAM
- Work Model: 100% Remote
- Project: Identity Security / Cybersecurity Platform
- Seniority: Senior
- Engagement: Full-time
- Location: LATAM
- Work Model: 100% Remote
- Project: Identity Security / Cybersecurity Platform
- Seniority: Senior
- English Level: B2 / C1
- Engagement: Full-time
At Darwoft, we build custom software solutions and partner with international companies to create high-impact digital products.
We are a Latin American technology company with a people-first culture, strong professional standards, and a long-term partnership mindset. We work with distributed teams, complex technical challenges, and clients whose technology has a direct impact on their businesses and users.
We are partnering with an innovative cybersecurity company building a next-generation Identity Security and Identity Observability platform .
The product helps enterprise security teams understand how human users, service accounts, workloads, applications, and AI-driven identities access systems across cloud, SaaS, network, endpoint, and on-premise environments .
At the core of the platform is a highly data-intensive architecture that collects and analyzes large volumes of events, logs, network telemetry, identity activity, and security signals . This information is processed to identify unusual access patterns, security risks, identity blind spots, infrastructure misconfigurations, and potential threats.
The platform integrates with complex enterprise ecosystems including identity providers, cloud platforms, SASE solutions, SIEMs, endpoint security tools, network infrastructure, and security services.
We're looking for a Senior Data Engineer focused on Data Ingestion, Integrations, and Collectors to become a key technical bridge between enterprise customers and the engineering organization.
This is not a traditional backend or internal data engineering position.
You’ll work directly with customers to understand their infrastructure, data sources, security architecture, network topology, APIs, and workflows, then design and implement scalable mechanisms to ingest that information into the platform.
You’ll combine data engineering, software engineering, solution architecture, cloud infrastructure, and technical consulting .
A major part of your mission will be building and evolving connectors and ingestion pipelines capable of integrating data from complex enterprise environments while maintaining reliability, scalability, security, and fast time-to-production.
- Design, develop, and maintain production-grade data collectors, connectors, and ingestion services .
- Integrate enterprise data sources including identity providers, cloud platforms, SASE solutions, network infrastructure, endpoints, applications, security platforms, and SIEM systems .
- Work directly with enterprise customers to understand their environments, data flows, APIs, security requirements, and integration constraints.
- Participate in technical onboarding processes, from initial discovery and architecture design through production deployment.
- Design scalable ingestion architectures for processing security events, logs, identity activity, network telemetry, and other high-volume data sources .
- Build integrations using REST APIs, cloud-native services, event-driven architectures, and modern software engineering practices .
- Develop backend services primarily using Python and/or Go .
- Work with SQL and NoSQL databases to store, transform, query, and analyze high-volume data.
- Build and operate containerized services using Docker and Kubernetes .
- Deploy and integrate solutions across AWS, Azure, and GCP environments .
- Troubleshoot complex customer deployment and integration issues across application, infrastructure, networking, API, and data layers.
- Optimize ingestion performance, reliability, scalability, and operational efficiency.
- Collaborate with Solution Architects, Sales Engineering, Customer Success, Product, and Engineering teams.
- Act as a trusted technical advisor around customer data architecture, collection strategies, security integrations, and implementation best practices.
- Identify recurring customer requirements and transform them into reusable connectors, platform capabilities, or improvements to the core product.
- Contribute technical insights from real-world implementations to the product roadmap.
- Support production adoption and continuously improve time-to-value for enterprise customers.
- Participate in occasional customer-facing technical sessions and, when necessary, onsite engagements.
- 10+ years of professional Software Engineering, Data Engineering, Platform Engineering, or related experience.
- Strong software development experience with:
- Python
- Go
- Strong experience designing and consuming REST APIs .
- Hands-on experience building data ingestion pipelines, connector frameworks, integration platforms, or similar systems .
- Experience integrating complex enterprise systems and third-party platforms.
- Solid understanding of both SQL and NoSQL databases .
- Production experience with at least one major cloud platform:
- AWS
- Azure
- GCP
- Experience working with Docker and Kubernetes .
- Strong understanding of Git-based development workflows and CI/CD practices .
- Experience designing reliable, scalable, and production-ready distributed systems.
- Strong debugging skills across APIs, infrastructure, networking, application services, and data pipelines.
- Experience working directly with customers, stakeholders, or technical partners.
- Ability to translate complex customer requirements into pragmatic technical solutions.
- Strong communication skills and the ability to operate effectively in customer-facing technical conversations.
- High degree of autonomy and ownership when solving ambiguous technical problems.
Experience with any of the following will be especially valuable:
- Identity and Access Management (IAM )
- Identity Providers (IdPs )
- Identity Security
- Cybersecurity platforms
- SIEM systems
- Security telemetry
- Network security
- Endpoint security
- SASE / Zero Trust environments
- Cloud security
- Large-scale event processing
- Streaming architectures
Previous experience integrating APIs or data from platforms such as the following is highly valuable:
- AWS
- Microsoft Azure
- Google Cloud
- Microsoft Entra ID / Active Directory
- Okta
- Google Workspace
- Duo
- CrowdStrike
- Zscaler
- Palo Alto
- Microsoft Defender
- SIEM and security analytics platforms
Direct experience with every platform is not required. What matters is a strong track record of learning complex APIs and transforming heterogeneous enterprise data sources into reliable integrations.
- Experience with large-scale data processing or streaming platforms.
- Experience designing reusable connector SDKs or integration frameworks .
- Understanding of identity-related security concepts such as authentication, authorization, credentials, service accounts, privileged access, and non-human identities.
- Experience with security events, audit logs, network telemetry, or cloud activity data.
- Familiarity with event-driven and distributed system architectures.
- Experience deploying AI/ML-enabled applications.
- Exposure to LLMs, AI agents, or AI-powered security applications .
- Solution Architecture or Forward Deployed Engineering experience.
- Previous experience in cybersecurity startups or high-growth SaaS environments.
Success in this role will be measured by your ability to:
- Deliver successful enterprise integrations and deployments.
- Reduce customer time-to-production.
- Build reliable and scalable ingestion solutions.
- Improve customer adoption and technical satisfaction.
- Resolve complex integration challenges efficiently.
- Turn customer-specific requirements into reusable platform capabilities.
- Contribute improvements that strengthen the scalability and flexibility of the core product.
- Contractor agreement with payment in USD
- 100% remote work
- Argentina's public holidays
- English classes
- Referral programAccess to learning platforms
'/%3e%3cpath%20d='M101.744%2012.4726L102.894%2016.1947L99.0021%2013.2201L101.744%2012.4726ZM105.3%208.40937H100.488L99.0026%203.59473L97.5116%208.41047L92.7%208.40385L96.5965%2011.3834L95.1055%2016.1942L99.0021%2013.2196L101.408%2011.3834L105.3%208.40992V8.40937Z'%20fill='white'/%3e%3cdefs%3e%3clinearGradient%20id='paint0_linear_5593_1381'%20x1='90'%20y1='9.44471'%20x2='99'%20y2='9.44471'%20gradientUnits='userSpaceOnUse'%3e%3cstop%20stop-color='%2300AD70'/%3e%3cstop%20offset='0.998594'%20stop-color='%2300B67A'/%3e%3cstop%20offset='1'%20stop-color='%23DCDCE6'/%3e%3c/linearGradient%3e%3c/defs%3e%3c/svg%3e){kind=small}
