Title : Senior Big Data Engineer (Kubernetes / AWS / Spark)

Location: Rockville, MD or Mclean, VA or NYC Metro area (Hybrid – 3 days onsite per week)

Duration : 6+ month with extension for long-term

Interview: F2F

Only Local candidates who can take Assessment and only who are in DC/VA/MD/NY/NJ who can got for F2F interview (Rope.AI Assessment)

Role Overview

• This role focuses on building and optimizing large-scale data platforms, with a strong emphasis on Kubernetes-based infrastructure and big data processing in cloud environments.
• Design, develop, and optimize data pipelines handling terabyte-scale datasets
• Work with complex algorithms to process and analyze large volumes of data efficiently
• Optimize code performance for scalability and high-throughput systems
• Build and maintain containerized, serverless data platforms using Kubernetes
• Support migration efforts from EMR/EC2-based systems to Kubernetes-based architecture
• Maintain and enhance existing Kubernetes infrastructure
• Develop and manage CI/CD pipelines using tools like GitLab and Bitbucket
• Collaborate on requirements documentation, system design, and implementation
• Contribute to emerging initiatives involving:
• GenAI integration
• AI agents and automation frameworks
• Technologies such as Kiro (Amazon GenAI) and MCP (Model Context Protocol)

Key Responsibilities

• Design, develop, and optimize data pipelines handling terabyte-scale datasets
• Work with complex algorithms to process and analyze large volumes of data efficiently
• Optimize code performance for scalability and high-throughput systems
• Build and maintain containerized, serverless data platforms using Kubernetes
• Support migration efforts from EMR/EC2-based systems to Kubernetes-based architecture
• Maintain and enhance existing Kubernetes infrastructure
• Develop and manage CI/CD pipelines using tools like GitLab and Bitbucket
• Collaborate on requirements documentation, system design, and implementation
• Contribute to emerging initiatives involving:
• GenAI integration
• AI agents and automation frameworks
• Technologies such as Kiro (Amazon GenAI) and MCP (Model Context Protocol)