Job Title: Software Development Engineer

Location: Bay Area, CA (onsite day 1)

Duration: Long Term

We have 3 open positions for Software Development Engineer

• Looking for Scala with spark and strong exp on fullstack(Java)
• Looking for Scala with spark combination of 50% Python and 50% Java
• Looking for Scala with spark and strong on Python

Description:

• Java
• Scala
• Python
• Spark, Hadoop, or Other Big Data Technologies
• Machine Learning and Deep Learning Technology
• Developing large-scale Machine Learning infrastructure, such as services, frameworks or tooling
• Object-oriented software design skills, data structures and algorithms

Responsibilities:

• Lead the development and optimization of data processing systems using Spark in Scala.
• Oversee the design and implementation of scalable and efficient data pipelines.
• Provide technical expertise in Spark to ensure high performance and reliability.
• Collaborate with cross-functional teams to integrate data solutions with existing systems.
• Ensure code quality and maintainability through code reviews and best practices.
• Develop and maintain documentation for data processing systems and workflows.
• Troubleshoot and resolve issues related to data processing and performance.
• Stay updated with the latest industry trends and technologies to drive innovation.
• Mentor and guide junior developers to enhance their technical skills.
• Participate in project planning and provide accurate time estimates for development tasks.
• Contribute to the continuous improvement of development processes and methodologies.
• Work closely with stakeholders to understand and meet their data processing needs.
• Ensure compliance with data security and privacy regulations in all development activities.

Minimum Qualifications:

• 10+ years hands-on experience in designing, developing, and deploying data processing applications and frameworks.
• Expertise at building jobs with Apache Spark using Scala.
• Expertise at building micro-services using Kubernetes or similar experience.
• Proficient in Java/Python with understanding of recent language features and frameworks.
• In-depth knowledge of SQL databases. Experience with distributed and cloud storage systems such as S3, HDFS, etc.
• An ability to communicate with users and drive adoption, to troubleshoot and diagnose issues, and to work effectively with teams adopting our tools.
• Ability to write and deploy well-tested production code. A drive for modern software engineering practices.