Job Title: – Azure Databricks Lead

Location: - Charlotte, NC (Primary) - Short Hills, NJ (Secondary) – Day 1 Onsite and Hybrid

Job Summary

The Databricks Lead will drive the design, development, and optimization of data engineering solutions on Azure. This role requires deep expertise in Databricks, PySpark, Python, and Azure Data Lake, along with strong ownership and the ability to deliver independently. The lead will guide technical direction, ensure platform excellence, and support enterprise analytics initiatives.

Key Skills

• Databricks (SQL, Notebooks, Workflows), PySpark, Python
• Azure cloud stack (ADLS, ADF, DevOps)
• Strong problem-solving, accountability, and independent execution
• Excellent communication and stakeholder engagement

Responsibilities

• Lead architecture and development of scalable data solutions using Databricks and Azure Data Lake.
• Build and optimize PySpark pipelines and Databricks SQL workloads.
• Drive best practices for cluster management, CI/CD, governance, and performance tuning.
• Collaborate with cross-functional teams to translate business needs into technical solutions.
• Ensure security, compliance, and operational reliability across data platforms.
• Mentor junior engineers and promote engineering excellence.
• Document architecture, patterns, and operational processes.
• Onsite / offshore co-ordination.