State Street
Bengaluru, Karnataka, India Posted 2d ago
State Street is seeking a seasoned Databricks Data Engineer to work extensively on Data Engineering and implementation at large scale across Azure and AWS clouds.
Key Responsibilities
Build and maintain data engineering solutions using Databricks platform and Spark optimization techniques.
Configure Unity Catalog and RBAC.
Implement programming and data processing solutions using Scala, Python, PySpark, and SQL.
Utilize Azure Services including Data Factory and Blob Storage.
Implement CI/CD pipelines with Azure DevOps.
Requirements
Extensive experience on Databricks Platform Expertise (Notebooks, Clusters, Jobs, Delta Lake).
Unity Catalog and RBAC configuration experience.
Proficiency in Programming & Data Processing using Scala, Python, PySpark, and SQL.
Experience with Azure Cloud, Azure Services, and CI/CD with Azure DevOps and Harness.
Preferred Qualifications
Experience in AWS using Docker, Kubernetes, and Helm framework.
