Mastercard
Pune, Maharashtra, India Posted 2d ago
Mastercard is seeking a Lead Data Engineer to build and evolve large scale backend data systems, real time and batch pipelines, and AI enabled services that power analytics and automation.
Key Responsibilities
Design, develop, and maintain backend services and data pipelines using Scala, Python and Java.
Build and optimize batch and streaming workloads on Cloudera Data Platform using Spark.
Work with Cloudera Manager to support platform configuration, monitoring, performance tuning, and operational stability.
Design, build, and integrate AI powered agents that support anomaly detection and workflow automation.
Build and support data and compute workloads in AWS environments and leverage Databricks for advanced analytics.
Requirements
BS/BA degree in Computer Science, Engineering, Information Systems, or related field.
Strong hands-on experience with Scala, Python or Java in backend or data intensive systems.
Experience working with Cloudera Data Platform and Spark.
Familiarity with Cloudera Manager for cluster administration, monitoring, or troubleshooting.
Strong understanding of data modeling concepts, distributed systems, and large scale data processing.
Preferred Qualifications
Hands on experience building, integrating, or supporting AI driven agents or intelligent automation solutions.
Experience with Databricks for data engineering or ML workloads.
Experience working in AWS services like S3, EC2, EMR, Glue, Lambda, and IAM.
Knowledge of streaming and big-data technologies like Kafka, Hadoop, and Hive.
