Citi
Pune, Maharashtra, India Posted 2d ago
Design, develop, and maintain scalable data pipelines and analytical solutions supporting large-scale data operations across enterprise systems at Citi.
Key Responsibilities
Design, develop, test, and maintain PySpark and Apache Spark data pipelines for batch and streaming workloads.
Manage and monitor Spark Streaming Jobs performing Eligibility and Data Quality checks.
Manage and monitor Spark Batch Jobs that store audit, inbound, and outbound data into Hive tables.
Respond to ITRS Alerts and system monitoring events.
Work with large-scale data sources across Oracle, Hadoop, Impala, and SAS Server.
Configure and manage JDBC connections from SAS, Python, and R.
Requirements
Proficiency in Python and PySpark for distributed data engineering.
Experience with Apache Spark in production environments.
Strong understanding of SQL and ability to work with relational and columnar databases.
Familiarity with Kafka or event-driven architectures.
Experience with job scheduling and monitoring tools like Autosys.
2-5 years of relevant experience.
Bachelor’s degree/University degree on information technology or equivalent.
