Senior Pyspark Developer

Senior PySpark Developer to join a leading investment bank's Trade Surveillance Team. You will design, develop and maintain data-intensive PySpark-based software solutions, translating business requirements into scalable data pipelines and analytics.

What You'll Do

• Develop software applications based on business requirements
• Maintain software applications and enhancements according to project specifications
• Participate in requirement analysis, design, development, testing, and implementation activities
• Propose new techniques and technologies for software development
• Perform unit testing and user acceptance testing to evaluate application functionality
• Ensure to complete the assigned development tasks within deadlines
• Work in compliance with coding standards and best practices
• Provide assistance to Junior Developers when needed
• Perform code reviews and recommend improvements
• Review business requirements and recommend changes to develop reliable applications
• Develop coding documentation and other technical specifications for assigned projects
• Act as primary contact for development queries and concerns
• Analyze and resolve development issues accurately

What We're Looking For

• 8+ years of experience in data intensive PySpark development
• Experience as a core Python developer
• Basic experience in Gen/Agentic AI
• Experience developing Classes, OOP, exception handling, parallel processing
• Strong knowledge of DB connectivity, data loading, transformation, calculation
• Extensive experience in Pandas/Numpy dataframes, slicing, data wrangling, aggregations
• Lambda functions, decorators
• Vector operations on Pandas dataframes/series
• Application of applymap, apply, map functions
• Concurrency and error handling in data pipelines (batch sizes of 1-10 GB)
• Ability to understand business requirements and translate into technical requirements
• Ability to design data pipeline architecture for concurrent data processing
• Familiar with creating/designing RESTful services and APIs
• Familiar with application unit tests
• Working with Git source control
• Service-oriented architecture considerations and integrations
• Debugging applications

Nice to Have

• Knowledge of web backend technologies Django, Python, PostgreSQL
• Apache Airflow
• Atlassian Jira
• Understanding of Financial Markets Asset Classes (FX, FI, Equities, Rates, Commodities & Credit) and various trade types
• Surveillance domain knowledge and related regulatory systems (MAR, MiFID, CAT, Dodd-Frank)
• English language proficiency (C2)