About this role
Lead PySpark Developer for the Surveillance Enhancement Program (SEP). This role will help build and maintain the data-intensive surveillance platform, with a focus on alerts enhancements, strategic data sourcing, UI/workflow improvements to meet MAR, Dodd-Frank, and other regulatory obligations, plus control room and suitability monitoring data.
What You'll Do
- Develop software applications based on business requirements and maintain enhancements per project specs
- Participate in requirement analysis, design, development, testing, and implementation activities
- Propose new techniques and technologies for software development and perform unit testing and user acceptance testing
- Ensure tasks are completed within deadlines and adhere to coding standards; mentor/assist Junior Developers
- Review requirements, perform code reviews, and develop technical documentation and specifications
- Act as primary development contact, analyze and resolve issues accurately, and support RESTful services/APIs
What We're Looking For
- 10+ years of experience in data-intensive PySpark development and proven experience leading a development team
- Strong Python development experience; good understanding of OOP, exception handling, and parallel processing
- Deep knowledge of DB connectivity, data loading, transformation, and calculation; extensive Pandas/Numpy dataframes experience
- Experience with data pipelines, concurrency, error handling, and batch processing; familiarity with apply/map/lambda patterns
- Experience with Git, RESTful services/APIs, service-oriented architecture, and designing scalable data architectures
- Practical experience with debugging, unit tests, and design of data pipelines for concurrent processing
Nice to Have
- Knowledge of Django, PostgreSQL, Apache Airflow, and Jira
- Understanding of Financial Markets asset classes (FX, FI, Equities, Rates, Commodities & Credit) and market regulation domains (MAR, MIFID, CAT, Dodd-Frank)
- English language proficiency (C2)