Multi-task self-supervised learning in graphs for stronger task generalization - BC-902
Project type: ResearchDesired discipline(s): Computer science, Mathematical Sciences, Mathematics
Company: Mastercard AI Garage
Project Length: 6 months to 1 year
Preferred start date: 08/01/2024
Language requirement: English
Location(s): BC, Canada
No. of positions: 1
Desired education level: PhD
Open to applicants registered at an institution outside of Canada: Yes
About the company:
We work to connect and power an inclusive, digital economy that benefits everyone, everywhere by making transactions safe, simple, smart and accessible. Using secure data and networks, partnerships and passion, our innovations and solutions help individuals, financial institutions, governments and businesses realize their greatest potential. Our decency quotient, or DQ, drives our culture and everything we do inside and outside of our company. We cultivate a culture of inclusion for all employees that respects their individual strengths, views, and experiences. We believe that our differences enable us to be a better team – one that makes better decisions, drives innovation and delivers better business results. At AI Garage, we use state-of-the-art AI techniques to solve some of the most important problems in the financial world.
Describe the project.:
Our initiative heavily invests in leveraging graph representations to enhance our understanding of transaction ecosystems. By adopting a graph-centric approach, we aim to uncover deeper insights into the various entities within the system and pioneer the development of innovative applications previously unattainable with tabular data. Recently, our team has concentrated on developing representations for users and merchants that capture the inherent graphical nature of transactions, thereby augmenting raw business-level features with additional information.
While Graph Neural Networks (GNNs) coupled with self-supervised learning frameworks have demonstrated significant performance enhancements across a spectrum of node-level tasks, they still grapple with various challenges contingent upon GNN design. Notably, existing research has illustrated that a singular self-supervised learning task often fails to excel across multiple downstream tasks. This limitation has been highlighted by seminal works such as Ju et al. ("Multi-task Self-supervised Graph Neural Networks Enable Stronger Task Generalization," 2023) and Jin et al. ("Automated Self-Supervised Learning for Graphs," 2022), which underscore the necessity for generating task-agnostic embeddings.
Addressing this challenge, our project aims to generate task-agnostic generalized embeddings for user and merchant nodes, showcasing performance improvements across multiple downstream tasks. We recognize that while previous research has laid the groundwork for such embeddings, the key hurdle lies in mitigating the task-interference problem. This obstacle has been extensively studied by researchers such as Xin et al. ("Do current multi-task optimization methods in deep learning even help?," 2022), Standley et al. ("Which tasks should be learned together in multi-task learning?," 2020), and Guangyuan et al. ("Recon: Reducing Conflicting Gradients From the Root For Multi-Task Learning," 2023).
In summary, our project endeavors to contribute to the field by tackling the task-interference problem and advancing the generation of task-agnostic embeddings. Through this endeavor, we aim to unlock new possibilities for enhancing our understanding of transaction data and pioneering novel applications within transaction ecosystems.
Required expertise/skills:
Python, Pytorch, DGL, Familiarity with Graph Neural Networks
