Providing Care Beyond Therapy Sessions With a Natural Language Processing-Based Recommender System That Identifies Cancer Patients Who Experience Psychosocial Challenges and Provides Self-care Support: Pilot Study

Yvonne W Leung; Bomi Park; Rachel Heo; Achini Adikari; Suja Chackochan; Jiahui Wong; Elyse Alie; Mathew Gancarz; Martyna Kacala; Graeme Hirst; Daswin de Silva; Leon French; Jacqueline Bender; Faye Mishna; David Gratzer; Damminda Alahakoon; Mary Jane Esplen

doi:10.2196/35893

Providing Care Beyond Therapy Sessions With a Natural Language Processing-Based Recommender System That Identifies Cancer Patients Who Experience Psychosocial Challenges and Provides Self-care Support: Pilot Study

JMIR Cancer. 2022 Jul 29;8(3):e35893. doi: 10.2196/35893.

Authors

Yvonne W Leung^{1

2

3}, Bomi Park^{1

4}, Rachel Heo⁵, Achini Adikari⁶, Suja Chackochan¹, Jiahui Wong¹, Elyse Alie¹, Mathew Gancarz¹, Martyna Kacala¹, Graeme Hirst⁷, Daswin de Silva⁶, Leon French², Jacqueline Bender^{2

8}, Faye Mishna⁹, David Gratzer^{2

10}, Damminda Alahakoon⁶, Mary Jane Esplen²

Affiliations

¹ de Souza Institute, University Health Network, Toronto, ON, Canada.
² Department of Psychiatry, University of Toronto, Toronto, ON, Canada.
³ College of Professional Studies, Northeastern University, Toronto, ON, Canada.
⁴ Faculty of Medicine, University of Toronto, Toronto, ON, Canada.
⁵ The Michael G DeGroote School of Medicine, McMaster University, Hamilton, ON, Canada.
⁶ Research Centre for Data Analytics and Cognition, LaTrobe University, Melbourne, Australia.
⁷ Department of Computer Science, University of Toronto, Toronto, ON, Canada.
⁸ The Department of Supportive Care, Princess Margaret Cancer Centre, Toronto, ON, Canada.
⁹ Faculty of Social Work, University of Toronto, Toronto, ON, Canada.
¹⁰ Centre for Addiction and Mental Health, Toronto, ON, Canada.

PMID: 35904877
PMCID: PMC9377447
DOI: 10.2196/35893

Abstract

Background: The negative psychosocial impacts of cancer diagnoses and treatments are well documented. Virtual care has become an essential mode of care delivery during the COVID-19 pandemic, and online support groups (OSGs) have been shown to improve accessibility to psychosocial and supportive care. de Souza Institute offers CancerChatCanada, a therapist-led OSG service where sessions are monitored by an artificial intelligence-based co-facilitator (AICF). The AICF is equipped with a recommender system that uses natural language processing to tailor online resources to patients according to their psychosocial needs.

Objective: We aimed to outline the development protocol and evaluate the AICF on its precision and recall in recommending resources to cancer OSG members.

Methods: Human input informed the design and evaluation of the AICF on its ability to (1) appropriately identify keywords indicating a psychosocial concern and (2) recommend the most appropriate online resource to the OSG member expressing each concern. Three rounds of human evaluation and algorithm improvement were performed iteratively.

Results: We evaluated 7190 outputs and achieved a precision of 0.797, a recall of 0.981, and an F1 score of 0.880 by the third round of evaluation. Resources were recommended to 48 patients, and 25 (52%) accessed at least one resource. Of those who accessed the resources, 19 (75%) found them useful.

Conclusions: The preliminary findings suggest that the AICF can help provide tailored support for cancer OSG members with high precision, recall, and satisfaction. The AICF has undergone rigorous human evaluation, and the results provide much-needed evidence, while outlining potential strengths and weaknesses for future applications in supportive care.

Keywords: artificial intelligence; natural language processing; online support groups; recommender system; supportive care in cancer.

©Yvonne W Leung, Bomi Park, Rachel Heo, Achini Adikari, Suja Chackochan, Jiahui Wong, Elyse Alie, Mathew Gancarz, Martyna Kacala, Graeme Hirst, Daswin de Silva, Leon French, Jacqueline Bender, Faye Mishna, David Gratzer, Damminda Alahakoon, Mary Jane Esplen. Originally published in JMIR Cancer (https://cancer.jmir.org), 29.07.2022.