COMPUTATIONAL APPROACHES TO MULTI-OBJECTIVE DECISION SUPPORT WITHIN HEALTHCARE

Abstract

Sequential Decision Problems (SDPs) are characterized by the utility of an agent depending upon a series of interconnected actions, typically formalized mathematically as Markov Decision Processes (MDPs). In single-objective MDPs, the desirability or undesirability of performing a particular action, a_t, in some state, s_t, is captured by a single scalar reward function, r(s_t, a_t), which quantifies the immediate consequence of that decision. However, many real-world settings, especially those existing in healthcare domains, require a more nuanced approach capable of balancing multiple conflicting objectives beyond a single scalar reward. While Multi-Objective Optimization (MOO) generally focuses on static methods for identifying trade-offs among competing objectives, Multi-Objective Sequential Decision-Making (MO-SDM) extends these ideas to sequential processes in which each decision affects future outcomes. In this thesis, we explore and address the unique challenges of both MO-SDM and MOO through four key objectives, with a particular focus on multi-level challenges prevalent in Skilled Nursing Facilities (SNFs) and other healthcare domains. In order to develop meaningful MO-SDM support tools for real-world environments, it is crucial to first develop an understanding of the contextual factors that drive current decisionmaking practices. Thus, our first primary objective was to perform a comprehensive analysis of the financial, clinical, and operational factors currently driving referral decisions in SNFs. In collaboration with PointClickCare, a leading cloud-based healthcare software provider, we performed the first large-scale multi-level quantitative analysis of patient referral decisions. This analysis provided us with a foundational knowledge of decision-making trends in referral acceptance within SNFs and revealed the gaps that demonstrated the need for advanced decision support tools. To address the need for decision-support tools in SNFs, our second objective was to develop SNFsim, an open-source stochastic simulator designed using real-world data to model the facility-level and patient-level decision-making processes in SNFs. The potential uses of this simulator are two-fold. First, it provides a new testbed for the development and comparison of multi-objective reinforcement learning (MORL) algorithms, and second, as the basis of a decision-support system that can be tailored to a variety of SNF environments to provide realworld decision support. On a broader scale, our third objective was to conduct the first review of the use of MOO techniques in healthcare domains, identifying research gaps and exploring potential future research directions. Through this survey, we intended to shine light on the modern MOO approaches used to address multi-objective challenges in long-standing multi-objective healthcare optimization problems. Finally, our fourth objective was to develop a copula-based weight selection technique for scalarization in MORL. This technique is helpful when preference weights are not provided a priori and a low compute budget must be considered, effective for when full Pareto frontier exploration is infeasible. Ultimately, this thesis provides a detailed look into the intricacies and challenges of MO-SDM and MOO within healthcare, and introduces solutions to help address these challenges.