Overview and Classification of Evaluation Metrics of Appointment Scheduling Systems

Mohammad Reza Mazaheri Habibi, Fahimeh Mohammad Abadi, Hamed Tabesh, Hasan Vakili Arki, Ameen Abu-Hanna, Saeid Eslami



Introduction: This study reviews the several metrics used to evaluate the performance of appointment scheduling systems.

Material and Methods: The articles in English were searched using PubMed, Scopus, and Web of Science databases and Google scholar search engine until July 23, 2023. We used queueing theory to classify evaluation metrics.

Results: Out of 23403 articles, 75 papers were prepared for detailed analysis. We classify evaluation metrics of appointment scheduling system along with their definition and frequency of use. A total of 24 measures containing twelve (%50), seven (%29), and five (%21) were related to the categories of arrivals (patient), queue (at clinic), and server (physician) were found, respectively.

Conclusion: To the best of our knowledge, this paper was one of the first studies collecting and classifying all evaluation metrics of appointment scheduling system in order to help other researchers. Most metrics pertained to patients which may highlight the importance of the patient’s perspective in evaluating appointment scheduling systems.


Appointments and Schedules; Ambulatory Care Facilities; Evaluation Metrics; Queueing Theory;


Gupta D, Denton B. Appointment scheduling in health care: Challenges and opportunities. IIE transactions. 2008; 40(9): 800-19.

Cayirli T, Veral E. Outpatient scheduling in health care: A review of literature. Production and Operations Management. 2003; 12(4): 519-49.

Zhang X, Yu P, Yan J. Patients’ adoption of the e-appointment scheduling service: A case study in primary healthcare. Stud Health Technol Inform. 2014; 204: 176-81. PMID: 25087546

Katre AN. Assessment of the correlation between appointment scheduling and patient satisfaction in a pediatric dental setup. Int J Dent. 2014; 2014: 453237. PMID: 25610464 DOI: 10.1155/2014/453237

Wang WY, Gupta D. Adaptive appointment systems with patient preferences. Manufacturing & Service Operations Management. 2011; 13(3): 373-89.

Atinga RA, Akosen G, Bawontuo V. Perceived characteristics of outpatient appointment scheduling association with patient satisfaction and treatment adherence: An innovation theory application. Hosp Pract (1995). 2021; 49(4): 298-306. PMID: 34121573 DOI: 10.1080/21548331.2021.1942878

Graham TAD, Ali S, Avdagovska M, Ballermann M. Effects of a web-based patient portal on patient satisfaction and missed appointment rates: Survey study. J Med Internet Res. 2020; 22(5): e17955. PMID: 32427109 DOI: 10.2196/17955

Mazaheri Habibi MR, Mohammad Abadi F, Tabesh H, Vakili Arki H, Abu-Hanna A, Eslami S. Evaluation of patient satisfaction of the status of appointment scheduling systems in outpatient clinics: Identifying patients' needs. J Adv Pharm Technol Res. 2018; 9(2): 51-5. PMID: 30131937 DOI: 10.4103/japtr.JAPTR_134_18

Gupta D, Wang L. Revenue management for a primary-care clinic in the presence of patient choice. Operations Research. 2008; 56(3): 576-92.

Lee S, Min D, Ryu Jh, Yih Y. A simulation study of appointment scheduling in outpatient clinics: Open access and overbooking. Simulation. 2013; 89(12): 1459-73.

Volk AS, Davis MJ, Abu-Ghname A, Warfield RG, Ibrahim R, Karon G, et al. Ambulatory access: Improving scheduling increases patient satisfaction and revenue. Plast Reconstr Surg. 2020; 146(4): 913-9. PMID: 32970013 DOI: 10.1097/PRS.0000000000007195

Mazaheri Habibi MR, Mohammad Abadi F, Tabesh H, Vakili Arki H, Abu-Hanna A, Eslami S. Effect of an online appointment scheduling system on evaluation metrics of outpatient scheduling system: A before-after multicenter study. J Med Syst. 2019; 43(8): 281. PMID: 31300894 DOI: 10.1007/s10916-019-1383-5

Palvannan RK, Teow KL. Queueing for healthcare. J Med Syst. 2012; 36(2): 541-7. PMID: 20703697 DOI: 10.1007/s10916-010-9499-7

McMullen MJ, Netland PA. Lead time for appointment and the no-show rate in an ophthalmology clinic. Clin Ophthalmol. 2015; 9: 513-6. PMID: 25834388 DOI: 10.2147/OPTH.S82151

Price RS, Balcer LJ, Galetta SL. Education research: A new system for reducing patient nonattendance in residents' clinic. Neurology. 2010; 74(10): e34-6. PMID: 20211902 DOI: 0.1212/WNL.0b013e3181d31de4

Lynn S, Edlund BJ, Dumas BP. Open access scheduling: Improving access to rural healthcare. Journal of Nursing Education and Practice. 2016; 6(9): 67.

Kopach R, DeLaurentis PC, Lawley M, Muthuraman K, Ozsen L, Rardin R, et al. Effects of clinical characteristics on successful open access scheduling. Health Care Manag Sci. 2007; 10(2): 111-24. PMID: 17608053 DOI: 10.1007/s10729-007-9008-9

Cayirli T, Veral E, Rosen H. Designing appointment scheduling systems for ambulatory care services. Health Care Manag Sci. 2006; 9(1): 47-58. PMID: 16613016 DOI: 10.1007/s10729-006-6279-5

Parente DH, Pinto MB, Barber JC. A pre-post comparison of service operational efficiency and patient satisfaction under open access scheduling. Health Care Manage Rev. 2005; 30(3): 220-8. PMID: 16093888 DOI: 10.1097/00004010-200507000-00006

Cohen SE, Reid MW, Wang HS, Kaung A, Patel A, Tashjian VC, et al. Tu1030 a novel algorithm for predicting patient absenteeism for gastroenterology procedures. Gastroenterology. 2014; 146(5): S-732.

Guy R, Hocking J, Wand H, Stott S, Ali H, Kaldor J. How effective are short message service reminders at increasing clinic attendance? A meta‐analysis and systematic review. Health Serv Res. 2012; 47(2): 614-32. PMID: 22091980 DOI: 10.1111/j.1475-6773.2011.01342.x

Granja C, Almada-Lobo B, Janela F, Seabra J, Mendes A. An optimization based on simulation approach to the patient admission scheduling problem using a linear programing algorithm. J Biomed Inform. 2014; 52: 427-37. PMID: 25194680 DOI: 10.1016/j.jbi.2014.08.007

DuMontier C, Rindfleisch K, Pruszynski J, Frey J. A multi-method intervention to reduce no-shows in an urban residency clinic. Fam Med. 2013; 45(9): 634-41. PMID: 24136694

Jamjoom A, Abdullah M, Abulkhair M, Alghamdi T, Mogbil A. Improving outpatient waiting time using simulation approach. European Modelling Symposium. IEEE; 2014.

Mehrotra A, Keehl-Markowitz L, Ayanian J. Implementation of open access scheduling in primary care: A cautionary tale. Ann Intern Med. 2008; 148(12): 915-22. PMID: 18559842 DOI: 10.7326/0003-4819-148-12-200806170-00004

Tuli SY, Thompson LA, Ryan KA, Srinivas GL, Fillipps DJ, Young CM, et al. Improving quality and patient satisfaction in a pediatric resident continuity clinic through advanced access scheduling. J Grad Med Educ. 2010; 2(2): 215-21. PMID: 21975623 DOI: 10.4300/JGME-D-09-00087.1

Lian J, DiStefano K, Shields SD, Heinichen C, Giampietri M, Wang L. Clinical appointment process: improvement through schedule defragmentation. IEEE Eng Med Biol Mag. 2010; 29(2): 127-34. PMID: 20659850 DOI: 10.1109/MEMB.2009.935718

Kennedy JG, Hsu JT. Implementation of an open access scheduling system in a residency training program. Fam Med. 2003; 35(9): 666-70. PMID: 14523666

Robinson LW, Chen RR. A comparison of traditional and open-access policies for appointment scheduling. Manufacturing & Service Operations Management. 2010; 12(2): 330-46.

White DL, Froehle CM, Klassen KJ. The effect of integrated scheduling and capacity policies on clinical efficiency. Production and Operations Management. 2011; 20(3): 442-55.

O’Connor ME, Matthews BS, Gao D. Effect of open access scheduling on missed appointments, immunizations, and continuity of care for infant well-child care visits. Arch Pediatr Adolesc Med. 2006; 160(9): 889-93. PMID: 16953011 DOI: 10.1001/archpedi.160.9.889

DOI: https://doi.org/10.30699/fhi.v13i0.573


  • There are currently no refbacks.