Designing a System to Empower Startup Businesses for Evaluating Digital Health Products

Mohammad Seifi, Elham Maserat



Introduction: Given the emergence of startup businesses in digital health and the importance of their products for human well-being, there is a critical need for consistent and continuous evaluation of these products. Therefore, producing high-quality and standard products in the health sector helps maintain the competitive market both nationally and internationally. Due to the lack of an evidence-based system that provides comprehensive educational information and an empowerment platform in health fields, this study aimed to design a system for empowering startup businesses to evaluate digital health products.

Material and Methods: This study was conducted in three phases: 1) descriptive, 2) development, and 3) evaluation. In the first phase, data elements and functional and non-functional requirements were identified through a literature review and validation by experts. In the second phase, the system was developed using SQL Server 2019, C# language, and an interface was created using ASP.NET Core in Visual Studio 2022, along with programming languages HTML, CSS, Bootstrap, and Java to design a database. In the evaluation phase, the designed system was assessed based on Nielsen’s questionnaire and usability indicators.

Results: Twenty functional and eleven non-functional components were identified for the design of the system, which were confirmed by related experts. The system comprises main modules, including educational resources, empowerment, a network of startups, a discussion forum, experiences, and a library for evaluating digital health products. In the empowerment section, the modules of consultation, webinars, coaching programs, tools, indicators, and evaluation frameworks were developed and implemented. The highest average severity of issues was related to the error prevention section rather than reminders, while the lowest average severity of issues was associated with the consistency and standards principle.

Conclusion: The design of empowerment systems in startup businesses can provide a suitable platform for enhancing evidence-based evaluation for their companies.


Startup Businesses; Empowerment; Digital Health Products; Evaluation;


Aluttis C, Bishaw T, Frank MW. The workforce for health in a globalized context: Global shortages and international migration. Glob Health Action. 2014; 7: 23611. PMID: 24560265 DOI: 10.3402/gha.v7.23611

Saturno-Hernández PJ, Martínez-Nicolás I, Flores-Hernández S, Poblano-Verástegui O. Quality of the health information system: A comparative analysis of reported indicators, OECD Mexico 2010-2016. Salud Publica Mex. 2019; 61(2): 184-92. PMID: 30958961 DOI: 10.21149/9688

Muhos M, Saarela M, Foit D, Rasochova L. Management priorities of digital health service start-ups in California. International Entrepreneurship and Management Journal. 2019; 15: 43-62.

Pinto R, Baracsi M. Creating an environment for innovative start-ups in healthcare. Health Policy and Technology. 2012; 1(4): 187-92.

Rocha A, Martins A, Junior JCF, Boulos MNK, Vicente ME, Feld R, et al. Innovations in health care services: The CAALYX system. Int J Med Inform. 2013; 82(11): e307-20. PMID: 21481633 DOI: 10.1016/j.ijmedinf.2011.03.003

Barnett J, Vasileiou K, Djemil F, Brooks L, Young T. Understanding innovators' experiences of barriers and facilitators in implementation and diffusion of healthcare service innovations: A qualitative study. BMC Health Serv Res. 2011; 11: 342. PMID: 22176739 DOI: 10.1186/1472-6963-11-342

Melchiorre MG, Lamura G, Barbabella F, Consortium IE. eHealth for people with multimorbidity: Results from the ICARE4EU project and insights from the “10 e’s” by Gunther Eysenbach. PLoS One. 2018; 13(11): e0207292. PMID: 30427924 DOI: 10.1371/journal.pone.0207292

Agarwal R, Gao G, DesRoches C, Jha AK. Research commentary-The digital transformation of healthcare: Current status and the road ahead. Information Systems Research. 2010; 21(4): 796-809.

Torab Miandoab A, Rezaei Hachesu P, Samad T, Habibi Chenaran S. Image processing technique for determining cold thyroid nodules. International Conference on Current Research in Computer Science and Information Technology. IEEE; 2017.

Iyawa GE, Herselman M, Botha A. Digital health innovation ecosystems: From systematic literature review to conceptual framework. Procedia Computer Science. 2016; 100: 244-52.

Najafi F, Irandoost M, Soltanpanah H, Sheikhahmadi A. A Model for relationship management with fintech and financial startups in banking industry. Journal of Business Strategies. 2020; 16(13): 1-18.

Blank SG, Dorf B. The startup owner's manual: The step-by-step guide for building a great company. Wiley; 2013.

Centobelli P, Cerchione R, Esposito E. Knowledge management in startups: Systematic literature review and future research agenda. Sustainability. 2017; 9(3): 361.

Goodman CS. Introduction to health technology assessment. National Library of Medicine (US); 2014.

Moshi MR, Tooher R, Merlin T. Development of a health technology assessment module for evaluating mobile medical applications. Int J Technol Assess Health Care. 2020; 36(3): 252-61. PMID: 32419676 DOI: 10.1017/S0266462320000288

Vis C, Bührmann L, Riper H, Ossebaard HC. Health technology assessment frameworks for eHealth: A systematic review. Int J Technol Assess Health Care. 2020; 36(3): 204-16. PMID: 32297588 DOI: 10.1017/S026646232000015X

Garrido MV, Gerhardus A, Røttingen J-A, Busse R. Developing health technology assessment to address health care system needs. Health Policy. 2010; 94(3): 196-202. PMID: 19889471 DOI: 10.1016/j.healthpol.2009.10.002

Agarwal P, Gordon D, Griffith J, Kithulegoda N, Witteman HO, Sacha Bhatia R, et al. Assessing the quality of mobile applications in chronic disease management: A scoping review. NPJ Digit Med. 2021; 4(1): 46. PMID: 33692488 DOI: 10.1038/s41746-021-00410-x

Guo C, Ashrafian H, Ghafur S, Fontana G, Gardner C, Prime M. Challenges for the evaluation of digital health solutions: A call for innovative evidence generation approaches. NPJ Digit Med. 2020; 3: 110. PMID: 32904379 DOI: 10.1038/s41746-020-00314-2

Sreenivasan A, Suresh M. Future of healthcare start-ups in the era of digitalization: Bibliometric analysis. International Journal of Industrial Engineering and Operations Management. 2022; 4(1/2): 1-18.

Behroozfar H. Usability evaluation of journal websites in the field of knowledge and information science based on the Nielsen model. Library and Information Science Research. 2023; 13(1): 44-61.

Torab Miandoab A, Samad Soltani T, Shams Vahdati S, Rezaei Hachesu P. An intelligent system for improving adherence to guidelines on acute stroke. Turk J Emerg Med. 2020; 20(3): 118-34. PMID: 32832731 DOI: 10.4103/2452-2473.290062

Haverinen J, Keränen N, Falkenbach P, Maijala A, Kolehmainen T, Reponen J. Digi-HTA: Health technology assessment framework for digital healthcare services. Finnish Journal of eHealth and eWelfare. 2019; 11(4): 326-41.

He J, King WR. The role of user participation in information systems development: Implications from a meta-analysis. Journal of Management Information Systems. 2008; 25(1): 301-31.

Bødker K, Kensing F, Simonsen J. Participatory design in information systems development. In: Isomäki H, Pekkola S (Eds). Reframing humans in information systems development. Springer Link; 2011.

Iivari J, Isomäki H, Pekkola S. The user–the great unknown of systems development: Reasons, forms, challenges, experiences and intellectual contributions of user involvement. Information Systems Journal. 2010; 20(2): 109-17.

Poortaheri N, Alimohammadzadeh K, Hosseini SM, Maher A, Bahadori M. Analysis of policies for organizing and managing informal settlements in Iran: A content analysis method. Evidence Based Health Policy, Management and Economics. 2021; 5(3): 194-206.

Mousavi SM, Takian A, Tara M. Sixteen years of eHealth experiences in Iran: A qualitative content analysis of national policies. Health Res Policy Syst. 2021; 19(1): 146. PMID: 34895250 DOI: 10.1186/s12961-021-00795-x

Hosseini SM, Boushehri SA, Alimohammadzadeh K. Challenges and solutions for implementing telemedicine in Iran from health policymakers’ perspective. BMC Health Serv Res. 2024; 24(1): 50. PMID: 38200535 DOI: 10.1186/s12913-023-10488-6

Oppong-Tawiah D, Bassellier G. Digital innovation, platform orientation and the performance of IT startups. International Conference on Information Systems: Transforming Society with Digital Innovation. Korea Association for Information Systems; 2017.

Bokhari RH. The relationship between system usage and user satisfaction: A meta‐analysis. Journal of Enterprise Information Management. 2005; 18(2): 211-34.

Bano M, Zowghi D, da Rimini F. User satisfaction and system success: An empirical exploration of user involvement in software development. Empirical Software Engineering. 2017; 22: 2339-72.

Lindgaard G, Dillon R, Trbovich P, White R, Fernandes G, Lundahl S, et al. User needs analysis and requirements engineering: Theory and practice. Interacting with Computers. 2006; 18(1): 47-70.

Chenthara S, Ahmed K, Wang H, Whittaker F. Security and privacy-preserving challenges of e-health solutions in cloud computing. IEEE Access. 2019; 7: 74361-82.

Abouelmehdi K, Beni-Hessane A, Khaloufi H. Big healthcare data: preserving security and privacy. Journal of Big Data. 2018; 5(1): 1-18.

Tangari G, Ikram M, Ijaz K, Kaafar M, Berkovsky S. Mobile health and privacy: Cross sectional study. BMJ. 2021; 373: 1248.

Salamzadeh A, Kawamorita Kesim H. Startup companies: Life cycle and challenges. International Conference on Employment, Education and Entrepreneurship. Entrepreneurship & Management eJournal; 2015.

Khan MS, Shaikh BT. What does it take to make a wrong decision? A qualitative study from Pakistan’s health sector. Dialogues in Health. 2023; 2: 100127.

Komashie A, Hinrichs-Krapels S, Clarkson PJ. Systems approaches to healthcare systems design and care delivery: An overview of the literature. Proceedings of the Design Society. Cambridge University Press; 2021.

Tran TNT, Felfernig A, Trattner C, Holzinger A. Recommender systems in the healthcare domain: State-of-the-art and research issues. Journal of Intelligent Information Systems. 2021; 57: 171-201.

Nicholson W. Risks and remedies for artificial intelligence in health care. Brookings Intitution; 2019.

McGill E, Marks D, Er V, Penney T, Petticrew M, Egan M. Qualitative process evaluation from a complex systems perspective: A systematic review and framework for public health evaluators. PLoS Med. 2020; 17(11): e1003368. PMID: 33137099 DOI: 10.1371/journal.pmed.1003368

Farzandipour M, Nabovati E, Sadeqi Jabali M. Comparison of usability evaluation methods for a health information system: Heuristic evaluation versus cognitive walkthrough method. BMC Med Inform Decis Mak. 2022; 22(1): 157. PMID: 35717183 DOI: 10.1186/s12911-022-01905-7

Hanbury A, Mafirakureva N, Chicken N, Bailey L. Evaluating change in a degraded healthcare system: A cross-sectional study of implementation outcomes using routine data indicators and proxies. Implement Sci Commun. 2023; 4(1): 96. PMID: 37587530 DOI: 10.1186/s43058-023-00471-x

Khachatryan T. The European e-commerce market: Digital single market [MSc Thesis]. Università Ca' Foscari Venezia; 2018.

Lagan S, Sandler L, Torous J. Evaluating evaluation frameworks: A scoping review of frameworks for assessing health apps. BMJ Open. 2021; 11(3): e047001. PMID: 33741674 DOI: 10.1136/bmjopen-2020-047001

Mathews SC, McShea MJ, Hanley CL, Ravitz A, Labrique AB, Cohen AB. Digital health: A path to validation. NPJ Digit Med. 2019; 2: 38. PMID: 31304384 DOI: 10.1038/s41746-019-0111-3

Stern AD, Brönneke J, Debatin JF, Hagen J, Matthies H, Patel S, et al. Advancing digital health applications: priorities for innovation in real-world evidence generation. Lancet Digit Health. 2022; 4(3): e200-6. PMID: 35216754 DOI: 10.1016/S2589-7500(21)00292-2

Laifenfeld D. Turning challenges into opportunities: Evaluation of digital health products. Applied Clinical Trials. 2021; 30(3): 9-10.

Holmes Fee C, Hicklen RS, Jean S, Abu Hussein N, Moukheiber L, de Lota MF, et al. Strategies and solutions to address digital determinants of health (DDOH) across underinvested communities. PLOS Digit Health. 2023; 2(10): e0000314. PMID: 37824481 DOI: 10.1371/journal.pdig.0000314



  • There are currently no refbacks.