Baseline Assessment
The ORCHA Assessment Development Team have translated the DHAF evaluation criteria into a set of 350 objective questions that trained ORCHA Assessors are looking to answer. Questions are scored, earning the app positive points or negative points depending on the nature of the question and the response. Each Review Domain (set of questions) produces a Domain score. These Review Domain scores are then combined to create an overall DHAF percentage score. Only apps with a score of 65% or higher are included in the ACP App Library. Any score below 65% would indicate that an app has some issues that users should investigate further prior to using this app. Scores below 45% indicate that an app has considerable issues or challenges and in its current form is potentially unhelpful or unsafe. This scoring process aims to deliver an objective and unbiased evaluation of all apps so they are treated fairly irrespective of their current popularity or financial position.
Enhanced Clinical Assessment
For the ACP App library, apps that are classified as Tiers Bii, Ci and Cii all go through an Enhanced Clinical Assessment process conducted by physicians and other healthcare professionals with expertise related to the App category. Through this process the content sources, published evidence and use cases are assessed, and the clinical assessor generates a subjective summary of the app.
Need for Professional Judgement
Through its work for national health systems around the world, ORCHA has developed the expertise and processes to review apps efficiently and carefully. For this reason, ACP is confident that the assessments provided by ORCHA are of high quality and will be helpful to physicians. However, physicians should use their professional judgement to determine whether a certain app may be appropriate and useful for a patient. It is recommended that a physician reviews the content of the app alongside the DHAF results to help determine if the app will meet their patient’s needs.