Development, Testing and Reporting of Mobile Apps for Psycho-social Interventions: Lessons from the Pharmaceuticals

The mobile Health offers great potential for delivery of Psycho-social Interventions. So far, the researchers have focused mainly on efficacy, and to some extent feasibility of Psycho-social Interventions. Poor reporting of development and initial testing has been highlighted in the recent published literature. This is not surprising considering the lack of guidance from the national regulatory agencies in developed countries. A lack of standard ways of development and testing of apps makes scientific evaluation of the intervention and replicability almost impossible. Replicabiliy is an essential component of good science. We are proposing a pathway to develop, test and report new mobile apps that deliver Psycho-social Intervention, that is based on current practices of approval of new drugs by the US, Food and Drug Administration (FDA). This proposed process consists of modifications in the current app development process, i.e., planning, alpha and beta testing. We are also suggesting an additional delta stage of testing. We note the advantages and limitations in using such approach.


Background
There has been extensive research in the use of eMedia [or more specifically digital media (dMedia)] delivered interventions for a variety of psychiatric disorders.Mobile apps delivering Psycho-social Interventions fall under the term mHealth.The term "mHealth" (mobile Health) is commonly used to describe the practice of medicine through mobile devices that are connected to internet.
Reviews in this area have so far focused on effects of eMedia delivered interventions using a variety of platforms [1][2][3].However, more recently the reviewers have started examining the literature for usability, acceptability and feasibility of these interventions [4].As far as we are aware there is no published literature that has reported development and initial testing of an eMedia delivered intervention in a scientific and systematic manner.Similarly, reported literature do not describe following a set of guidelines for quality control to develop eMedia delivered interventions.Only a few studies report details of the intervention developers, and an even smaller number of interventions tested in RCTs (Randomised Controlled Trials) are available in the public arena.
For replicability to be possible interventions should be clearly described.Poor description of interventions threatens the potential for good science in trials and this would seem an area of research where pioneering trialists could leave their work difficult to replicate through inexact descriptions of interventions.There is currently an interest in standardizing interventions in this area, and there have been attempts to establish criteria to self certify eMedia delivered interventions [5].Similarly, there is an interest in evaluation of risk assessment of such interventions [6].Surprisingly, the regulatory agencies have not paid sufficient attention to eMedia-delivered interventions until recently.Therefore, only little guidance is available for clinicians or end users.The Food and Drug Administration (FDA) released guidance recently, focusing primarily on mHealth apps (applications) that transform the mobile platform into a regulated medical device [7].The remaining eMedia will be subject to no regulation [8].In Australia guidance is available from the Therapeutic Goods Administration for health care practitioners, but this is very limited [9].The Business Standards Institution (BSI) in England, has developed "PAS 277 Health and wellness apps -Quality criteria across the life cycle -Code of practice" [10] in conjunction with Innovate UK (www.innovateuk.gov.uk).There are however, no concrete guidelines available in any of the above systems on developing, initial testing and reporting of the mobile apps for Psycho-social Interventions.The mHealth delivered Psycho-social Interventions might offer unique opportunities and challenges, e.g., they might be delivered as stand alone interventions for common mental health problems, without medical support or supervision.Therefore, there is a need to develop strict protocols and quality standards that the app developers in this area should follow.
We are suggesting possible pathway to development and testing mobile apps for Psycho-social Interventions, using an example of drug approval by the FDA.

Stages of Development and Testing of Drugs
The pharmaceuticals follow fixed procedures to ensure quality of new drugs, and in order to report its efficacy, safety and dosage.Any new drug in the United States must go through an extensive review and approval process by the (FDA) before it can be prescribed to patients.A drug is any product that is meant to be used in diagnosing, treating, or preventing diseases by affecting the structure or function of the body.Prescription drugs, which must be obtained with the written authorization of a licensed health care professional, are regulated by the FDA, and some over-the-counter drugs are regulated by the FDA as well [11].

Pre-clinical trial stage
When a substance is considered to be a potential drug, it is initially tested in a laboratory.This is carried out in two steps.During the first step lab studies are conducted to test the drug's effects on living cells, as well its toxic effects.The second step involves laboratory testing on animals to investigate the drug's safety, and its efficacy.
If the results of the laboratory and animal studies are encouraging, the drug sponsor submits an IND (Investigational New Drug) application to the FDA.The IND application summarizes information from the laboratory and animal testing and provides a proposal for obtaining clinical data from human patients [11].The FDA team reviewing the application can be composed of doctors, scientists, and statisticians.

Clinical trial stage (safety, dosage and efficacy)
If the FDA approves the IND application, clinical trials involving human participants start.The purpose of these trials is to test the safety, dosage and efficacy of the new drug.Clinical trials are divided into the following phases, although in real life one phase might test more than one outcome, Phase 1 trials -normally involve a small number of participants, who are healthy volunteers.The focus is on drug safety.Phase 2 trials -involve a reasonable number of participants with a disease or condition and focus on optimal dose and the ability of the drug to treat the condition.Phase 3 trials -involve even a higher number of participants with a disease or condition.The focus of these trials is safety and efficacy.These trials use an RCT design and commonly compare the drug with a placebo or a gold standard drug.These trials look into more in-depth questions, such as effects on certain groups of patients.

Marketing Stage
Once the clinical trials are completed, the pharmaceutical submits an NDA (New Drug Application), which requests approval of the drug for marketing in the United States.If the drug is approved by the FDA, the official label for the drug is written, which describes the indications, known side effects and warnings about the product [11].
The goals of the NDA are to provide sufficient information to permit FDA to establish the followings: [1] whether the drug is safe and effective when used as directed?[2] are these benefits more than the risks?[3] Is the drug's labelling appropriate, and what information should it contain?[4] Did the pharmaceuticals use appropriate methods in developing, testing and reporting the drug?and [5] Does the label tells the complete story of drug development, psychopharmacology, toxicity etc.

Post Marketing and Effectiveness Testing
The label may be updated to include information about the drug's new indications and the side effects.The pharmaceutical is required to submit safety updates, and both the prescribers and the patients can also report serious events related to the drug to the FDA.Some drugs are withdrawn if they cause very serious side effects.Further trials are conducted to test the effectiveness (pragmatic trials) of the drug, to measure the degree of beneficial effect under "real world" clinical settings (Table 1).

Stages of Development and Testing of Mobile Apps
An end to end testing process begins with engaging the testing team upfront while requirements are being defined.These should include non-functional, regulatory and performance requirements.These requirements must be inspected by all the stakeholders and signed off.It is mandatory for all the participating business/engineering teams to follow step-by-step processes, that can be summarized in following steps;,

Pre-alpha stage
The pre-alpha stage refers to all activities performed during the software project before testing.These activities can include requirements analysis, software design, software development, and unit test

Alpha testing
The alpha phase is the first step to software testing.In this phase, the app is tested using a variety of techniques, most notably white box techniques, that test the internal structures or workings of an app [12], and the black box techniques that test the functionality of an app.Additionally, the gray box testing combine both black and white box techniques, by another testing team [13].

Beta testing
Once the app is feature complete, the beta testing is carried out.This can reveal unknown defects or bugs.This is normally tests usability or acceptability of the app.The process of delivering a beta version to the users is called "beta release" and this is typically the first time that the software is available outside of the organization that developed it.When new features and functionality are constantly added to the app it is called "perpetual beta".The beta testing can either be "closed beta" in which beta testing is done by a closed group of individuals or "open beta" in which testing is done by a larger group.

Release of the app
Once the beta testing is complete, the app is called a release candidate (RC) or the final product.In this stage of product stabilization, all product features have been designed, coded and tested through one or more beta cycles with no known "show-stopper-class bug".This release is also called "code complete".Once released, the software is generally known as the "stable release".

Support and end-of-life
During its supported lifetime, software is sometimes subjected to service releases, or service packs, sometimes also called "interim releases".When software is no longer sold or supported, the product is said to have reached end-of-life.

The way forward: A proposed pathway to development and testing of the mobile apps
We suggest a pathway to develop and test apps that deliver Psychosocial Interventions, that can follow the following sequence;

Development stage
In addition to the steps described in pre alpha testing stage, the initial development stage should also consider the type of the interventions.The following types of interventions are normally delivered by eMedia; Type 1 [An intervention delivered by a human therapist through eMedia (e.g., telephone delivered problem solving by a therapist, Avatar Therapy)], Type 2 [An intervention based on a manualized, well established therapy delivered through eMedia (e.g., CBT delivered by a website that is based on a manual, or thought diaries)], and, Type 3 [A new intervention that did not exist before, and is not based on previous theory or on therapeutic principles (e.g, electronic dispensing), or if it is, it is based loosely in theory].It, therefore, follows that both Type 1 and 2 interventions will require less

Preclinical Testing Alpha testing
The process of app testing can start normally with alpha testing (white and black box testing).

Artificial intelligence (AI) testing
Artificial Intelligence is about making machines intelligent [14].There has been a recent increase in interest in the application of Artificial Intelligence (AI) techniques to Software Engineering (SE) problems.The work in this field is typified by recent advances in Search Based Software Engineering, but also by long established work in Probabilistic reasoning and machine learning for Software Engineering [14].This is an area that is full of controversies.To start with, for example, there is no agreement on definition or function (s) of intelligence, among scientists.There are published reports of intelligent Real Time Therapy (iRTT), using experiential sampling methods (ESM) [15], however, none of the available interventions in this area can be considered intelligent using the current criteria for artificial intelligence.This is, however an area that can be considered in future app testing.

Clinical testing
This can typically focus on efficacy, but the secondary objectives include, testing for dosage and safety.This stage can also include acceptability [User Acceptance Testing (UAT) how the end user experience the app] and feasibility [whether the intervention can be implemented given time, financial, legal, personal, and social constraints].This can be divided into the following steps:

Beta testing
We recommend that Beta testing should be conducted in two stages, i.e., closed and open beta testing.

Closed Beta testing (phase 1 trial)
This phase can involve a small number of healthy volunteers to test the safety (side effects, privacy and security).There is evidence that Psycho-social Interventions cause adverse outcomes [16].Furthermore, there may be risks particular to the eMedia such as reduction in social activities, or even addiction to eMedia [6].However, we are not aware of published literature in this area.Delivery through eMedia complicates and possibly increases risks, in terms of security and privacy of data [17].Additionally, preliminary data on acceptability and feasibility can be collected.

Open Beta testing (Phase 2 trial)
This can typically test dosage (we believe that the concept of dosage is important here, to offer advice on duration of the app use, in order to avoid risks of side effects), acceptability, and feasibility.This phase can further test efficacy by the end users in a pre and post test design.

Gamma testing (Phase 3 trials)
This phase can focus solely on efficacy using RCT design, that compare the app with either a placebo (for example, a game app that has no activity) or an already approved app that is known to work for the condition being treated.Secondary objectives can include testing safety and acceptability.

Download history and user feedback
In addition to the Phase 2 and 3 trials, the number of downloads and feedback from the participants should be essential part of the evaluation Box 1: Suggested mobile app information sheet

Marketing and Post marketing stage and effectiveness trials
Once the initial testing has been carried out, the app can be launched.The app at this stage should provide "information sheets" similar to those provided by the drug company (Please see Box 1).Future trials can focus on effectiveness, including cost effectiveness.

Summary
The increasing number of mobile apps requires careful consideration of how they are developed, tested and marketed.There is a need for professionals from both the IT and the Health background to work jointly in these areas.Mobile apps for Psycho-social Interventions can be developed using a process similar to the discovery, development, testing and marketing of drugs.There are notable differences between the two, for example the apps have a relatively short life, compared with the drugs, due to rapid developments in the field.This, however, also highlights a further complication, due to an ongoing process of improvement based on technological advances and user feedback, the app might change radically after some time.This also means, the period for app

Page 2 of 5 J
and development Discovery of a new substance, lab development and initial testing Requirements analysis, software design, software development, and unit testing etc. Involvement of health experts and patients IND (Investigational New Drug) application Clinical testing (toxicity and efficacy) Preclinical pharmacology and toxicology-animal testing Alpha testing (within the organization) -includes, white box, black box and gray box testing Alpha testing as in routine practice Clinical testing (safety, dosage and efficacy)phase I, II and III trials Beta testing (testing of final version by the end user) Closed beta testing (phase 1 trial) by a small number of healthy volunteers to test the safety (side effects, privacy and security).Preliminary data on acceptability and feasibility.Open beta testing (phase 2 trial) to test dosage, acceptability feasibility.Preliminary data on efficacy.Gamma testing (phase 3 trial) -efficacy testing in an RCT against a placebo or another app Download statistics Citation: Naeem F, Syed Y, Xiang S, Shokraneh F, Munshi T, et al (2015) Development, Testing and Reporting of Mobile Apps for Psycho-social Interventions: Lessons from the Pharmaceuticals.J Med Diagn Meth 4: 191.doi:10.4172/2168-9784.1000191Med Diagn Meth, an open access journal ISSN:App launched with information about its use (See Box 1) Post marketing stage and effectiveness testing Post marketing Testing (effectiveness trial) in routine clinical care Delta testing (effectiveness trials) Post marketing support Post marketing support, e.g., updates, security, functional patches etc Monitoring of safety in addition to providing support

Citation:Page 3 of 5 J
Naeem F, Syed Y, Xiang S, Shokraneh F, Munshi T, et al (2015) Development, Testing and Reporting of Mobile Apps for Psycho-social Interventions: Lessons from the Pharmaceuticals.J Med Diagn Meth 4: 191.doi:10.4172/2168-9784.1000191Med Diagn Meth, an open access journal ISSN:2168-9784 Volume 4 • Issue 4 • 1000191 rigorous testing.Although, type 1 might only be used by a therapist (on prescription versus over the counter apps).

Table 1 :
Comparison of drug and mobile apps for psycho social interventions: Development and testing.