Once researchers have an idea for a research study, they need to determine how the study will be executed. Study designs provide guidelines about how a study can be planned, but many studies are hybrids or approximations of these designs.³ Study designs are categorized in different ways; for the purpose of simplification, we will review considerations for studies that involve interventions and those that do not.⁴ Descriptions of common study designs are provided in Table 1. 

Noninterventional, or observational, studies involve data collected without modifying the environment, introducing a treatment, or testing a new device. Examples include cross-sectional, case-control, and cohort studies. The researcher typically gathers information on factors that may be related to a specific condition or outcome.⁷ Noninterventional studies may be prospective or retrospective, and patient interactions may or may not be involved (eg, surveys or interviews vs medical record reviews). Each of the types of noninterventional designs has its strengths and weaknesses.^3(p120) Overall, it is not possible to make strong causal inferences based on the results of noninterventional studies because randomization is not possible. These designs are useful for gaining information about differences that could be used to design a subsequent interventional design. 

In interventional studies, such as randomized controlled trials (RCTs), a group of participants receives a treatment or intervention so that researchers may determine whether that treatment or intervention is safe and effective or efficacious. This type of study is also called a clinical trial and can be either controlled or uncontrolled. Because of the nature of this study design, trials are always prospective. To ensure adequate reporting of RCTs, researchers should consult the CONSORT, or Consolidated Standards of Reporting Trials, guidelines.⁸ Included in the guidelines are a checklist and flow diagram. Of note, an RCT may need to be registered with a public registry (eg, ClinicalTrials.gov) before participants are enrolled in the study. Institutional review boards will assist with this registration. 

Designs for interventional studies require more steps than those of other types of studies. At the outset, researchers need to address the study population, number of participants, randomization, hypotheses concerning the anticipated outcomes, and statistical analyses required to determine whether the results have adequate validity. Interventional studies provide the strongest causal inferences and tests of hypotheses with minimal bias, but they also can be time consuming, be expensive, and require special care to ensure validity of the outcome. 

A research project can be developed by using these 2 types of designs individually or in sequence. That is, a researcher can start with a noninterventional study and then continue with an interventional trial. Hulley et al³ provide the following example: 

A noninterventional study would ask: This noninterventional study could be followed by an evaluation of associations on the same topic: Finally, this evaluation of associations could evolve into an interventional trial that asks: Researchers should choose the study design that will best answer their research question. For the remainder of the article, we focus on specific considerations for interventional studies. 

The statistical test determines whether or not the null hypothesis can be rejected. At its simplest, if a statistical test of the difference between 2 means (ie, the mean quantity of analgesics prescribed for men vs the mean quantity prescribed for women) is statistically significant (eg, P<.05), then the null hypothesis can be rejected and it is reasonable to conclude that a difference in prescribing patterns exists. Statistical analyses can be much more complicated, however, so novice researchers should seek the assistance of a biostatistician early in the study design process. 

Several other design considerations, such as how many participants are required to conduct a valid study, how participants are to be allocated to different groups, and how to determine inclusion and exclusion criteria must be taken into account as researchers design their study. 

To determine the number of participants needed for a valid result, researchers need to establish some initial information, including the expected mean for each of the study groups, the expected standard deviation for each of the groups, and the anticipated or desired effect size, or expected change in outcomes.⁷ The expected mean, standard deviation, and effect size are estimated assuming the results will be what the researchers are anticipating. 

Researchers should work with their statistician to determine these parameters and estimate the sample size necessary to achieve a given power. 

Participants for all groups of a study must be drawn from the same population; they may be patients from a certain hospital or hospital group, patients with a specific disease, or patients with a particular stage of a disease. 

Composition of the study population can also depend on how far researchers want to generalize the results. If researchers are not concerned about generalizing beyond their site, then they may select a representative sample of the patients or participants at their location. If researchers want to generalize to a larger population, their sample should be representative of their state or country. It may be hard for researchers to obtain a diverse sample if they are drawing their sample from a homogeneous population. However, researchers should try to recruit a study sample with a balance of participants of different ages, sexes, and major ethnic groups (unless, of course, the study is focused on a specific age, sex, or ethnic group). Results cannot be generalized if most of the people in a sample are white, for example. Of note, the National Institutes of Health requires a representative sample of sexes and ethnic groups for studies it funds.⁹ 

Most studies specify inclusion and exclusion criteria for participant selection. For example, Coglianese et al¹⁰ identified the desired composition of their study sample: For this study, inclusion criteria were participation in the Framingham Heart Study and ages 50 to 65 years, and exclusion criterion was history of heart failure. Exclusion criteria often include medications or conditions that may affect the outcome of the study. Researchers conducting a study of the use of OMT for back pain would likely exclude those with fractures of the spine, osteoporosis, and other conditions that could be exacerbated by OMT. 

Researchers also need to determine the number of groups needed for their study and whether a control group should be used. For example, if researchers are comparing the effects of 3 drugs, their study may need 3 groups. If the researchers would like to compare the effects of the 3 drugs with no treatment or standard treatment, a control group is also needed, or 4 groups total. 

A control group, or a group that does not receive an intervention but receives standard or no treatment, is used to determine whether an effect is caused by an intervention or by some other factor, such as spontaneous remission, time, or an external influence. A control group also permits a better statistical analysis to demonstrate effectiveness. 

The number of independent variables in a study, or variables that are being manipulated, will affect the number of groups needed. Typically, researchers will have 1 independent variable, such as drug dosage, and 1 moderator variable (a secondary independent variable, such as sex). A study can have 2 independent variables, such as drug type and dosage, and add the moderator variable for a 3-way design. With this type of study, however, researchers need a large total number of participants to maintain a reasonable number per group. For example, researchers conducting a study of 2 drugs would need to have 2 groups. If the researchers would like to study 2 dosages for each drug, they would then need 4 groups. If sex is added as a moderator variable, then the researchers need 8 groups. If 10 participants are needed for each group, then 80 participants are needed. Researchers may need to determine how many participants they can access to determine the number of groups that their study can support. They should enlist the help of a statistician. 

After the study groups are established, the participants are randomly assigned to the groups. Randomization³ is essential for interventional studies to ensure that groups are equivalent. The assumption is that any individual has an equal chance of ending up in any of the groups. 

Probability sampling^3,11 is a valid randomization method that meets the criteria for effective randomization. The preferred method is simple random sampling, in which participants are assigned to groups by use of a random number generator (eg, if the last digit of the random number is even, the participant is assigned to group 1; if the last digit is odd, the participant is assigned to group 2). Simple random samples can also be obtained by drawing a colored marble from a bowl (eg, blue for group 1, red for group 2) and other similar procedures. 

For observational studies, it is important for researchers to ensure that they have a random sample of the participants so that they do not have an excess of 1 type of participant. For example, in a cross-sectional study of patients with the avian influenza strain H5N1 (ie, “bird flu”), researchers should randomly select participants from different sexes, ethnic groups, and living conditions. 

By the time the study sample is composed, researchers should have determined their dependent variables, or outcomes (eg, temperature, blood pressure, number of emergency room visits). Figure 1 lists possible data sources for novice researchers, especially for those conducting cross-sectional research. 

For example, questionnaires or surveys can be free, purchased, and experimenter developed. Researchers should use caution when using experimenter-developed surveys, however. Without adequate reliability and validity (discussed in the following section), the survey results will probably be meaningless. Reliability and validation analyses should be conducted before research use. Additional considerations for survey-based studies are discussed in the Sidebar. 

One-on-one interviews can be used as a follow-up to surveys or questionnaires. More detailed information can be gained from structured one-on-one interviews. Structured interviews involve a preplanned series of questions with alternate interview routes based on participants' responses. For example, this type of interview was used by 1 of the authors (D.P.Y.) to determine the effectiveness of a self-instructional board review course and ways in which it could be improved. 

Focus groups provide a safe environment for a small group to examine a problem or plan at a deep level. Such groups can be used to examine possible new directions for a hospital or to plan evaluation research. In the United States, focus groups are commonly used in consumer research. However, they are a potentially useful tool for medical research as well. Although focus groups for medical research seem to be more common outside the United States,¹⁵ they are receiving attention within US medical school curricula¹⁶ and were recently used in a US community-based study assessing a rural community's involvement in testing a computer-based informed consent process.¹⁷ 

Once researchers have selected a research design, composed their sample, and identified the variables they will be using, they are ready to develop their study's protocol, or how they will conduct the study. 

Three major concerns must be kept in mind during the planning and research conduct phases: reliability, validity, and blinding. 

Reliability means consistency or precision: Would researchers get the same results if they repeated the measurement? Appropriate measures of reliability are specific to different situations, such as the reliability of a survey form, a pre- or posttest measure, or an instrument (eg, blood-pressure cuff). 

Researchers can take various steps to ensure that their study is reliable. For example, instruments must be calibrated in accordance with manufacturer's instructions or recalibrated if unusual readings are obtained. Blood pressure readings should be taken from the same arm (left or right) of all patients because of potential differences in the side of the body, especially if repeated readings are to be used. 

Reliability can be especially problematic in osteopathic medical research. The process of touching a patient can change the tissue or texture of the area touched, reducing the reliability of repeated measurements.¹⁸ A brief discussion of this issue is provided by Noll.¹⁹ 

Researchers may want to provide reliability data for instrumental readings if sophisticated instruments are used in their study. 

Validity can be thought of as accuracy. It also can be thought of as appropriateness.³ The research study should evaluate what it is purported to evaluate. In addition, the results should be free of bias and generalizable to other settings, places, and populations. 

Validity is at the heart of every research study. Without valid measures applied in a valid design to a valid sample, interpretation will be hindered and the study will be a waste of human, temporal, and fiscal resources.³ 

Researchers should consider 2 types of validity when planning their study: internal and external. Internal validity refers to the accuracy of the study: Is it flawed by internal problems? External validity asks whether the study can be generalized to other sites or populations. 

Classical design problems that may affect the internal validity of the results are described in Table 2. Additional information on these threats to validity is described by Neutens and Rubinson.¹¹ 

To ensure external validity, researchers should ensure that the instrument used in their study is grounded in the literature.¹¹ Does the literature contain examples of the variables to be used? For example, if researchers are assessing the efficacy of different dosages of a drug, they should review previous research for guidance concerning the effect of the dosages they plan to use. 

In addition, the outcomes of a study must be realistic, measurable, and relevant. It may be easy for researchers to determine whether outcomes are realistic and relevant, but in some cases it is difficult to determine whether they are adequately measurable. Adequate measuring can be a problem in OMT research: How do researchers know whether a specific manipulative procedure worked? Asking patients whether they feel better is not very accurate (eg, how much better?). Pain scales and range-of-motion assessments may be useful ways to measure outcomes in OMT research. Of note, pain scales can be an accurate measure of an individual's perception of pain, but perception of pain can vary widely among individuals and might result in a large variance among participants. This variance could mask the real study outcomes. However, absent a valid biophysical measure of pain, validated pain scales may be the best option for researchers conducting OMT research. 

A final consideration concerning experimental design is blinding.³ Blinding is used in most randomized controlled trials. Ideally, both the experimenters and the participants should not know the groups to which participants are assigned. The experimenters should not know because they usually have some idea of the result they are expecting (ie, the research hypothesis). It is possible that the experimenter will use subtle, possibly not conscious, behaviors that could influence the results, such as encouraging the experimental group in some way. Likewise, the participants might behave or react differently if they know the group to which they have been assigned. For example, if participants in a drug trial knew they were receiving the experimental drug, they might be optimistic and report more improvement to symptoms, or they might report higher rates of side effects. Conversely, if participants knew they were receiving a standard treatment or placebo, they might feel discouraged and not respond to the intervention. Blinding is especially important in high-stakes trials such as the evaluation of new drugs for cancer, human immunodeficiency virus, or AIDS. 

Researchers should take various steps to ensure that participants do not know which group they are in. If pills or capsules are used, those for the control group must be identical to those in the experimental group so participants cannot tell them apart. For research using procedures, the sham protocol should appear identical (or as close as possible) to the treatment protocol. 

In OMT research, blinding is difficult. A typical sham treatment consists of light touch or massage on the same areas on which OMT is performed. Touch or massage will have some effect; thus, any difference in outcomes would be a result of the OMT treatment. Novice researchers should be aware that OMT research can be complicated; they should seek the assistance of someone experienced in this type of research before they develop their research project. 

Researchers should also take steps to ensure that they adhere to guidelines regarding patient privacy and safety. First, they should have a plan. A data collection form should be used to collect and record data in a systematic way. Only deidentified or nonidentifiable data should be used, unless researchers are doing a follow-up study or need to combine data from different sources. Typically, it is not permissible to use any type of tracking or coding, including recording names, Social Security Numbers, medical record numbers, addresses, or phone numbers. Recording this kind of information is in violation of the Health Insurance Portability and Accountability Act (HIPAA). The purpose of HIPAA is to protect the confidentiality of patients. Researchers should carefully review the documents available on the HIPAA website before developing any study protocol.²¹ 

Electronic data should be stored on a password-protected computer that is located in a locked, limited-access facility. Paper files should be stored separately from any patient records and in a locked cabinet. 

Patient safety must be a paramount concern for any interventional research in which patient safety is a risk. The major role of IRBs is to ensure patient safety. Researchers should work with their IRB during the early stages of research design to obtain guidance on patient safety. 

As previously mentioned, all researchers should seek the assistance of a statistician. In studies in which data are obtained from each participant, spreadsheet software, such as Microsoft Excel, can be used to organize the data. Typically, participant data comprise the rows, and data categories (pretest value, posttest value, etc) comprise the columns. 

Statistical analysis is completed after the data are collected and entered. Statisticians will be able to determine the best statistical analysis program to use. 

Once researchers have addressed key aspects of their study design and methodology, they are ready to develop their research proposal. In all cases, the research proposal is initially submitted to the appropriate IRB. Institutional review boards usually provide formats for the proposal; researchers should obtain the format from the IRB to which the proposal will be submitted and follow it exactly. Statisticians and IRBs are the best resources for developing an effective research proposal. Additional resources for designing a research project and preparing a research proposal are listed in Figure 2. 

Good research is a very exacting process. When developing a research project, novice researchers should ensure they choose the appropriate design for their study, determine the appropriate size and composition of their sample, use appropriate data sources, and ensure the reliability and validity of their research.