A Research Primer, Part 2: Guidelines for Developing a Research Project

David P. Yens; Grace D. Brannan; Jane Z. Dumsha

doi:10.7556/jaoa.2014.006

Abstract

In this article, the second in a series on the basic concepts of research, the authors review aspects of research design including participant considerations, randomization, reliability and validity of measurements, and data collection and management. The authors also discuss considerations for research using questionnaires and tests. The goal of this article is to assist the novice researcher in identifying potential problems that must be addressed during the design of a research project.

The osteopathic medical profession has seen small advances in its research culture in recent years. A few osteopathic medical schools have developed research programs, and The Osteopathic Research Center at the University of North Texas Health Science Center was established in 2001. In addition, research projects by osteopathic residents are encouraged and supported by Osteopathic Postdoctoral Training Institutions.¹ However, there is a critical need in osteopathic medicine to increase the amount of research being conducted, especially that focusing on osteopathic principles and practice. The profession needs to encourage research by more people, both within educational establishments and by community practitioners. We believe that many physicians would be willing to conduct research if they were more familiar with the research process.

The purpose of the present article, the second in a series on research,² is to provide novice researchers an accessible introduction to developing a research project. Although research can cover a wide spectrum of activities, such as biochemical research and research with animals, we focus on human subject research.

Study Design

Once researchers have an idea for a research study, they need to determine how the study will be executed. Study designs provide guidelines about how a study can be planned, but many studies are hybrids or approximations of these designs.³ Study designs are categorized in different ways; for the purpose of simplification, we will review considerations for studies that involve interventions and those that do not.⁴ Descriptions of common study designs are provided in Table 1.

Table 1.

Common Research Study Designs⁴

Study Design	Definition	Examples
Cross-Sectional Study^a	Used to determine relationships between risk factors and outcomes concurrently. All data are gathered at the same time; it has been described as similar to a cohort study, but with all measurements made at about the same time and no follow-up period.	To determine whether drinking high-calorie sodas is related to obesity, a large number of high school students are given a survey that asks their height and weight and how many sodas they drink.
Case-Control Study	A sample is taken from a population that has a disease, another sample is taken from the population at risk for the disease, and predictor variables are measured.	A sample of people with lung cancer is obtained, a second sample of smokers who do not have lung cancer is obtained, and the groups are compared with regard to differences in the predictors or the antecedent conditions.
Cohort Study	Characteristics that might be related to an outcome of interest are measured in a group of participants. Cohort studies can be either prospective or retrospective.	The classic British study that assessed health outcomes of physician smokers with physician nonsmokers over a 20-year period.⁵
Prospective	The group is observed over time to see whether the outcome occurs.	The Framingham study, in which many characteristics were measured and, in subsequent years, related to cardiovascular events (http://www.framinghamheartstudy.org).
Retrospective	Characteristics that existed in the past are examined as they relate to current outcomes.	A history of smoking could be related to present instance of lung cancer.
Randomized Controlled Trial	An intervention is performed on an experimental group and a control group (or more than 1 of either) and outcomes are measured.	To evaluate a new drug, 2 groups are formed: an experimental group that receives the drug and a control group that receives a placebo. Potential participants are randomly assigned to 1 group or the other. The trial continues for a predefined period, and outcomes of interest are compared between the 2 groups.
		The study by Eisenhart et al⁶ evaluated the use of osteopathic manipulative treatment in the management of ankle sprains.
Factorial Designs	These studies extend beyond the basic randomized controlled trial by adding other factors, such as sex or race, to the analysis. Researchers must be careful to recruit and randomize participants from these different groups.	A study of the efficacy of different dosages of a new drug includes analyses of the results by sex and race.

a Survey methodology is commonly used for this design.

View Large

Table 1.

Common Research Study Designs⁴

Study Design	Definition	Examples
Cross-Sectional Study^a	Used to determine relationships between risk factors and outcomes concurrently. All data are gathered at the same time; it has been described as similar to a cohort study, but with all measurements made at about the same time and no follow-up period.	To determine whether drinking high-calorie sodas is related to obesity, a large number of high school students are given a survey that asks their height and weight and how many sodas they drink.
Case-Control Study	A sample is taken from a population that has a disease, another sample is taken from the population at risk for the disease, and predictor variables are measured.	A sample of people with lung cancer is obtained, a second sample of smokers who do not have lung cancer is obtained, and the groups are compared with regard to differences in the predictors or the antecedent conditions.
Cohort Study	Characteristics that might be related to an outcome of interest are measured in a group of participants. Cohort studies can be either prospective or retrospective.	The classic British study that assessed health outcomes of physician smokers with physician nonsmokers over a 20-year period.⁵
Prospective	The group is observed over time to see whether the outcome occurs.	The Framingham study, in which many characteristics were measured and, in subsequent years, related to cardiovascular events (http://www.framinghamheartstudy.org).
Retrospective	Characteristics that existed in the past are examined as they relate to current outcomes.	A history of smoking could be related to present instance of lung cancer.
Randomized Controlled Trial	An intervention is performed on an experimental group and a control group (or more than 1 of either) and outcomes are measured.	To evaluate a new drug, 2 groups are formed: an experimental group that receives the drug and a control group that receives a placebo. Potential participants are randomly assigned to 1 group or the other. The trial continues for a predefined period, and outcomes of interest are compared between the 2 groups.
		The study by Eisenhart et al⁶ evaluated the use of osteopathic manipulative treatment in the management of ankle sprains.
Factorial Designs	These studies extend beyond the basic randomized controlled trial by adding other factors, such as sex or race, to the analysis. Researchers must be careful to recruit and randomize participants from these different groups.	A study of the efficacy of different dosages of a new drug includes analyses of the results by sex and race.

a Survey methodology is commonly used for this design.

Noninterventional Studies

Noninterventional, or observational, studies involve data collected without modifying the environment, introducing a treatment, or testing a new device. Examples include cross-sectional, case-control, and cohort studies. The researcher typically gathers information on factors that may be related to a specific condition or outcome.⁷ Noninterventional studies may be prospective or retrospective, and patient interactions may or may not be involved (eg, surveys or interviews vs medical record reviews). Each of the types of noninterventional designs has its strengths and weaknesses.^3(p120) Overall, it is not possible to make strong causal inferences based on the results of noninterventional studies because randomization is not possible. These designs are useful for gaining information about differences that could be used to design a subsequent interventional design.

Interventional Studies

In interventional studies, such as randomized controlled trials (RCTs), a group of participants receives a treatment or intervention so that researchers may determine whether that treatment or intervention is safe and effective or efficacious. This type of study is also called a clinical trial and can be either controlled or uncontrolled. Because of the nature of this study design, trials are always prospective. To ensure adequate reporting of RCTs, researchers should consult the CONSORT, or Consolidated Standards of Reporting Trials, guidelines.⁸ Included in the guidelines are a checklist and flow diagram. Of note, an RCT may need to be registered with a public registry (eg, ClinicalTrials.gov) before participants are enrolled in the study. Institutional review boards will assist with this registration.

Designs for interventional studies require more steps than those of other types of studies. At the outset, researchers need to address the study population, number of participants, randomization, hypotheses concerning the anticipated outcomes, and statistical analyses required to determine whether the results have adequate validity. Interventional studies provide the strongest causal inferences and tests of hypotheses with minimal bias, but they also can be time consuming, be expensive, and require special care to ensure validity of the outcome.

A research project can be developed by using these 2 types of designs individually or in sequence. That is, a researcher can start with a noninterventional study and then continue with an interventional trial. Hulley et al³ provide the following example:

A noninterventional study would ask:

What is the average number of servings of fish per week in the diet of Americans with a history of coronary heart disease (CHD)?

This noninterventional study could be followed by an evaluation of associations on the same topic:

Is there an association between fish intake and risk of recurrent myocardial infarction in people with a history of CHD?

Finally, this evaluation of associations could evolve into an interventional trial that asks:

Does treatment with fish oil capsules reduce total mortality in people with CHD?

Researchers should choose the study design that will best answer their research question. For the remainder of the article, we focus on specific considerations for interventional studies.

Research Hypothesis

For interventional studies, it is necessary to state at least 1 hypothesis concerning the anticipated outcomes. This step is required to permit an appropriate statistical analysis and to ensure that the research is properly focused. The research hypothesis must be able to be tested. What is tested by statistics is the null hypothesis. A null hypothesis is typically a statement that assumes no difference between groups.⁷ For example, researchers in a study of analgesic use could state the following hypotheses:

Research hypothesis: Analgesics are prescribed for more women than men.
Null hypothesis: The same quantity of analgesics is prescribed for women and men.

The statistical test determines whether or not the null hypothesis can be rejected. At its simplest, if a statistical test of the difference between 2 means (ie, the mean quantity of analgesics prescribed for men vs the mean quantity prescribed for women) is statistically significant (eg, P<.05), then the null hypothesis can be rejected and it is reasonable to conclude that a difference in prescribing patterns exists. Statistical analyses can be much more complicated, however, so novice researchers should seek the assistance of a biostatistician early in the study design process.

Participants

Several other design considerations, such as how many participants are required to conduct a valid study, how participants are to be allocated to different groups, and how to determine inclusion and exclusion criteria must be taken into account as researchers design their study.

Sample Size Determination

To determine the number of participants needed for a valid result, researchers need to establish some initial information, including the expected mean for each of the study groups, the expected standard deviation for each of the groups, and the anticipated or desired effect size, or expected change in outcomes.⁷ The expected mean, standard deviation, and effect size are estimated assuming the results will be what the researchers are anticipating.

Researchers should work with their statistician to determine these parameters and estimate the sample size necessary to achieve a given power.

Composition of the Sample

Study Population

Participants for all groups of a study must be drawn from the same population; they may be patients from a certain hospital or hospital group, patients with a specific disease, or patients with a particular stage of a disease.

Composition of the study population can also depend on how far researchers want to generalize the results. If researchers are not concerned about generalizing beyond their site, then they may select a representative sample of the patients or participants at their location. If researchers want to generalize to a larger population, their sample should be representative of their state or country. It may be hard for researchers to obtain a diverse sample if they are drawing their sample from a homogeneous population. However, researchers should try to recruit a study sample with a balance of participants of different ages, sexes, and major ethnic groups (unless, of course, the study is focused on a specific age, sex, or ethnic group). Results cannot be generalized if most of the people in a sample are white, for example. Of note, the National Institutes of Health requires a representative sample of sexes and ethnic groups for studies it funds.⁹

Study Criteria

Most studies specify inclusion and exclusion criteria for participant selection. For example, Coglianese et al¹⁰ identified the desired composition of their study sample:

[Participants] from the Framingham Heart Study who were 50 to 65 years old and free of [heart failure].

For this study, inclusion criteria were participation in the Framingham Heart Study and ages 50 to 65 years, and exclusion criterion was history of heart failure. Exclusion criteria often include medications or conditions that may affect the outcome of the study. Researchers conducting a study of the use of OMT for back pain would likely exclude those with fractures of the spine, osteoporosis, and other conditions that could be exacerbated by OMT.

Study Groups

Researchers also need to determine the number of groups needed for their study and whether a control group should be used. For example, if researchers are comparing the effects of 3 drugs, their study may need 3 groups. If the researchers would like to compare the effects of the 3 drugs with no treatment or standard treatment, a control group is also needed, or 4 groups total.

A control group, or a group that does not receive an intervention but receives standard or no treatment, is used to determine whether an effect is caused by an intervention or by some other factor, such as spontaneous remission, time, or an external influence. A control group also permits a better statistical analysis to demonstrate effectiveness.

The number of independent variables in a study, or variables that are being manipulated, will affect the number of groups needed. Typically, researchers will have 1 independent variable, such as drug dosage, and 1 moderator variable (a secondary independent variable, such as sex). A study can have 2 independent variables, such as drug type and dosage, and add the moderator variable for a 3-way design. With this type of study, however, researchers need a large total number of participants to maintain a reasonable number per group. For example, researchers conducting a study of 2 drugs would need to have 2 groups. If the researchers would like to study 2 dosages for each drug, they would then need 4 groups. If sex is added as a moderator variable, then the researchers need 8 groups. If 10 participants are needed for each group, then 80 participants are needed. Researchers may need to determine how many participants they can access to determine the number of groups that their study can support. They should enlist the help of a statistician.

Randomization

After the study groups are established, the participants are randomly assigned to the groups. Randomization³ is essential for interventional studies to ensure that groups are equivalent. The assumption is that any individual has an equal chance of ending up in any of the groups.

Probability sampling^3,11 is a valid randomization method that meets the criteria for effective randomization. The preferred method is simple random sampling, in which participants are assigned to groups by use of a random number generator (eg, if the last digit of the random number is even, the participant is assigned to group 1; if the last digit is odd, the participant is assigned to group 2). Simple random samples can also be obtained by drawing a colored marble from a bowl (eg, blue for group 1, red for group 2) and other similar procedures.

Nonprobability (convenience) sampling³ is not true randomization; researchers should use this sampling method with caution and clearly describe their sampling procedure in their research proposal or in any publication. The following are common types of nonprobability sampling:

Consecutive sampling: Participants are alternately assigned to groups as they register for the study (eg, the first participant is placed in group A, the second in group B, the third in group A, etc). This method is not truly random; a participant's placement into a group is determined by his or her place in the sequence.
Convenience sampling: Only those participants who are readily available at the time of the study are included in the randomization. Patients in a particular ward of a hospital could be considered a convenience sample. Researchers can randomize within a convenience sample, but it is difficult to generalize the results to any other setting because the same results may not occur in a different setting.
Judgmental sampling: Participants are assigned to groups at the whim of the investigator. Even with well-meaning investigators, this type of sampling can introduce serious bias into the study. For example, sicker patients may be assigned to the experimental group, while less ill patients may be assigned to the control group.

For observational studies, it is important for researchers to ensure that they have a random sample of the participants so that they do not have an excess of 1 type of participant. For example, in a cross-sectional study of patients with the avian influenza strain H5N1 (ie, “bird flu”), researchers should randomly select participants from different sexes, ethnic groups, and living conditions.

Data Sources

By the time the study sample is composed, researchers should have determined their dependent variables, or outcomes (eg, temperature, blood pressure, number of emergency room visits). Figure 1 lists possible data sources for novice researchers, especially for those conducting cross-sectional research.

Figure 1.

Potential data sources for research projects.

View Original | Slide (.ppt)

For example, questionnaires or surveys can be free, purchased, and experimenter developed. Researchers should use caution when using experimenter-developed surveys, however. Without adequate reliability and validity (discussed in the following section), the survey results will probably be meaningless. Reliability and validation analyses should be conducted before research use. Additional considerations for survey-based studies are discussed in the Sidebar.

One-on-one interviews can be used as a follow-up to surveys or questionnaires. More detailed information can be gained from structured one-on-one interviews. Structured interviews involve a preplanned series of questions with alternate interview routes based on participants' responses. For example, this type of interview was used by 1 of the authors (D.P.Y.) to determine the effectiveness of a self-instructional board review course and ways in which it could be improved.

Focus groups provide a safe environment for a small group to examine a problem or plan at a deep level. Such groups can be used to examine possible new directions for a hospital or to plan evaluation research. In the United States, focus groups are commonly used in consumer research. However, they are a potentially useful tool for medical research as well. Although focus groups for medical research seem to be more common outside the United States,¹⁵ they are receiving attention within US medical school curricula¹⁶ and were recently used in a US community-based study assessing a rural community's involvement in testing a computer-based informed consent process.¹⁷

Considerations for the Study Protocol

Once researchers have selected a research design, composed their sample, and identified the variables they will be using, they are ready to develop their study's protocol, or how they will conduct the study.

Three major concerns must be kept in mind during the planning and research conduct phases: reliability, validity, and blinding.

Reliability

Reliability means consistency or precision: Would researchers get the same results if they repeated the measurement? Appropriate measures of reliability are specific to different situations, such as the reliability of a survey form, a pre- or posttest measure, or an instrument (eg, blood-pressure cuff).

Researchers can take various steps to ensure that their study is reliable. For example, instruments must be calibrated in accordance with manufacturer's instructions or recalibrated if unusual readings are obtained. Blood pressure readings should be taken from the same arm (left or right) of all patients because of potential differences in the side of the body, especially if repeated readings are to be used.

Reliability can be especially problematic in osteopathic medical research. The process of touching a patient can change the tissue or texture of the area touched, reducing the reliability of repeated measurements.¹⁸ A brief discussion of this issue is provided by Noll.¹⁹

Researchers may want to provide reliability data for instrumental readings if sophisticated instruments are used in their study.

Validity

Validity can be thought of as accuracy. It also can be thought of as appropriateness.³ The research study should evaluate what it is purported to evaluate. In addition, the results should be free of bias and generalizable to other settings, places, and populations.

Validity is at the heart of every research study. Without valid measures applied in a valid design to a valid sample, interpretation will be hindered and the study will be a waste of human, temporal, and fiscal resources.³

Researchers should consider 2 types of validity when planning their study: internal and external. Internal validity refers to the accuracy of the study: Is it flawed by internal problems? External validity asks whether the study can be generalized to other sites or populations.

Classical design problems that may affect the internal validity of the results are described in Table 2. Additional information on these threats to validity is described by Neutens and Rubinson.¹¹

Table 2.

Design Problems That May Affect the Internal Validity of a Study's Results²⁰

Design Problem	Description	Example
History	External events may affect the experimental and control groups differently, especially if they are evaluated at different times. All trials must be conducted simultaneously.	A researcher tries to compare the results of a new drug trial with the results of a previous trial. However, during the new trial, information is widely published on risks in using the new drug that could influence the participants, making the 2 trials incomparable.
Maturation	Changes may take place in participants during the study that are different between groups and are not related to the treatment. In particular, children and older adults change rapidly in many ways.	A new high-protein diet could not be tested on children because a researcher would not be able to tell how much of the children's gain in weight (if any) is a result of maturation and how much is a result of the treatment.
Testing	In some cases, a premeasure or pretest can have an impact on subsequent evaluations; the control group might be sensitized to the goals of the study, thus impacting their behavior.^a	If drawing blood or performing a spinal tap is part of a pretest, participants with phobias or sensitivity to pain may have a physical reaction to the posttest, affecting the results.
Instrumentation	Changes or modifications in measurement instruments or research methods during the study can affect outcomes.	If instruments are not calibrated or if blood pressures are taken at different sites of the body or at different times, findings may differ.
Statistical regression	The tendency for extreme values, outliers from the mean, to move (regress) toward the mean when measured again. Also known as the floor and ceiling effect. Outliers must be evaluated carefully, and a biostatistician should be consulted.	Extreme groups are included in the analysis and the middle group is eliminated.
Selection	If groups are not equivalent in age, sex, race, or other variables, any differences in the dependent variable are not valid. Randomization must be used for a valid study.	A study that includes only white participants cannot be generalized to populations of other races.
Experimental mortality	If different percentages or numbers of participants in the groups being studied are lost, the resulting groups may not be comparable. Researchers should determine if there is some systematic reason for this differential loss.	In a study of the efficacy of an osteopathic manipulative treatment technique, 25% of the sham treatment group dropped out of the study, compared with 4% of the osteopathic manipulative treatment group. This difference was statistically significant, reducing the comparability of the 2 groups and the study's validity.
Stability	If the findings of the study are unreliable because of variations in the way measures are taken, because the instruments used are not reliable, or because of differences in the application of the study protocol, the study results are not valid.	Different methods are used to determine blood pressure in a study. Each method could have yielded different findings, making the results suspect.

a Not common in medical research.

View Large

Table 2.

Design Problems That May Affect the Internal Validity of a Study's Results²⁰

Design Problem	Description	Example
History	External events may affect the experimental and control groups differently, especially if they are evaluated at different times. All trials must be conducted simultaneously.	A researcher tries to compare the results of a new drug trial with the results of a previous trial. However, during the new trial, information is widely published on risks in using the new drug that could influence the participants, making the 2 trials incomparable.
Maturation	Changes may take place in participants during the study that are different between groups and are not related to the treatment. In particular, children and older adults change rapidly in many ways.	A new high-protein diet could not be tested on children because a researcher would not be able to tell how much of the children's gain in weight (if any) is a result of maturation and how much is a result of the treatment.
Testing	In some cases, a premeasure or pretest can have an impact on subsequent evaluations; the control group might be sensitized to the goals of the study, thus impacting their behavior.^a	If drawing blood or performing a spinal tap is part of a pretest, participants with phobias or sensitivity to pain may have a physical reaction to the posttest, affecting the results.
Instrumentation	Changes or modifications in measurement instruments or research methods during the study can affect outcomes.	If instruments are not calibrated or if blood pressures are taken at different sites of the body or at different times, findings may differ.
Statistical regression	The tendency for extreme values, outliers from the mean, to move (regress) toward the mean when measured again. Also known as the floor and ceiling effect. Outliers must be evaluated carefully, and a biostatistician should be consulted.	Extreme groups are included in the analysis and the middle group is eliminated.
Selection	If groups are not equivalent in age, sex, race, or other variables, any differences in the dependent variable are not valid. Randomization must be used for a valid study.	A study that includes only white participants cannot be generalized to populations of other races.
Experimental mortality	If different percentages or numbers of participants in the groups being studied are lost, the resulting groups may not be comparable. Researchers should determine if there is some systematic reason for this differential loss.	In a study of the efficacy of an osteopathic manipulative treatment technique, 25% of the sham treatment group dropped out of the study, compared with 4% of the osteopathic manipulative treatment group. This difference was statistically significant, reducing the comparability of the 2 groups and the study's validity.
Stability	If the findings of the study are unreliable because of variations in the way measures are taken, because the instruments used are not reliable, or because of differences in the application of the study protocol, the study results are not valid.	Different methods are used to determine blood pressure in a study. Each method could have yielded different findings, making the results suspect.

a Not common in medical research.

To ensure external validity, researchers should ensure that the instrument used in their study is grounded in the literature.¹¹ Does the literature contain examples of the variables to be used? For example, if researchers are assessing the efficacy of different dosages of a drug, they should review previous research for guidance concerning the effect of the dosages they plan to use.

View Original | Slide (.ppt)

In addition, the outcomes of a study must be realistic, measurable, and relevant. It may be easy for researchers to determine whether outcomes are realistic and relevant, but in some cases it is difficult to determine whether they are adequately measurable. Adequate measuring can be a problem in OMT research: How do researchers know whether a specific manipulative procedure worked? Asking patients whether they feel better is not very accurate (eg, how much better?). Pain scales and range-of-motion assessments may be useful ways to measure outcomes in OMT research. Of note, pain scales can be an accurate measure of an individual's perception of pain, but perception of pain can vary widely among individuals and might result in a large variance among participants. This variance could mask the real study outcomes. However, absent a valid biophysical measure of pain, validated pain scales may be the best option for researchers conducting OMT research.

Blinding

A final consideration concerning experimental design is blinding.³ Blinding is used in most randomized controlled trials. Ideally, both the experimenters and the participants should not know the groups to which participants are assigned. The experimenters should not know because they usually have some idea of the result they are expecting (ie, the research hypothesis). It is possible that the experimenter will use subtle, possibly not conscious, behaviors that could influence the results, such as encouraging the experimental group in some way. Likewise, the participants might behave or react differently if they know the group to which they have been assigned. For example, if participants in a drug trial knew they were receiving the experimental drug, they might be optimistic and report more improvement to symptoms, or they might report higher rates of side effects. Conversely, if participants knew they were receiving a standard treatment or placebo, they might feel discouraged and not respond to the intervention. Blinding is especially important in high-stakes trials such as the evaluation of new drugs for cancer, human immunodeficiency virus, or AIDS.

Researchers should take various steps to ensure that participants do not know which group they are in. If pills or capsules are used, those for the control group must be identical to those in the experimental group so participants cannot tell them apart. For research using procedures, the sham protocol should appear identical (or as close as possible) to the treatment protocol.

In OMT research, blinding is difficult. A typical sham treatment consists of light touch or massage on the same areas on which OMT is performed. Touch or massage will have some effect; thus, any difference in outcomes would be a result of the OMT treatment. Novice researchers should be aware that OMT research can be complicated; they should seek the assistance of someone experienced in this type of research before they develop their research project.

Data Collection and Management

Patient Privacy and Safety

Researchers should also take steps to ensure that they adhere to guidelines regarding patient privacy and safety. First, they should have a plan. A data collection form should be used to collect and record data in a systematic way. Only deidentified or nonidentifiable data should be used, unless researchers are doing a follow-up study or need to combine data from different sources. Typically, it is not permissible to use any type of tracking or coding, including recording names, Social Security Numbers, medical record numbers, addresses, or phone numbers. Recording this kind of information is in violation of the Health Insurance Portability and Accountability Act (HIPAA). The purpose of HIPAA is to protect the confidentiality of patients. Researchers should carefully review the documents available on the HIPAA website before developing any study protocol.²¹

Electronic data should be stored on a password-protected computer that is located in a locked, limited-access facility. Paper files should be stored separately from any patient records and in a locked cabinet.

Patient safety must be a paramount concern for any interventional research in which patient safety is a risk. The major role of IRBs is to ensure patient safety. Researchers should work with their IRB during the early stages of research design to obtain guidance on patient safety.

Statistical Analysis

As previously mentioned, all researchers should seek the assistance of a statistician. In studies in which data are obtained from each participant, spreadsheet software, such as Microsoft Excel, can be used to organize the data. Typically, participant data comprise the rows, and data categories (pretest value, posttest value, etc) comprise the columns.

Statistical analysis is completed after the data are collected and entered. Statisticians will be able to determine the best statistical analysis program to use.

Next Steps

Once researchers have addressed key aspects of their study design and methodology, they are ready to develop their research proposal. In all cases, the research proposal is initially submitted to the appropriate IRB. Institutional review boards usually provide formats for the proposal; researchers should obtain the format from the IRB to which the proposal will be submitted and follow it exactly. Statisticians and IRBs are the best resources for developing an effective research proposal. Additional resources for designing a research project and preparing a research proposal are listed in Figure 2.

Figure 2.

Resources for developing a research project.

View Original | Slide (.ppt)

Conclusion

Good research is a very exacting process. When developing a research project, novice researchers should ensure they choose the appropriate design for their study, determine the appropriate size and composition of their sample, use appropriate data sources, and ensure the reliability and validity of their research.

Acknowledgments

We thank the American Association of Colleges of Osteopathic Medicine for its support; Karen Collins, MPA, for her editing assistance; and Godwin Dogbey, PhD, for his critical review and comments.

Financial Disclosures: None reported.

References

1

American Osteopathic Association. AOA Basic Documents for Postdoctoral Training. Chicago, IL: American Osteopathic Association; 2013. http://www.osteopathic.org/inside-aoa/accreditation/postdoctoral-training-approval/postdoctoral-training-standards/Documents/aoa-basic-document-for-postdoctoral-training.pdf. Accessed December 10, 2013.

2

Brannan GD, Dumsha JZ, Yens DP. A research primer: basic guidelines for the novice researcher. J Am Osteopath Assoc. 2013;113(7):556-563. doi:10.7556/jaoa.2013.007. [CrossRef] [PubMed]

3

Hulley SB, Cummings SR, Browner WS, Grady DG, Newman TB. Designing Clinical Research. 3rd ed. Philadelphia, PA: Lippincott Williams & Wilkins; 2007.

4

Katz DL. Clinical Epidemiology & Evidence-Based Medicine: Fundamental Principles of Clinical Reasoning & Research. Thousand Oaks, CA: Sage Publications; 2001:151.

5

Doll R, Peto R, Boreham J, Sutherland I. Mortality in relation to smoking: 50 years' observations on male British doctors [published online June 22, 2004]. BMJ. 2004;328(7455):1519. doi:10.1136/bmj.38142.554479.AE. [CrossRef] [PubMed]

6

Eisenhart AW, Gaeta TJ, Yens DP. Osteopathic manipulative treatment in the emergency department for patients with acute ankle injuries. J Am Osteopath Assoc. 2003;103(9):417-421. [PubMed]

7

AMA Manual of Style: A Guide for Authors and Editors. 10th ed. New York, NY: Oxford University Press, Inc; 2007.

8

The CONSORT Statement. The CONSORT Group website. http://www.consort-statement.org/consort-statement/. Accessed December 8, 2013.

9

National Institutes of Health Office of Extramural Research. Inclusion of women and minorities as participants in research. National Institutes of Health website. http://grants.nih.gov/grants/funding/women_min/women_min.htm. Accessed December 5, 2013.

10

Coglianese EE, Qureshi MM, Vasan RS, Wang TJ, Moore LL. Usefulness of the blood hematocrit level to predict development of heart failure in a community [published online October 12, 2011]. Am J Cardiol. 2012;109(2):241-245. doi:10.1016/j.amjcard.2011.08.037. [CrossRef] [PubMed]

11

Neutens JJ, Rubinson L. Research Techniques for the Health Sciences. 4th ed. San Francisco, CA: Benjamin Cummings; 2010.

12

National Cancer Institute. SEER-Medicare linked database. National Cancer Institute website. http://seer.cancer.gov/about/linked_databases.html. Accessed March 15, 2013.

13

Borg WR, Gall MD. Educational Research: An Introduction. 5th ed. White Plains, NY: Longman Publishing Group; 1989.

14

Plake BS, Spies RA, eds. The Sixteenth Mental Measurements Yearbook. Lincoln, NE: Buros Institute; 2005. Buros Mental Measurements Yearbooks.

15

Kitzinger J. Qualitative research: introducing focus groups. BMJ. 1995;311(7000):299-302. doi:10.1136/bmj.311.7000.299. [CrossRef] [PubMed]

16

Cronin K. Department of family medicine focus group resource guide. University of Wisconsin School of Medicine and Public Health website. https://inside.fammed.wisc.edu/sites/default/files/dmt/focus-group-resource-guide.pdf. Accessed December 5, 2013.

17

Mahnke AN, Plasek JM, Hoffman DGet al. A rural community's involvement in the design and usability testing of a computer-based informed consent process for the personalized medicine research project [published online November 22, 2013]. Am J Med Genet A. doi:10.1002/ajmg.a.36220.

18

Halma KD, Degenhardt BF, Snider KT, Johnson JC, Flaim MS, Bradshaw D. Intraobserver reliability of cranial strain patterns as evaluated by osteopathic physicians: a pilot study. J Am Osteopath Assoc. 2008;108(9):493-502. [PubMed]

19

Noll DR. The effect of OMT on postoperative medical and functional recovery of coronary artery bypass graft patients [letter]. J Am Osteopath Assoc. 2013;113(8):595-596. [CrossRef] [PubMed]

20

Campbell DT, Stanley J. Experimental and Quasi-Experimental Designs for Research. Chicago, IL: Rand-McNally; 1963.

21

HIPAA privacy rule: information for researchers. National Institutes of Health website. http://privacyruleandresearch.nih.gov/pr_08.asp. Accessed April 4, 2013.

Table 1.

Common Research Study Designs⁴

Study Design	Definition	Examples
Cross-Sectional Study^a	Used to determine relationships between risk factors and outcomes concurrently. All data are gathered at the same time; it has been described as similar to a cohort study, but with all measurements made at about the same time and no follow-up period.	To determine whether drinking high-calorie sodas is related to obesity, a large number of high school students are given a survey that asks their height and weight and how many sodas they drink.
Case-Control Study	A sample is taken from a population that has a disease, another sample is taken from the population at risk for the disease, and predictor variables are measured.	A sample of people with lung cancer is obtained, a second sample of smokers who do not have lung cancer is obtained, and the groups are compared with regard to differences in the predictors or the antecedent conditions.
Cohort Study	Characteristics that might be related to an outcome of interest are measured in a group of participants. Cohort studies can be either prospective or retrospective.	The classic British study that assessed health outcomes of physician smokers with physician nonsmokers over a 20-year period.⁵
Prospective	The group is observed over time to see whether the outcome occurs.	The Framingham study, in which many characteristics were measured and, in subsequent years, related to cardiovascular events (http://www.framinghamheartstudy.org).
Retrospective	Characteristics that existed in the past are examined as they relate to current outcomes.	A history of smoking could be related to present instance of lung cancer.
Randomized Controlled Trial	An intervention is performed on an experimental group and a control group (or more than 1 of either) and outcomes are measured.	To evaluate a new drug, 2 groups are formed: an experimental group that receives the drug and a control group that receives a placebo. Potential participants are randomly assigned to 1 group or the other. The trial continues for a predefined period, and outcomes of interest are compared between the 2 groups.
		The study by Eisenhart et al⁶ evaluated the use of osteopathic manipulative treatment in the management of ankle sprains.
Factorial Designs	These studies extend beyond the basic randomized controlled trial by adding other factors, such as sex or race, to the analysis. Researchers must be careful to recruit and randomize participants from these different groups.	A study of the efficacy of different dosages of a new drug includes analyses of the results by sex and race.

a Survey methodology is commonly used for this design.

View Large

Table 1.

Common Research Study Designs⁴

Study Design	Definition	Examples
Cross-Sectional Study^a	Used to determine relationships between risk factors and outcomes concurrently. All data are gathered at the same time; it has been described as similar to a cohort study, but with all measurements made at about the same time and no follow-up period.	To determine whether drinking high-calorie sodas is related to obesity, a large number of high school students are given a survey that asks their height and weight and how many sodas they drink.
Case-Control Study	A sample is taken from a population that has a disease, another sample is taken from the population at risk for the disease, and predictor variables are measured.	A sample of people with lung cancer is obtained, a second sample of smokers who do not have lung cancer is obtained, and the groups are compared with regard to differences in the predictors or the antecedent conditions.
Cohort Study	Characteristics that might be related to an outcome of interest are measured in a group of participants. Cohort studies can be either prospective or retrospective.	The classic British study that assessed health outcomes of physician smokers with physician nonsmokers over a 20-year period.⁵
Prospective	The group is observed over time to see whether the outcome occurs.	The Framingham study, in which many characteristics were measured and, in subsequent years, related to cardiovascular events (http://www.framinghamheartstudy.org).
Retrospective	Characteristics that existed in the past are examined as they relate to current outcomes.	A history of smoking could be related to present instance of lung cancer.
Randomized Controlled Trial	An intervention is performed on an experimental group and a control group (or more than 1 of either) and outcomes are measured.	To evaluate a new drug, 2 groups are formed: an experimental group that receives the drug and a control group that receives a placebo. Potential participants are randomly assigned to 1 group or the other. The trial continues for a predefined period, and outcomes of interest are compared between the 2 groups.
		The study by Eisenhart et al⁶ evaluated the use of osteopathic manipulative treatment in the management of ankle sprains.
Factorial Designs	These studies extend beyond the basic randomized controlled trial by adding other factors, such as sex or race, to the analysis. Researchers must be careful to recruit and randomize participants from these different groups.	A study of the efficacy of different dosages of a new drug includes analyses of the results by sex and race.

a Survey methodology is commonly used for this design.

Table 2.

Design Problems That May Affect the Internal Validity of a Study's Results²⁰

Design Problem	Description	Example
History	External events may affect the experimental and control groups differently, especially if they are evaluated at different times. All trials must be conducted simultaneously.	A researcher tries to compare the results of a new drug trial with the results of a previous trial. However, during the new trial, information is widely published on risks in using the new drug that could influence the participants, making the 2 trials incomparable.
Maturation	Changes may take place in participants during the study that are different between groups and are not related to the treatment. In particular, children and older adults change rapidly in many ways.	A new high-protein diet could not be tested on children because a researcher would not be able to tell how much of the children's gain in weight (if any) is a result of maturation and how much is a result of the treatment.
Testing	In some cases, a premeasure or pretest can have an impact on subsequent evaluations; the control group might be sensitized to the goals of the study, thus impacting their behavior.^a	If drawing blood or performing a spinal tap is part of a pretest, participants with phobias or sensitivity to pain may have a physical reaction to the posttest, affecting the results.
Instrumentation	Changes or modifications in measurement instruments or research methods during the study can affect outcomes.	If instruments are not calibrated or if blood pressures are taken at different sites of the body or at different times, findings may differ.
Statistical regression	The tendency for extreme values, outliers from the mean, to move (regress) toward the mean when measured again. Also known as the floor and ceiling effect. Outliers must be evaluated carefully, and a biostatistician should be consulted.	Extreme groups are included in the analysis and the middle group is eliminated.
Selection	If groups are not equivalent in age, sex, race, or other variables, any differences in the dependent variable are not valid. Randomization must be used for a valid study.	A study that includes only white participants cannot be generalized to populations of other races.
Experimental mortality	If different percentages or numbers of participants in the groups being studied are lost, the resulting groups may not be comparable. Researchers should determine if there is some systematic reason for this differential loss.	In a study of the efficacy of an osteopathic manipulative treatment technique, 25% of the sham treatment group dropped out of the study, compared with 4% of the osteopathic manipulative treatment group. This difference was statistically significant, reducing the comparability of the 2 groups and the study's validity.
Stability	If the findings of the study are unreliable because of variations in the way measures are taken, because the instruments used are not reliable, or because of differences in the application of the study protocol, the study results are not valid.	Different methods are used to determine blood pressure in a study. Each method could have yielded different findings, making the results suspect.

a Not common in medical research.

View Large

Table 2.

Design Problems That May Affect the Internal Validity of a Study's Results²⁰

Design Problem	Description	Example
History	External events may affect the experimental and control groups differently, especially if they are evaluated at different times. All trials must be conducted simultaneously.	A researcher tries to compare the results of a new drug trial with the results of a previous trial. However, during the new trial, information is widely published on risks in using the new drug that could influence the participants, making the 2 trials incomparable.
Maturation	Changes may take place in participants during the study that are different between groups and are not related to the treatment. In particular, children and older adults change rapidly in many ways.	A new high-protein diet could not be tested on children because a researcher would not be able to tell how much of the children's gain in weight (if any) is a result of maturation and how much is a result of the treatment.
Testing	In some cases, a premeasure or pretest can have an impact on subsequent evaluations; the control group might be sensitized to the goals of the study, thus impacting their behavior.^a	If drawing blood or performing a spinal tap is part of a pretest, participants with phobias or sensitivity to pain may have a physical reaction to the posttest, affecting the results.
Instrumentation	Changes or modifications in measurement instruments or research methods during the study can affect outcomes.	If instruments are not calibrated or if blood pressures are taken at different sites of the body or at different times, findings may differ.
Statistical regression	The tendency for extreme values, outliers from the mean, to move (regress) toward the mean when measured again. Also known as the floor and ceiling effect. Outliers must be evaluated carefully, and a biostatistician should be consulted.	Extreme groups are included in the analysis and the middle group is eliminated.
Selection	If groups are not equivalent in age, sex, race, or other variables, any differences in the dependent variable are not valid. Randomization must be used for a valid study.	A study that includes only white participants cannot be generalized to populations of other races.
Experimental mortality	If different percentages or numbers of participants in the groups being studied are lost, the resulting groups may not be comparable. Researchers should determine if there is some systematic reason for this differential loss.	In a study of the efficacy of an osteopathic manipulative treatment technique, 25% of the sham treatment group dropped out of the study, compared with 4% of the osteopathic manipulative treatment group. This difference was statistically significant, reducing the comparability of the 2 groups and the study's validity.
Stability	If the findings of the study are unreliable because of variations in the way measures are taken, because the instruments used are not reliable, or because of differences in the application of the study protocol, the study results are not valid.	Different methods are used to determine blood pressure in a study. Each method could have yielded different findings, making the results suspect.

a Not common in medical research.

A Research Primer, Part 2: Guidelines for Developing a Research Project

Related content

Related Topics