HOD 1700 Exam
Terms
undefined, object
copy deck
 Theory

a model (not a belief) that makes sense of realities, a picture worth a
thousand words, combines observations to create or refute a testable hypothesis, never proven to be true or falsetakes one fact/test to dispute it, a way of viewing the world, misconception: people misuse the word that it can be proven true  Hypothesis
 can be created from a theory and deductive reasoning, testable, clear and concise, tentative statement about the relationship between two variables, an educated guess that is specified and directed, a hunch about the research, provides specific focus, stated in declarative form
 Inductive logic
 general conclusions are drawn from specific observations, used in qualitative research, generalizations, process of theory development from the â€œground upâ€ or â€œbottom up,â€ like a funnel, allows openness for new ways of understanding, limited b/c depends on local data and idiosynchratic observations, generalizations often restrictive and hard to apply to broader theory
 Deductive logic
 formulate hypotheses and gather data to prove or disprove them, gathered from the â€œtop downâ€, derive from theory, helps build meaningful body of knowledge, findings integrated with existing theories
 Conceptual definitions
 constitutive, uses other words and concepts to describe the variable, like a dictionary describes things as⬦, describe what variable is being investigated but not the meaning of the variables,
 Operational definitions
 indicates how the variable is measured or manipulated, describes the â€œoperationsâ€ are performed on the variable, necessary for the meaning of the results b/c each researcher uses diff. operations, sometimes arbitrary and not explicitly stated
 Independent variables
 the variable that is manipulated, the cause, precedes/influences/predicts the dependent variable
 Dependent variable
 the variable that changes/is observed, the effect, affected/predicted by the independent variable
 Samples
 group of subjects/participants from whom the data is collected, subset within the population (represents the pop.), usually describes with an adjective (ex: purposive sampling)
 Populations
 the general population that research is inferred upon, larger population to whom the results can be generalized
 Measures of central tendency
 modescores that occur most frequently, median score in the middle of the distribution, mean authentic average of all scores (concerns related outlying scores), use of a single score to characterize a set of scores, provide statistics that indicate the average or typical score in the distribution
 Measures of variability
 how the scores scatter around the mean or median, how spread out the dist. of scores is from the mean, how much scatter/dispersion exists in the dist., high variability/variance: large degree of dispersion, small variance/variability: scores are very similar, provides complete description
 Range
 the difference between the highest and lowest scores, particularly misleading in highly skewed distributions
 Standard deviation
 average distance of the scores from the mean (average deviation), tells the average variability of the scores, calculate diff. from mean for each score (deviation scores) then average these numbers
 Correlation
 measure of relationship between two or more quantitative variables, variables vary together (value of one can be predicted by knowing value of other)
 Correlation coefficient
 plot results on graph, number between 1 and +1 that indicates the direction and strength of the relationship btwn 2 variables, shows magnitude//if pos. or neg., if less than 0 negative if more than 0 positive if 0 no correlation and plot will be a circle
 Validity
 extent to which inferences are appropriate/applicable/meaningful to the inference/study, valid test of _____ for the purpose of _______, canâ€™t have validity w/o having reliability, ex. visual acuity test: valid if testing for eyesight but not for height etc., not a characteristic of a test/study but depends on what used for
 Characteristics of Validity
 appropriateness: validity of inference, â€“specific to a particular use/interpretation (no measure valid for all purposes/each interpretation has diff. level of validity, â€“unitary concept: not different types of validity, â€“ overall evaluative judgment: own personal judgment is used to determine extent of validity
 Sources of Validity
 evidence based on test content: representative sample of larger/universal domain , evidence based on internal structure: showing item relationships, evidence based on relationships to other variables: showing relationships to external variables
 Reliability
 findings are free from error, consistency of scores/measurements, can have reliability without having validity
 Types of Reliability
 stability: measured by giving the same instrument twice over a period time, equivalence: two forms of the same test, equivalence and stability: two forms given at different times, internal consistency: correlation of items measuring the same trait, donâ€™t need validity to have reliability
 Comparative design
 (quantitative) differences between groups on a variable of difference, 2 or more groups on a variable, ex: what is the difference between male and female drinking amounts?
 Causalcomparative designs
 (quantitative) describes the relationship btwn past and future correlations, ex: do students who were privately tutored in junior high have higher academic self esteem than those that were not tutored when they reach high school?, allows you to determine causal relationships without manipulating independent variable when it is not practical or ethical, ex post facto: type of causal comparative design
 Cross sectional
 information collected from 1 or more sample given at one point in time, studies a phenomenon as it occurs as one time: ex political surveys (study an attitude/ characteristic of a group), compare diff. age categories of subjects to find developmental differences/relationships: evaluate self concept of students between 6th12th grade, +: convenient, tentative conclusions about change over time, : can be a result of differences between subjects in the groups
 Longitudinal surveys
 same/similar subjects surveyed over time (trend study: greater pop., cohort study: specific pop., panel sample: same individuals surveyed each time data is collected), +: greater depth, : loss of subjects (lose certain types of subjects), difficulty tracking subjects over time, exoensive
 Questionnaires
 subjects responses to written items to obtain subjectâ€™s perceptions/attitudes/beliefs/values/perspectives/other traits, : tendency to respond the same way, faking: deliberately responding inaccurately to please researcher, canâ€™t elaborate, may be biased b/c of low response rate and those that respond might not be representative
 Interviews
 oral questions and answers recorded, informal in nature, requires a skilled interviewer, +: can see nonverbal responses, greater depth/richness of information, can clarify questions and follow up leads (probing), reduces neutral/no answers, usually have higher return rates, : expensive, time consuming, high response rate needed to avoid biases, if untrained interviewer can end up leading responses then itâ€™s biased, no anonymity, possible inaccurate recording of responses
 Internal Validity

control of extraneous variables (strong if account for plausible threats to dependent variable)
history: extraneous events
selection: characteristics of subjects (must have random assignments)
maturation: changes in subjects over time
pretesting: effect of taking pretest
instrumentation: unreliability or change in measurement
treatment replications: insufficient replication of treatments
subject attrition: loss of subjects
statistical regression: change of extreme scores to those closer to the mean
diffusion of treatment: treatment effecting other groups
experimenter effects: characteristics/expectations of the experimenter
subject effects: effects of awareness of being a subject  External validity

: generalizability of results to other subjects/measures/treatments/procedures/settings, credibility and usefulness of findings
subject effects
situation: characteristics of the setting (naturally occurring or contrived)
time: change over time
treatments: way treatment is conceptualized and administered
measures: nature/ type of measures used to collect information  Quasiexperimental
 design does not have random assignment, useful when subjects are in preexisting groups
 True experimental designs
 subjects randomly assigned to different groups, shows causality
 Single subject designs (ABAB); multiple baseline design
 an individualâ€™s (not group) behavior recorded before and after a treatment, shows internal validity and external validity relatively weak
 Characteristics of ABAB:

reliable measurement
repeated measurement
description of conditions
baseline and treatment conditions
singlevariable rule  Types of multiple baseline designs:

 ABA (baseline, treatment, baseline)
 ABA withdrawal: treatment removed after implementation: strong causal inference b/c shows lack of extraneous variables
(: the treatment can last a long time so donâ€™t see change/causality in 2nd baseline test)  multiple baseline design:
 more than one subject/behavior/setting (provide better evidence for internal validity and generalizability of the results
 Criteria for evaluating SingleSubject research

reliable measurement of the target behavior: standardized and consistent
target behavior clearly defined operationally: detailed def. of dependent variable and described how measured operationally
sufficient measures needed to establish stability in behavior: minimum of 34 observations in each phase of the study (especially important at baseline)
procedure, subjects, settings fully described: b/c external validity weak so usefulness of result depends on
single standardized treatment used: so same is given each time
experimenter/observer effects controlled for: show how controlled
results practically significant  Descriptive statistics
 describe data (central tendency, variation, relationships, etc.)
 Inferential statistics
 use descriptive statistics as the base from which inferences are drawn, specific statistical procedures that accomplish this purpose
 Purpose and nature of statistics
 draw inferences about a population on the basis of an estimate from a sample
 Sampling error
 without measuring entire population leads to a degree of uncertainty, the larger the proportion of the population sampled the lower the sampling error, need to estimate level of sampling error relative to inferences being drawn
 Measurement error

regardless of sampling error measurements can still be inaccurate b/c of measurement errors (lack of reliability and validity), each measure is somewhat different
~ultimate goal to draw accurate conclusions about the population  Terminology

Null Hypothesis: no difference between groups, no relationship btwn variables
Level of Significance: probability of being wrong in rejecting the null hypothesis (known as alpha 0)
Types of errors:
 Type I: rejecting the null hypothesis when itâ€™s true
Type II: not rejecting (accepting) the null hypothesis when it is not true  Ethnography
 indepth involvement in a culture to describe naturally occurring behavior, interpretation of cultural patterns and meanings within a culture/ social group: people who share patterns of beliefs, behavior, normative expectations (shared: not individualistic)
 Case studies

in depth analysis of a single experience or entity
Types:
historical organizational
observational
life history
situation analysis
multicase: several independent entities studied
multisite: many sites/participants used to develop theory  What are the required elements in good qualitative and quantitative research problem statements?

Quantitative: 1) type of research (descriptive, correlation, experimental) 2) variables and the relationships btwn them (independent and dependent) 3) the subject
Qualitative: 1) central phenomenon 2) participants 3) site  What are statistical significance, effect size, and practical significance? What does each tell us? How do they compare with each other?

Statistical significance: a statistical inference, coefficient that is calculated is a probability different from zerono relationship
null hypothesis: statement of no difference/relationship
level of significance: probability of being wrong in rejecting the null hypothesis
Type I error: rejecting when true
Type II error: not rejecting when not true
typical levels of significance: .10, .50, .01
affecting factors: actual differences between groups, degree to which sampling measurement errors exist, size of the sample  Practical significance
 importance/meaningfulness of practical value of the correlation, more subjective (judgment made by reader and researcher)
 Effect size

quantifying the degree of difference btwn 2 groups, measure of practical significance, difference btwn 2 group means in terms of the control group standard deviation
 .30 small
.50 moderate
.75 large
Each consumer of the research must judge the balance between the statistical significance and the practical significance of the statistical results given the context in which the results might be used  Objectives
 gain the most information on a particular phenomena, uses almost all nonprobability sampling
 Nonprobability sampling
 pragmatically driven, probability of being selected is not known, purpose to sample most knowledgeable/informed subjects, often not possible to use probability sampling techniques due to access, time, resource or financial constraints
 Purposeful sampling

selection of particularly informative/useful/knowledgeable participants, a few information rich subjects that are studied in depth
sampling error: difference btwn observed and true results attributed to using sample instead of population
sampling bias: attributed to mistake on the part of the researcher  Types of non probability sampling techniques used in qualitative

typical case
extreme case
maximum variation
snowball
critical case  typical case sampling
 selecting representative participants (most like others)
 extreme case sampling
 selecting unique/atypical participants
 maximum variation sampling
 selecting participants to represent extreme cases, represent both ends of the spectrum on a given subject to provide maximum variation
 snowball sampling
 selecting participants from recommendations of other participants
 critical case sampling
 selecting most important participants/dramatic illustration of phenomena to understand phenomena studied
 Quantitative Sampling
 obtain a group of subjects that will be representative of a larger group of individuals
 Types of nonprobability used in quantitative

convenience sampling
quota sampling  convenience sampling
 select subjects/groups based on availability to the researcher, limited generalizability, used most commonly in educational research
 quota sampling
 nonrandom sampling representative of a larger population, used when researcher restrained (canâ€™t use probability sample, but wants somewhat representative study), similar to stratified but non random selection used
 Probability sampling
 statistically driven and known probability of being selected is known, use random sampling
 Types of probability sampling

Simple random sampling
Systematic sampling
Stratified sampling
Proportional stratified sampling
Disproportional stratified sampling
Cluster sampling  Simple random sampling
 each member of the populations has the same probability of being selected
 Systematic sampling
 every nth member of the population is selected
 Stratified sampling
 subjects selected from strata or groups of the population
 Proportional stratified sampling
 reflects a proportion of stratum in the population
 Disproportional stratified sampling
 number of subjects in each stratum does not reflect proportion in population
 Cluster sampling
 naturally occurring groups are selected when impossible/impractical to sample individuals from population as a whole
 So whatâ€™s so good about randomized pretestposttest experimental control experiments?

true experimental design calls for comparison groups randomly assigned to different treatments/to a treatment and control condition
random assignment: each subject has the same probability of being in either the treatment/comparison/control group,
purpose: equalize the characteristics of subjects in each group/controls selection as a threat to internal validity
pretest used to equalize groups statistically
researchers use a pretest with random assignment where there may be small/subtle effects of different treatments, when differential subject attrition is possible, and when there is a need to analyze subgroups who differ on pretest
subjects can be randomly assigned before or after the pretest
pretest eliminates possibility that randomly one group is initially much different from the other
overall credibility is judged not by the design but by how well the researcher controls for outside threats 
What counterargument are you trying to minimize by controlling for threats to
internal validity in your research design?  minimizing argument that another variable (other than the treatment/extraneous) is responsible for the results
 How does a researcher gain credibility in qualitative research?

Credibility: believability and trustworthiness (accurate/plausible/consistent/ meaningful)
Proven through:
Triangulation: compares the findings of different techniques, use diff. methods of collecting data, data with different samples, at different times, in different places
Reliability: extent to which what is recorded is what actually occurred, detailed field notes, teams of researchers to ensure comprehensiveness and accuracy, review of field notes by participants (member checking), quotations, tape recorders/photographs
Internal validity: match btwn researcherâ€™s categories and interpretations as true, observer effect of primary concern
External validity: generalizability of findings (usually weak in qualitative), use terms translatability/comparability  What is triangulation? What is its value?
 Compares different approaches to the same question/shows credibility by proving that a pattern occurs in diff. approaches
 What are the benefits and drawbacks of â€œmixed methodsâ€ research?

Mixed method design: combines quantitative and qualitative designs
+: provides more comprehensive picture of phenomena emphasizes outcomes (quantitative) and process (qualitative), ex: use of a questionnaire and interviews, use of a scale and focus groups
: requires researcher to have expertise in both, requires extensive data collections and more sources, one method may be used superficially  Describe the five major sections of a research paper and what is included in each section.

Introduction: quantitative: specific research questions qualitative: general problem statement
Literature Review: quantitative: extensive qualitative: brief
Methods: quantitative: subjects, instruments, procedures qualitative: participants and settings/sites
Results: quantitative: statistical explanations quantitative: narrative descriptions
Discussion: review of results as relate to past research, conclusions drawn, limitations, and strengths  What are the kinds of questions that can and canâ€™t be researched using systematic inquiry?

Canâ€™t: questions of values, religion, beliefs, things that canâ€™t be tested
Can: correlates, causes, effects, relationships, asks specific questions, things of the natural world  Systematic inquiry and purpose

scientific inquiry search for knowledge using recognized methods of data collection, analysis, and interpretation
Purpose: explain natural phenomena, understand their underlying relationships, use info to predict/influence behavior
knowledge represented in the form of theories: means of simplifying and understanding complex realities  Characteristics of systematic inquiry
 1) objectivity 2) control of bias 3) willingness to change 4) verifiable 5) inductive 6) precise 7) truthful
 What are different â€œways of knowingâ€ and why do some â€œcountâ€ more than others?

personal experience (biased, all are different)
tradition (based on idealized notions of past)
authority (can be wrong, sometimes authorities disagree)
research (systematic, testable, objective, most trusted) 
What is the difference between correlation and causality? What is needed to
create a strong argument for causality? 
Correlation: relationship btwn 2 or more variables
Causality: implies correlation (correlation does not imply correlation), research design (controlled for extraneous variables) needed to create a strong argument for causality 
What is a hypothesis? What are the most important characteristics of a
hypothesis? How is it related to a theory? 
Theory: a model (not a belief) that makes sense of realities, a picture worth a
thousand words, combines observations to create or refute a testable hypothesis, never proven to be true or falsetakes one fact/test to dispute it, a way of viewing the world, misconception: people misuse the word that it can be proven true Hypothesis: can be created from a theory and deductive reasoning, testable, clear and concise, tentative statement about the relationship between two variables, an educated guess that is specified and directed, a hunch about the research  What are the significance and the purpose of a literature review?

Refining research problems, developing significance for the research, identifying methodological techniques, identifying contradictory findings, developing research hypotheses, learning new information, establish credibility, learn about methods, operational definitions, avoid previous mistakes, instruments to use, significance of problem, method, theoretical framework
Characteristics: current, appropriate level of detail, analyze work, relevance to your study, contradictions/limitations, organized by topic, hypothesis should be aligned 
Why do you think we spent so much time in this course discussing research
design rather than statistics?  Statistics take care of themselves, canâ€™t get right statistics if your research design is flawed, harder to design a good project than it is to generate good statistics
 Quantitative Research

hard statistical data, structured, formal, specific, replication, ideal studies: random, context: manipulate control setting, researchers: detached and objective, quantitative designs: nonexperimental (descriptive, comparative, correlation, causalcomparative) and experimental (true experimental, quasi experimental, single subject), Ingredients: 1) Research type (descriptive, correlation â€“ relationship, experimental â€“ effect 2) subject 3) variable and the relationship between them
Example: What is the effect of computerassisted instruction on 4th grade student achievement?
Goal is to prove theory  Qualitative Research
 long term observation, interpretive, open, few subjects, flexible, general, social construct/context, ideal studies: purposeful most informed, context: naturalistic, researchers: observer highly involved, qualitative designs: case studies, phenomenology, ethnography, grounded theory, Ingredients: 1) site 2) participant 3) central phenomenon