# HOD 1700 Exam

## Terms

undefined, object
copy deck
Theory
a model (not a belief) that makes sense of realities, a picture worth a
thousand words, combines observations to create or refute a testable hypothesis, never proven to be true or false-takes one fact/test to dispute it, a way of viewing the world, misconception: people misuse the word that it can be proven true
Hypothesis
can be created from a theory and deductive reasoning, testable, clear and concise, tentative statement about the relationship between two variables, an educated guess that is specified and directed, a hunch about the research, provides specific focus, stated in declarative form
Inductive logic
general conclusions are drawn from specific observations, used in qualitative research, generalizations, process of theory development from the â€œground upâ€ or â€œbottom up,â€ like a funnel, allows openness for new ways of understanding, limited b/c depends on local data and idiosynchratic observations, generalizations often restrictive and hard to apply to broader theory
Deductive logic
formulate hypotheses and gather data to prove or disprove them, gathered from the â€œtop downâ€, derive from theory, helps build meaningful body of knowledge, findings integrated with existing theories
Conceptual definitions
constitutive, uses other words and concepts to describe the variable, like a dictionary- describes things as⬦, describe what variable is being investigated but not the meaning of the variables,
Operational definitions
indicates how the variable is measured or manipulated, describes the â€œoperationsâ€ are performed on the variable, necessary for the meaning of the results b/c each researcher uses diff. operations, sometimes arbitrary and not explicitly stated
Independent variables
the variable that is manipulated, the cause, precedes/influences/predicts the dependent variable
Dependent variable
the variable that changes/is observed, the effect, affected/predicted by the independent variable
Samples
group of subjects/participants from whom the data is collected, subset within the population (represents the pop.), usually describes with an adjective (ex: purposive sampling)
Populations
the general population that research is inferred upon, larger population to whom the results can be generalized
Measures of central tendency
mode-scores that occur most frequently, median- score in the middle of the distribution, mean- authentic average of all scores (concerns related outlying scores), use of a single score to characterize a set of scores, provide statistics that indicate the average or typical score in the distribution
Measures of variability
how the scores scatter around the mean or median, how spread out the dist. of scores is from the mean, how much scatter/dispersion exists in the dist., high variability/variance: large degree of dispersion, small variance/variability: scores are very similar, provides complete description
Range
the difference between the highest and lowest scores, particularly misleading in highly skewed distributions
Standard deviation
average distance of the scores from the mean (average deviation), tells the average variability of the scores, -calculate diff. from mean for each score (deviation scores) -then average these numbers
Correlation
measure of relationship between two or more quantitative variables, variables vary together (value of one can be predicted by knowing value of other)
Correlation coefficient
plot results on graph, number between -1 and +1 that indicates the direction and strength of the relationship btwn 2 variables, shows magnitude//if pos. or neg., if less than 0- negative if more than 0- positive if 0- no correlation and plot will be a circle
Validity
extent to which inferences are appropriate/applicable/meaningful to the inference/study, valid test of _____ for the purpose of _______, canâ€™t have validity w/o having reliability, ex. visual acuity test: valid if testing for eyesight but not for height etc., not a characteristic of a test/study but depends on what used for
Characteristics of Validity
-appropriateness: validity of inference, â€“specific to a particular use/interpretation (no measure valid for all purposes/each interpretation has diff. level of validity, â€“unitary concept: not different types of validity, â€“ overall evaluative judgment: own personal judgment is used to determine extent of validity
Sources of Validity
-evidence based on test content: representative sample of larger/universal domain , -evidence based on internal structure: showing item relationships, evidence based on relationships to other variables: showing relationships to external variables
Reliability
findings are free from error, consistency of scores/measurements, can have reliability without having validity
Types of Reliability
stability: measured by giving the same instrument twice over a period time, equivalence: two forms of the same test, equivalence and stability: two forms given at different times, internal consistency: correlation of items measuring the same trait, donâ€™t need validity to have reliability
Comparative design
(quantitative) differences between groups on a variable of difference, 2 or more groups on a variable, ex: what is the difference between male and female drinking amounts?
Causal-comparative designs
(quantitative) describes the relationship btwn past and future correlations, ex: do students who were privately tutored in junior high have higher academic self esteem than those that were not tutored when they reach high school?, allows you to determine causal relationships without manipulating independent variable when it is not practical or ethical, ex post facto: type of causal comparative design
Cross sectional
information collected from 1 or more sample given at one point in time, studies a phenomenon as it occurs as one time: ex- political surveys (study an attitude/ characteristic of a group), compare diff. age categories of subjects to find developmental differences/relationships: evaluate self concept of students between 6th-12th grade, +: convenient, tentative conclusions about change over time, -: can be a result of differences between subjects in the groups
Longitudinal surveys
same/similar subjects surveyed over time (trend study: greater pop., cohort study: specific pop., panel sample: same individuals surveyed each time data is collected), +: greater depth, -: loss of subjects (lose certain types of subjects), difficulty tracking subjects over time, exoensive
Questionnaires
subjects responses to written items to obtain subjectâ€™s perceptions/attitudes/beliefs/values/perspectives/other traits, -: tendency to respond the same way, faking: deliberately responding inaccurately to please researcher, canâ€™t elaborate, may be biased b/c of low response rate and those that respond might not be representative
Interviews
oral questions and answers recorded, informal in nature, requires a skilled interviewer, +: can see non-verbal responses, greater depth/richness of information, can clarify questions and follow up leads (probing), reduces neutral/no answers, usually have higher return rates, -: expensive, time consuming, high response rate needed to avoid biases, if untrained interviewer can end up leading responses then itâ€™s biased, no anonymity, possible inaccurate recording of responses
Internal Validity
control of extraneous variables (strong if account for plausible threats to dependent variable)
-history: extraneous events
-selection: characteristics of subjects (must have random assignments)
-maturation: changes in subjects over time
-pretesting: effect of taking pretest
-instrumentation: unreliability or change in measurement
-treatment replications: insufficient replication of treatments
-subject attrition: loss of subjects
-statistical regression: change of extreme scores to those closer to the mean
-diffusion of treatment: treatment effecting other groups
-experimenter effects: characteristics/expectations of the experimenter
-subject effects: effects of awareness of being a subject
External validity
: generalizability of results to other subjects/measures/treatments/procedures/settings, credibility and usefulness of findings
-subject effects
-situation: characteristics of the setting (naturally occurring or contrived)
-time: change over time
-treatments: way treatment is conceptualized and administered
-measures: nature/ type of measures used to collect information
Quasi-experimental
design does not have random assignment, useful when subjects are in pre-existing groups
True experimental designs
subjects randomly assigned to different groups, shows causality
Single subject designs (ABAB); multiple baseline design
an individualâ€™s (not group) behavior recorded before and after a treatment, shows internal validity and external validity relatively weak
Characteristics of ABAB:
-reliable measurement
-repeated measurement
-description of conditions
-baseline and treatment conditions
-single-variable rule
Types of multiple baseline designs:
- A-B-A (baseline, treatment, baseline)
- A-B-A withdrawal: treatment removed after implementation: strong causal inference b/c shows lack of extraneous variables
(-: the treatment can last a long time so donâ€™t see change/causality in 2nd baseline test)
multiple baseline design:
more than one subject/behavior/setting (provide better evidence for internal validity and generalizability of the results
Criteria for evaluating Single-Subject research
-reliable measurement of the target behavior: standardized and consistent
-target behavior clearly defined operationally: detailed def. of dependent variable and described how measured operationally
-sufficient measures needed to establish stability in behavior: minimum of 3-4 observations in each phase of the study (especially important at baseline)
-procedure, subjects, settings fully described: b/c external validity weak so usefulness of result depends on
-single standardized treatment used: so same is given each time
-experimenter/observer effects controlled for: show how controlled
-results practically significant
Descriptive statistics
describe data (central tendency, variation, relationships, etc.)
Inferential statistics
use descriptive statistics as the base from which inferences are drawn, specific statistical procedures that accomplish this purpose
Purpose and nature of statistics
draw inferences about a population on the basis of an estimate from a sample
Sampling error
without measuring entire population leads to a degree of uncertainty, the larger the proportion of the population sampled the lower the sampling error, need to estimate level of sampling error relative to inferences being drawn
Measurement error
regardless of sampling error measurements can still be inaccurate b/c of measurement errors (lack of reliability and validity), each measure is somewhat different
~ultimate goal to draw accurate conclusions about the population
Terminology
-Null Hypothesis: no difference between groups, no relationship btwn variables
-Level of Significance: probability of being wrong in rejecting the null hypothesis (known as alpha 0)
-Types of errors:
- Type I: rejecting the null hypothesis when itâ€™s true
-Type II: not rejecting (accepting) the null hypothesis when it is not true
Ethnography
in-depth involvement in a culture to describe naturally occurring behavior, interpretation of cultural patterns and meanings within a culture/ social group: people who share patterns of beliefs, behavior, normative expectations (shared: not individualistic)
Case studies
in depth analysis of a single experience or entity
Types:
-historical organizational
-observational
-life history
-situation analysis
-multicase: several independent entities studied
-multisite: many sites/participants used to develop theory
What are the required elements in good qualitative and quantitative research problem statements?
Quantitative: 1) type of research (descriptive, correlation, experimental) 2) variables and the relationships btwn them (independent and dependent) 3) the subject
Qualitative: 1) central phenomenon 2) participants 3) site
What are statistical significance, effect size, and practical significance? What does each tell us? How do they compare with each other?
Statistical significance: a statistical inference, coefficient that is calculated is a probability different from zero-no relationship
-null hypothesis: statement of no difference/relationship
-level of significance: probability of being wrong in rejecting the null hypothesis
Type I error: rejecting when true
Type II error: not rejecting when not true
-typical levels of significance: .10, .50, .01
-affecting factors: actual differences between groups, degree to which sampling measurement errors exist, size of the sample
Practical significance
importance/meaningfulness of practical value of the correlation, more subjective (judgment made by reader and researcher)
Effect size
quantifying the degree of difference btwn 2 groups, measure of practical significance, difference btwn 2 group means in terms of the control group standard deviation
- .30 small
-.50 moderate
-.75 large
Each consumer of the research must judge the balance between the statistical significance and the practical significance of the statistical results given the context in which the results might be used
Objectives
gain the most information on a particular phenomena, uses almost all non-probability sampling
Non-probability sampling
pragmatically driven, probability of being selected is not known, purpose to sample most knowledgeable/informed subjects, often not possible to use probability sampling techniques due to access, time, resource or financial constraints
Purposeful sampling
selection of particularly informative/useful/knowledgeable participants, a few information rich subjects that are studied in depth
-sampling error: difference btwn observed and true results attributed to using sample instead of population
-sampling bias: attributed to mistake on the part of the researcher
Types of non probability sampling techniques used in qualitative
typical case
extreme case
maximum variation
snowball
critical case
typical case sampling
selecting representative participants (most like others)
extreme case sampling
selecting unique/atypical participants
maximum variation sampling
selecting participants to represent extreme cases, represent both ends of the spectrum on a given subject to provide maximum variation
snowball sampling
selecting participants from recommendations of other participants
critical case sampling
selecting most important participants/dramatic illustration of phenomena to understand phenomena studied
Quantitative Sampling
obtain a group of subjects that will be representative of a larger group of individuals
Types of non-probability used in quantitative
convenience sampling
quota sampling
convenience sampling
select subjects/groups based on availability to the researcher, limited generalizability, used most commonly in educational research
quota sampling
non-random sampling representative of a larger population, used when researcher restrained (canâ€™t use probability sample, but wants somewhat representative study), similar to stratified but non random selection used
Probability sampling
statistically driven and known probability of being selected is known, use random sampling
Types of probability sampling
Simple random sampling
Systematic sampling
Stratified sampling
Proportional stratified sampling
Disproportional stratified sampling
Cluster sampling
Simple random sampling
each member of the populations has the same probability of being selected
Systematic sampling
every nth member of the population is selected
Stratified sampling
subjects selected from strata or groups of the population
Proportional stratified sampling
reflects a proportion of stratum in the population
Disproportional stratified sampling
number of subjects in each stratum does not reflect proportion in population
Cluster sampling
naturally occurring groups are selected when impossible/impractical to sample individuals from population as a whole
So whatâ€™s so good about randomized pretest-posttest experimental control experiments?
-true experimental design calls for comparison groups randomly assigned to different treatments/to a treatment and control condition
-random assignment: each subject has the same probability of being in either the treatment/comparison/control group,
-purpose: equalize the characteristics of subjects in each group/controls selection as a threat to internal validity
-pretest used to equalize groups statistically
-researchers use a pretest with random assignment where there may be small/subtle effects of different treatments, when differential subject attrition is possible, and when there is a need to analyze subgroups who differ on pretest
-subjects can be randomly assigned before or after the pretest
-pretest eliminates possibility that randomly one group is initially much different from the other
-overall credibility is judged not by the design but by how well the researcher controls for outside threats
What counter-argument are you trying to minimize by controlling for threats to
internal validity in your research design?
minimizing argument that another variable (other than the treatment/extraneous) is responsible for the results
How does a researcher gain credibility in qualitative research?
Credibility: believability and trustworthiness (accurate/plausible/consistent/ meaningful)
Proven through:
Triangulation: compares the findings of different techniques, use diff. methods of collecting data, data with different samples, at different times, in different places
Reliability: extent to which what is recorded is what actually occurred, detailed field notes, teams of researchers to ensure comprehensiveness and accuracy, review of field notes by participants (member checking), quotations, tape recorders/photographs
Internal validity: match btwn researcherâ€™s categories and interpretations as true, observer effect of primary concern
External validity: generalizability of findings (usually weak in qualitative), use terms- translatability/comparability
What is triangulation? What is its value?
Compares different approaches to the same question/shows credibility by proving that a pattern occurs in diff. approaches
What are the benefits and drawbacks of â€œmixed methodsâ€ research?
Mixed method design: combines quantitative and qualitative designs
+: provides more comprehensive picture of phenomena emphasizes outcomes (quantitative) and process (qualitative), ex: use of a questionnaire and interviews, use of a scale and focus groups
-: requires researcher to have expertise in both, requires extensive data collections and more sources, one method may be used superficially
Describe the five major sections of a research paper and what is included in each section.
Introduction: quantitative: specific research questions qualitative: general problem statement
Literature Review: quantitative: extensive qualitative: brief
Methods: quantitative: subjects, instruments, procedures qualitative: participants and settings/sites
Results: quantitative: statistical explanations quantitative: narrative descriptions
Discussion: review of results as relate to past research, conclusions drawn, limitations, and strengths
What are the kinds of questions that can and canâ€™t be researched using systematic inquiry?
Canâ€™t: questions of values, religion, beliefs, things that canâ€™t be tested
Can: correlates, causes, effects, relationships, asks specific questions, things of the natural world
Systematic inquiry and purpose
scientific inquiry- search for knowledge using recognized methods of data collection, analysis, and interpretation
Purpose: explain natural phenomena, understand their underlying relationships, use info to predict/influence behavior
-knowledge represented in the form of theories: means of simplifying and understanding complex realities
Characteristics of systematic inquiry
1) objectivity 2) control of bias 3) willingness to change 4) verifiable 5) inductive 6) precise 7) truthful
What are different â€œways of knowingâ€ and why do some â€œcountâ€ more than others?
-personal experience (biased, all are different)
-tradition (based on idealized notions of past)
-authority (can be wrong, sometimes authorities disagree)
-research (systematic, testable, objective, most trusted)
What is the difference between correlation and causality? What is needed to
create a strong argument for causality?
Correlation: relationship btwn 2 or more variables
Causality: implies correlation (correlation does not imply correlation), research design (controlled for extraneous variables) needed to create a strong argument for causality
What is a hypothesis? What are the most important characteristics of a
hypothesis? How is it related to a theory?
Theory: a model (not a belief) that makes sense of realities, a picture worth a
thousand words, combines observations to create or refute a testable hypothesis, never proven to be true or false-takes one fact/test to dispute it, a way of viewing the world, misconception: people misuse the word that it can be proven true Hypothesis: can be created from a theory and deductive reasoning, testable, clear and concise, tentative statement about the relationship between two variables, an educated guess that is specified and directed, a hunch about the research
What are the significance and the purpose of a literature review?
Refining research problems, developing significance for the research, identifying methodological techniques, identifying contradictory findings, developing research hypotheses, learning new information, establish credibility, learn about methods, operational definitions, avoid previous mistakes, instruments to use, significance of problem, method, theoretical framework
Characteristics: current, appropriate level of detail, analyze work, relevance to your study, contradictions/limitations, organized by topic, hypothesis should be aligned
Why do you think we spent so much time in this course discussing research
design rather than statistics?
Statistics take care of themselves, canâ€™t get right statistics if your research design is flawed, harder to design a good project than it is to generate good statistics
Quantitative Research
hard statistical data, structured, formal, specific, replication, ideal studies: random, context: manipulate control setting, researchers: detached and objective, quantitative designs: non-experimental (descriptive, comparative, correlation, causal-comparative) and experimental (true experimental, quasi experimental, single subject), Ingredients: 1) Research type (descriptive, correlation â€“ relationship, experimental â€“ effect 2) subject 3) variable and the relationship between them
Example: What is the effect of computer-assisted instruction on 4th grade student achievement?
Goal is to prove theory
Qualitative Research
long term observation, interpretive, open, few subjects, flexible, general, social construct/context, ideal studies: purposeful most informed, context: naturalistic, researchers: observer highly involved, qualitative designs: case studies, phenomenology, ethnography, grounded theory, Ingredients: 1) site 2) participant 3) central phenomenon

85