Final Study Guide

Unit I. Approaches to Social Research (Course preview)

Four principal research strategies (experiments, surveys, field and observational research , and use of available data)

concepts and measures (operational definitions)

measurement reliability and validity

independent variable, dependent variable 

The interplay of theory and research

deduction and induction (wheel of science)

Evidence supporting causality (criteria)

1. association

2. direction of influence

3. eliminate rival explanations (control procedures)

Logic of Experimentation

manipulated independent variable (treatment)

random assignment

test of statistical significance

internal and external validity

Reading research reports

units of analysis 

aggregate data

ecological fallacy 

time frame

cross-sectional design 

longitudinal design

trend study

panel study 

cohort study (age, period, cohort effects)

 

Reading tables

positive and negative (inverse) relationships

statistical significance

percentaging and reading tables

multivariate analysis

logic (let one independent variable vary at a time, while controlling the others)

partial regression coefficients (slopes)

dummy variables (binary, dichotomous)

Unit II. From Concepts to Measures

Operational definitions (manipulation operations, measurement operations)  

sources of variation: true values, systematic error (bias), random error

coding responses to open-ended questions

Levels of measurement

nominal

mutually exclusive, collectively exhaustive 

ordinal

interval

ratio 

Reliability assessment: 

stability measures (test-retest) 

equivalence measures (split-half/internal consistency, intercoder reliability) 

Validity assessment: 

1. subjective (content, face)  

2. criterion-related 

3. construct 

a. correlations with related variables
b. convergent validity
c. discriminant validity
d. known groups

 

Unit III. Generalizability of Findings

Reasons for probability sampling (reduced cost, greater speed, better quality, greater flexibility, unbiased selection, and efficient investment of resources)

Initial steps

1. define target population of interest

2. construct sampling frame (list, rule)

Review of statistical inference (population, sample, and sampling distribution) 

parameter, statistic 

standard error (factors affecting sampling error) 

nonsampling errors in surveys (incomplete sampling frame, incomplete data collection, response errors)

Probability sampling designs 

simple random sampling (SRS)

systematic sampling

stratified sampling 

cluster sampling  

area sampling

Nonprobability designs 

convenience sampling

purposive or judgmental sampling 

quota sampling 

snowball sampling

Unit IV. Experiments

Staging Experiments

randomization and matching

experimental manipulation checks

cover story

debriefing

pretesting

experimental and mundane realism

Threats to internal validity

history

maturation

testing effects

instrumentation

statistical regression (regression toward the mean)

selection

attrition (mortality)

differential attrition

External validity issues

“Pre-experimental” designs

1. one-shot case study

          X     Y

2. one group pretest-posttest design
 

Y1     X      Y2
 

3. static-group comparison
 

X    Y1
       Y2
 

True experimental designs

4. pretest-posttest control group design
 

Y1     X      Y2

R 

Y3              Y4
 

5. posttest-only control group design
 

     X      Y1

  R 

              Y2
 

6. Factorial experimental designs

Quasi-experimental designs

Experimentation outside the laboratory

 

Unit V. Survey Research

 

Typical Survey Features

1. Large N (sample size), probability sample

2. Structured (standardized) instrument

Unstructured interviews

3. Sophisticated data analysis

Data-collection Modes

face-to-face interview

computer-assisted personal interviewing

telephone interview

computer-assisted telephone interviewing

random digit dialing

self-administered electronic surveys

E-mail

Web      

Interactive Voice Response

Computer-assisted self-administered interviewing

self-administered questionnaire

 Surveys Compared to Experiments

 Survey Instrument Design

question ordering

opening question

sensitive, routine questions

open vs. closed questions

Writing Effective Questions

1.     lack of clarity or precision

2.     inappropriate vocabulary

3.     double-barreled question

4.     loaded word or leading question

5.      insensitive wording

Pretesting

 Cognitive Interviewing

"Thinkaloud" interviews

Probing questions

Paraphrasing follow-ups

Field Pretesting

Behavioral coding

Respondent debriefings

Interviewer debriefings

Split-panel tests

Response analysis

Analysis and Interpretation of Nonexperimental Data

Modeling Steps

1. Start with theoretical model with causal ordering of variables

2. Compare predictions of model with the data at hand

3. Revise/extend model, etc.

Basic Control Design Principle

Let one independent variable vary at a time (e.g., percentage tables, partial regression coefficients, factorial experimental designs)

Modeling Example: “Elaboration”

Arrow diagrams and empirical requirements, partial tables

Antecedent variables

Intervening variables

Ideal types (outcomes)

explanation

interpretation

specification (interaction)

replication

suppressor variable

distorter variable 

Unit VI. Field Research and Observational Studies

Advantages of Field Research 

1. Study fleeting or dynamic situations

2. Preserve natural order of things in complex settings

3. Does not require expensive resources (money and personnel)

4. When methodological problems, resources, or ethics preclude other research strategies

5. When very little is known about the topic under investigation

Thick description

Limitations of Field Research 

1. Time-consuming and inefficient way to gather certain information (population characteristics; test causal hypotheses)

2. Necessarily limited to a few settings

3. Highly dependent on observational skills of the researcher

Observation: degree of structure

Researcher’s degree of participation:  participant vs. nonparticipant

Participation problems: “testing effects”, sampling, reliability, “going native”, “instrument decay”

Design and sampling

Systematic Observation

Problems

1.  Gaining access

Gatekeepers

Key Informants

2.  Presenting oneself

3.  Gathering Information

On-the-spot note taking

Video- and audio-taping

Memory

4.  Analysis

Unit VII. Analyzing Available Data

Sources

Public Documents and Official Records

Vital Statistics, Censuses, Public Use Microdata Sample (PUMS)

Private Documents

Mass Media

Physical, Nonverbal Evidence

Social Science Data Archives

Secondary analysis of survey and ethnographic data

Advantages

1. Understanding the Past

2. Understanding Social Change

3. Studying Problems Cross-Culturally

4. Improving Knowledge through Replication and Increased Sample Size

5. Savings on Research Costs

6. Nonreactive Measurement

Problems

Incompleteness (selective deposit and survival)

Indirect measurement

Reliability and Validity

Comparative/Historical

Primary Source, Secondary Source

Necessary and Sufficient Conditions

Content Analysis

1. Selecting and Defining Content Categories

2. Defining the Unit of Analysis                             

3. Deciding on a System of Enumeration

Time/space measures; Appearance; Frequency; Intensity

4. Carrying Out the Analysis

Unit VIII Multiple Methods

Triangulation

Replications

Indexes and scales

Unidimensional

Comparison of Four Basic Approaches to Social Research

Ethnosurvey

Meta-analysis

Unit IX. Ethics

A. Data Collection and Analysis

B. Treatment of Human Subjects, Respondents, Informants

1. Harm

Cost-benefit analysis

2. Informed consent

3. Deception

Debriefing

4. Privacy

Anonymity

Confidentiality

   Institutional Review Boards (IRBs)

C. Responsibility to Society