Fundamentals of measurement in applied research / Theresa A. Thorkildsen

CONTENTS
CHAPTER 1 The Role of Measurement in Research 000
UNDERSTANDING THE STRUCTURE OF RESEARCH 000
Distinguishing Philosophical, Theoretical, and Measurement Assumptions 000
Distinguishing Research Design and Measurement 000
Distinguishing Methods of Analysis, Interpretation, and Measurement 000
Disseminating Findings 000
IS A NEW STUDY ALWAYS NECESSARY? 000
DEFINING A MEASUREMENT PURPOSE 000
Measurement Scales 000
Linking an Agenda with Tools 000
DIFFERENTIATING MEASUREMENT CONCERNS 000
CHAPTER 2 Integrity in the Research Enterprise 000
ETHICAL PRINCIPLES 000
Respect for Persons 000
Beneficence 000
Justice 000
Continuing Moral Tensions 000
IS A STUDY WARRANTED? 000
INTERACTING WITH PARTICIPANTS 000
Planning a Design 000
Recruiting Participants and Obtaining Consent 000
Interactions during a Study 000
Putting Planning into Action 000
IDENTIFYING APPROPRIATE MEASUREMENT TOOLS 000
How Will Data Be Utilized? 000
Are the Limitations of Each Tool Acknowledged? 000
What Types of Error Can Be Minimized? 000
How Will Respondents? Attitudes Influence the Performance of a Tool? 000
How Will Respondents Be Informed about the Use of Their Responses? 000
Debriefing as a Source of Information 000
IMPLEMENTING A STUDY WITH ACCURACY 000
Minimizing Invasiveness 000
FM.QXD 7/13/2004 3:49 PM Page v
Accommodating the Design to Volunteers 000
Compare an Accommodated Design to Theoretical Concerns 000
Isolating Valid Conclusions 000
REPORTING RESULTS 000
Promote Parsimony in Analysis 000
Evaluate the Balance and Fairness of Each Measurement Outcome 000
Acknowledge the Limitations of Each Measurement Technique 000
Compare the Design and Final Measures 000
Determine Authorship 000
Credit Sources 000
Coordinating Issues 000
QUESTIONING PERSONAL JUDGMENT 000
Update Training 000
Develop Professional Relationships 000
Seek Feedback 000
Evaluate Measurement Skills 000
LEARNING ETHICS 000
SUGGESTED READINGS 000
SUGGESTED WEB SITES 000
CHAPTER 3 Assumptions of Psychometric Measurement 000
COORDINATING THEORY AND MEASUREMENT 000
DIFFERENCES IN PHYSICAL AND PSYCHOLOGICAL MEASUREMENT 000
STRATEGIES FOR LINKING INDICANTS AND CONSTRUCTS 000
MEASUREMENT LANGUAGE 000
Reliability and Error Measurement 000
Differentiating Reliability and Validity 000
Adjusted True Score Theory 000
Summary 000
NUMERICAL REPRESENTATIONS 000
Connecting Indicants to Numbers 000
Characteristics of a Distribution 000
Reliability as Stability and Accuracy 000
Interpreting Raw Scores 000
Valid Alignment with Theory 000
EXPLORING RELATIONSHIPS AMONG INDICANTS 000
What Is Factor Analysis? 000
Do All Indicants Belong on the Same Measure? 000
What Does a Factor Look Like? 000
How Many Dimensions Are Evident? 000
What Makes a Factor Reliable? 000
vi CONTENTS
FM.QXD 7/13/2004 3:49 PM Page vi
DISTORTIONS AND OTHER RESPONSE EFFECTS 000
Self-Report Bias 000
Design Distortions 000
Unintended Distortions 000
Limiting Distortion 000
COMPARING ASSUMPTIONS 000
SUGGESTED READINGS 000
Introductory Books 000
Advanced Books 000
Suggested Web Sites 000
CHAPTER 4 Measuring Attitudes 000
SCALE DEVELOPMENT PROCEDURES 000
TYPES OF SCALES 000
PERSON SCALING 000
Methods of Summated Ratings 000
Advantages and Disadvantages of Person Scaling 000
STIMULUS, THEN PERSON SCALING 000
Method of Paired-Comparison 000
Method of Equal-Appearing Intervals 000
Method of Successive Intervals 000
Advantages and Disadvantages of Stimulus, Then Person Scales 000
DILEMMAS FOR SINGLE-ITEM SCALES 000
SIMULTANEOUS STIMULUS AND PERSON SCALING 000
Guttman Scales 000
Advantages and Disadvantages of Simultaneous Stimulus and Person Scaling 000
EVALUATING THE QUALITIES OF INDICANTS: ITEM ANALYSIS 000
Person Scales 000
Stimulus, Then Person Scales 000
Evaluating Indicants 000
EVALUATING RELATIONSHIPS AMONG INDICANTS: RELIABILITY 000
Person Scales 000
Stimulus, Then Person Scales 000
Simultaneous Stimulus and Person Scales 000
Relationships among Indicants 000
EVALUATING RELATIONSHIPS BETWEEN INDICANTS
AND CONCEPTS: VALIDITY 000
Content Validity 000
Criterion Validity 000
Construct Validity 000
Theoretical Validity 000
CONTENTS vii
FM.QXD 7/13/2004 3:49 PM Page vii
ATTITUDES AS BELIEFS, OPINIONS, OR VALUES 000
SUGGESTED READINGS 000
SUGGESTED WEB SITES 000
CHAPTER 5 Measuring Personality 000
TYPES OF PERSONALITY MEASURES 000
Categories of Measurement 000
NOMOTHETIC MEASUREMENT 000
Self-Report Data 000
Observational Data 000
Standardized Test Data 000
Summary 000
IDEAL QUALITIES OF INDICANTS 000
What Do Indicants Measure? 000
Which Item Formats Work Best? 000
When Are Indicants Associated? 000
What Makes a Strong Indicant? 000
How Can Diverse Items Be Evaluated? 000
How Can Measures of Function Be Evaluated? 000
How Can Indicants Be Linked to Source Traits? 000
Summary 000
DIMENSIONS OF RELIABILITY 000
Establishing Inter-Rater Reliability 000
Calibrating Nomothetic Instruments 000
Transferability across Subpopulations 000
Summary 000
VALIDITY CONSIDERATIONS 000
Content Validity 000
Criterion Validity 000
Construct Validity 000
Theoretical Validity 000
Summary 000
IMPEDIMENTS TO MEASURING PERSONALITY 000
Participant?Generated Distortion 000
Investigator?Generated Distortion 000
CHARACTERISTICS OF PERSONALITY 000
SUGGESTED READINGS 000
SUGGESTED WEB SITES 000
viii CONTENTS
FM.QXD 7/13/2004 3:49 PM Page viii
CHAPTER 6 Measuring Intelligence 000
WHAT IS INTELLIGENCE? 000
WHAT IS ABILITY? 000
DIFFERENTIATING ABILITY AND SKILL 000
INFERRING INTELLIGENCE FROM ESTIMATES OF ABILITY 000
ESTABLISHING A THEORETICAL BLUEPRINT 000
The Purpose of Blueprints 000
COMMON ITEM FORMATS 000
Multiple-Choice Items 000
Matching Items 000
True-False Formats 000
Interpretive Selection Exercises 000
Short-Answer Items 000
Essay Items 000
Choosing a Format 000
Assembling a Test 000
ITEM-ANALYSIS PROCEDURES 000
Evaluating Items for Ability Tests 000
Evaluating Items for Intelligence Tests 000
RELIABILITY 000
Steps for Verifying Reliability 000
Exogenous Influences on Reliability 000
VALIDITY 000
Content Validity 000
Criterion Validity 000
Construct Validity 000
MEASURING GENERAL COGNITIVE FUNCTIONING 000
SUGGESTED READINGS 000
SUGGESTED WEB SITES 000
CHAPTER 7 Measuring Competence and Skill 000
WHAT IS COMPETENCE? 000
ARE INTELLIGENCE AND COMPETENCE DIFFERENT CONSTRUCTS? 000
WHAT IS A SKILL? 000
CONTENTS ix
FM.QXD 7/13/2004 3:49 PM Page ix
DESIGNING SKILL-FOCUSED TOOLS 000
Fundamentals of Item-Response Theory 000
ESTABLISHING A SKILL-BASED BLUEPRINT 000
Blueprints for Computer Adaptive Tests 000
Blueprints for Sequential Measures of Skill 000
PROCEDURES FOR TEST CONSTRUCTION 000
Designing a Computer Adaptive Test 000
Designing a Sequential Skills Test 000
RELIABILITY 000
Precision and Computer Adaptive Testing 000
Conventional Criterion-Referenced Approaches to Reliability 000
VALIDITY 000
Content Validity 000
Criterion Validity 000
Threats to Validity 000
Generalizations 000
MEASURING SPECIFIC COGNITIVE FUNCTIONING 000
SUGGESTED READINGS 000
SUGGESTED WEB SITES 000
CHAPTER 8 Measuring Behavior and Physiological Reactions 000
MEASURING BEHAVIOR 000
Matching Theoretical Concerns with Indicants 000
Designing Tools: Categorical Classification Systems 000
Designing Tools: Descriptive Systems 000
Evaluating the Qualities of Indicants: Sources of Error 000
Reliability for Observational Techniques 000
Validity of Observational Techniques 000
MEASURING PHYSIOLOGICAL REACTIONS 000
Psychophysical Scaling 000
Measures of Isolated Physical Properties 000
Measures of the Central Nervous System 000
MEASURING ACTIONS AND REACTIONS 000
SUGGESTED READINGS 000
CHAPTER 9 Assumptions of Developmental Measurement 000
STUDYING A DYNAMIC SYSTEM 000
How Is Change Explored? 000
x CONTENTS
FM.QXD 7/13/2004 3:49 PM Page x
Theoretical Parameters 000
Comparing Psychometric and Developmental Measurement 000
Components of Individual Development 000
Differences in Tools and Their Use 000
DIMENSIONS OF CHANGE 000
Levels of Organization 000
Plasticity versus Stability 000
Continuity versus Discontinuity 000
Quantitative versus Qualitative Differences 000
Change in a Dynamic System 000
TYPES OF CHANGE 000
Representational Change 000
Temporal Change 000
Diversity and Individual Variation 000
Theorizing about Change 000
MEASURING CHANGE 000
Measuring Representational Change 000
Measuring Temporal Change 000
Measuring Diversity 000
A PROGRAMMATIC EXAMPLE 000
SUGGESTED READINGS 000
SUGGESTED WEB SITES 000
CHAPTER 10 Representing Structure and Functioning 000
COMMON STRUCTURAL FEATURES 000
COMMON STRUCTURAL FORMS 000
Hierarchical Structures 000
Completely Coordinated Structures 000
Completely Independent Structures 000
Overlapping Structures 000
Anticipating Structural Features 000
DILEMMAS IN DEFINING STRUCTURES 000
Common Questions 000
QUESTIONING THE OVERDEPENDENCE ON JUDGMENTS 000
Limitations of Direct Observation 000
Limitations of Interviews 000
Limitations of Surveys and Tests 000
Confounding Inferences in Experimental Designs 000
Potential Theoretical Blind Spots 000
MEASURING JUDGMENTS AND EXPLANATIONS 000
Verbal Behavior about Events Remote in Time 000
CONTENTS xi
FM.QXD 7/13/2004 3:49 PM Page xi
Verbal Behavior about Immediate Events 000
Mixing Verbal and Nonverbal Behavior 000
Nonverbal Behavior 000
Clinical Concentric Studies 000
Designing Activities 000
INVESTIGATORS? SKILLS 000
Job Skills 000
Personal Attributes 000
Work-Related Knowledge 000
Verifying Qualifications 000
CALIBRATING TOOLS 000
Liberated and Spontaneous Convictions 000
Constrained Convictions 000
The Importance of Justifications 000
STANDARDIZATION 000
Variation in Theoretical Commitments 000
Differentiating Trivial and Essential Forms of Standardization 000
Maximizing Interaction 000
Bracket Indicants for an Age Group 000
The Value of Comparison 000
GENERALIZABILITY 000
Generalizability as the Verification of Order 000
Generalizability as Synchrony of Concepts 000
Generalizations about Stages 000
Generalizations Involving Children 000
Generalizations about Generalizations 000
REPRESENTING DEVELOPMENT 000
SUGGESTED READINGS 000
SUGGESTED WEB SITES 000
CHAPTER 11 Measuring Temporal Change 000
STEPS IN THEORY BUILDING 000
Representational Change 000
Individual Differences 000
Temporal Change 000
Multiple Designs, Multiple Tools 000
Verifying the Practicality of Longitudinal Research 000
Types of Temporal Change 000
CHALLENGES OF MEASURING TEMPORAL CHANGE 000
Design and Variable Constraints 000
Control for Initial Differences 000
Difference Scores 000
xii CONTENTS
FM.QXD 7/13/2004 3:49 PM Page xii
Comparability of Stimuli 000
Appropriate Measurement Scales 000
Finding Enough Longitudinal Data 000
DIFFERENT FORMS OF TEMPORAL CHANGE 000
Age or Proxies for Time? 000
Developmental or Nondevelopmental Change? 000
Why Use Age? 000
CALIBRATING TEMPORAL VARIABLES 000
Limitations of Change as Integration 000
Change 000
Stability 000
Order 000
Chaos 000
Procedures for Calibrating Tools 000
STANDARDIZING TEMPORAL VARIABLES 000
Assumptions about Time 000
Change in Integration 000
Change in Rate, Magnitude, or Duration of Time 000
Coordinating Qualitative and Quantitative Measures 000
MAKING GENERALIZATIONS 000
Indicators of Integration 000
Quantitative Change 000
VERIFYING DEVELOPMENT 000
SUGGESTED READINGS 000
CHAPTER 12 Measuring Diversity in Human Functioning 000
INDIVIDUAL VARIATION 000
Theoretical Assumptions 000
Empirical Decisions 000
DIVERSITY 000
Theoretical Distinctions 000
Empirical Distinctions 000
INTERACTIVE MEASUREMENT 000
Interview Parameters 000
Interview Styles 000
Designing Interviews 000
Conducting Interviews 000
Adaptive Testing 000
Lessons from Interactive Methods 000
NORMATIVE MEASUREMENT 000
The Role of Inferences 000
CONTENTS xiii
FM.QXD 7/13/2004 3:49 PM Page xiii
The Effects of Context 000
Participants? Comfort 000
Lessons from Normative Methods 000
CALIBRATING TOOLS 000
Interactive Tools 000
Normative Tools 000
Coordinating Interactive and Normative Tools 000
STANDARDIZING TOOLS 000
Interactive Tools 000
Normative Tools 000
SUPPORTING GENERALIZATIONS 000
Broad Validity Concerns 000
Constraints on Generalizations 000
Purposes of Generalizations 000
COORDINATING INDIVIDUAL VARIATION AND DIVERSITY 000
SUGGESTED READINGS 000
SUGGESTED WEB SITES 000
CHAPTER 13 Assumptions about Interpretive Measurement 000
Levels of Interpretation 000
A Broader Notion of Validity 000
APPROACHES TO INTERPRETATION 000
Interpretive Measurement Assumptions 000
Conclusions in Interpretive Research 000
TYPES OF INTERPRETIVE RESEARCH 000
Phenomenology 000
Critical Theory 000
Interpretive Analysis 000
Deconstructionism 000
PURPOSES FOR INTERPRETIVE RESEARCH 000
Rule-Governed Interpretations 000
MINIMIZING BIAS 000
Nature of Bias 000
Addressing Bias 000
Misconceptions 000
Respectable Bias 000
DEPENDABILITY AND CREDIBILITY 000
Dependability 000
Credibility 000
xiv CONTENTS
FM.QXD 7/13/2004 3:49 PM Page xiv
Grounded Interpretations 000
Verifying Dependability and Credibility 000
CONFIRMABILITY 000
Balancing Internal and External Validity 000
Levels of the Validation Process 000
Confirmation Processes 000
PURPOSES FOR INTERPRETIVE MEASUREMENT 000
SUGGESTED READINGS 000
CHAPTER 14 Measuring Cycles of a Life?s Course 000
STUDYING LIVES IN PROGRESS 000
Purposes for Exploring Individual Lives 000
MEASURING IDENTITY 000
Nomothetic Approaches 000
Idiographic Approaches 000
Idiothetic Approaches 000
Interpretive Approaches 000
MEASURING INTERACTIONS 000
Discourse Analysis 000
Case Studies 000
MEASURING LIVES IN SOCIOCULTURAL CONTEXTS 000
Framing 000
Action Research 000
MEANS OF TEXT PRODUCTION 000
Self-Report Approaches 000
Observational Records 000
Standardized Tests and Personal Documents 000
CALIBRATING MEASURES 000
Describing Individuals 000
Describing Interactions 000
Describing Transpersonal Relations 000
DEPENDABLE AND CREDIBLE EVIDENCE 000
Dependable and Credible Fixed Variables 000
Dependable and Credible Random Variables 000
Dependable and Credible Interpretations 000
CONFIRMING GENERALIZATIONS 000
Comparing Scores across Methods 000
Triangulation 000
Validity in Reference to Sociopolitical Contexts 000
CONTENTS xv
FM.QXD 7/13/2004 3:49 PM Page xv
THE STORY OF A LIFE 000
SUGGESTED READINGS 000
SUGGESTED WEB SITES 000
CHAPTER 15 Measuring Group Structures and Functioning 000
ASSUMPTIONS ABOUT GROUPS 000
PURPOSES FOR MEASURING GROUPS 000
Social Status 000
Social Networks 000
Structures in Ethnography and Ethnology 000
MINIMIZING BIAS IN DESCRIBING GROUPS 000
Social Status 000
Social Networks 000
Ethnographic Structures 000
Differentiating Informative and Uninformative Bias 000
DEPENDABILITY AND CREDIBILITY IN INFERENCES 000
Social Status 000
Social Networks 000
Ethnographic Structures 000
Verifying Second-Order or Third-Order Inferences 000
CONFIRMING CLASSIFICATIONS, NETWORKS, AND STRUCTURES 000
Social Status 000
Social Networks 000
Ethnographic Structures 000
Choosing an Approach to Validity 000
GROUPS AS LEVELS OF ORGANIZATION 000
SUGGESTED READINGS 000
CHAPTER 16 Measuring Micro- and Macrocultures 000
PURPOSES FOR CULTURAL THEORY 000
COMMON RESEARCH PARADOXES 000
THE LOGIC OF CULTURAL THEORIES 000
ANALYSIS OF TEXT 000
Building Relationships 000
Centrality of Text in Analyzing Interactions 000
Rely on Literal Interpretations 000
xvi CONTENTS
FM.QXD 7/13/2004 3:49 PM Page xvi
Multiple Questions, Multiple Approaches 000
Limitations of Text Analysis 000
MEASURING CULTURAL RITUALS 000
The Importance of Participant-Observation 000
Communication as Negotiation 000
Procedures for Fieldwork 000
The Cyclical Nature of Measurement 000
Guiding Questions 000
Summarizing Evidence 000
Describing Rituals 000
COLLECTING ARTIFACTS 000
Purposes for Artifacts 000
Culture as Principles of Interpretation 000
Guiding Questions 000
The Value of Artifacts 000
Limitations of Artifacts 000
COMBINING SOURCES OF EVIDENCE 000
Procedural Advice 000
MINIMIZING BIAS 000
Text Analysis 000
Cultural Rituals 000
Cultural Artifacts 000
DEPENDABILITY AND CREDIBILITY 000
Text Analysis 000
Cultural Rituals 000
Cultural Artifacts 000
Triangulation 000
CONFIRMING CONCLUSIONS 000
Guiding Questions 000
Choose an Exemplary Vignette 000
Write a General Description 000
Evaluate Interpretive Commentaries 000
Evaluate Causal Networks 000
Construct an Historical Record 000
Compare the Typical and Unusual 000
Limitations of Ethnographic Evidence 000
STUDYING CULTURES AS SOCIAL ORGANIZATIONS 000
SUGGESTED READINGS 000
Appendix: Tables 000
References 000
Index

"Fundamentals of Measurement in Applied Research introduces students to common measurement techniques from applied research so that they can design, produce, and use new tools. The author shows how users of research and assessment tools can become proficient in the production of new instruments. The text reviews details of how psychometric, developmental, and interpretive approaches to measurement are used in a multitude of educational and social sciences. Supporting activities illustrate how researchers use key measurement concepts. The text can be covered one chapter at a time in a formal measurement course or used as a supplementary text for courses in research design."