Basic Research Design

What is Research Design?

  • Definition of Research Design: A procedure for generating answers to questions, crucial in determining the reliability and relevance of research outcomes.
  • Importance of Strong Designs: Strong designs lead to answers that are accurate and close to their targets, while weak designs may result in misleading or irrelevant outcomes.
  • Criteria for Assessing Design Strength: Evaluating a design’s strength involves understanding the research question and how the design will yield reliable empirical information.

The Four Elements of Research Design (Blair et al., 2023)

  • The MIDA Framework: Research designs consist of four interconnected elements – Model (M), Inquiry (I), Data strategy (D), and Answer strategy (A), collectively referred to as MIDA.
  • Two Sides of MIDA:
    • Theoretical Side (M and I): This encompasses the researcher’s beliefs about the world (Model) and the target of inference or the primary question to be answered (Inquiry).
    • Empirical Side (D and A): This includes the strategies for collecting (Data strategy) and analyzing or summarizing information (Answer strategy).
  • Interplay between Theoretical and Empirical Sides: The theoretical side sets the research challenges, while the empirical side represents the researcher’s responses to these challenges.
  • Representation of MIDA Elements:
    • Relation among MIDA Components: The diagram above shows how the four elements of a design are interconnected and how they relate to both real-world and simulated quantities.
    • Parallelism in Design Representation: The illustration highlights two key parallelisms in research design – between actual and simulated processes, and between the theoretical (M, I) and empirical (D, A) sides.
    • Importance of Simulated Processes: The parallelism between actual and simulated processes is crucial for understanding and evaluating research designs.
  • Balancing Theoretical and Empirical Aspects: Effective research design requires a balance between theoretical considerations (models and inquiries) and empirical methodologies (data and answer strategies).

Research Design Principles (Blair et al., 2023)

  • Design Holistically
    • Integration of Components: Designs are effective not merely due to their individual components but how these components work together.
    • Focus on Entire Design: Assessing a design requires examining how each part, such as the question, estimator, and sampling method, fits into the overall design.
    • Importance of Diagnosis: The evaluation of a design’s strength lies in diagnosing the whole design, not just its parts.
    • Strong Design Characteristics: Designs with parallel theoretical and empirical aspects tend to be stronger.
    • The M:I:D:A Analogy: Effective designs often align data strategies with models and answer strategies with inquiries.
  • Design Agnostically
    • Flexibility in Models: Good designs should perform well even under varying world scenarios, not just under expected conditions.
    • Broadening Model Scope: Designers should consider a wide range of models, assessing the design’s effectiveness across these.
    • Robustness of Inquiries and Strategies: Inquiries should yield answers and strategies should be applicable regardless of variations in real-world events.
    • Diagnosis Across Models: It’s important to understand for which models a design excels and for which it falters.
  • Design for Purpose
    • Specificity of Purpose: A design is deemed good when it aligns with a specific purpose or goal.
    • Balancing Multiple Criteria: Designs should balance scientific precision, logistical constraints, policy goals, and ethical considerations.
    • Diverse Goals and Assessments: Different designs may be optimal for different goals; the purpose dictates the design evaluation.
  • Design Early
    • Early Planning Benefits: Designing early allows for learning and improving design properties before data collection.
    • Avoiding Post-Hoc Regrets: Early design helps avoid regrets related to data collection or question formulation.
    • Iterative Improvement: The process of declaration, diagnosis, and redesign improves designs, ideally done before data collection.
  • Design Often
    • Adaptability to Changes: Designs should be flexible to adapt to unforeseen circumstances or new information.
    • Expanding or Contracting Feasibility: The scope of feasible designs may change due to various practical factors.
    • Continual Redesign: The principle advocates for ongoing design modification, even post research completion, for robustness and response to criticism.
  • Design to Share
    • Improvement Through Sharing: Sharing designs via a formalized declaration makes it easier for others to understand and critique.
    • Enhancing Scientific Communication: Well-documented designs facilitate better communication and justification of research decisions.
    • Building a Design Library: The idea is to contribute designs to a shared library, allowing others to learn from and build upon existing work.

The Basics of Social Science Research Designs (Panke, 2018)

Deductive and Inductive Research

 

  • Inductive Research (Bottom-Up Approach)
    • Starting Point: Begins with empirical observations or exploratory studies.
    • Development of Hypotheses: Hypotheses are formulated after initial empirical analysis.
    • Case Study Analysis: Involves conducting explorative case studies and analyzing dynamics at play.
    • Generalization of Findings: Insights are then generalized across multiple cases to verify their applicability.
    • Application: Suitable for novel phenomena or where existing theories are not easily applicable.
    • Example Cases: Exploring new events like Donald Trump’s 2016 nomination or Russia’s annexation of Crimea in 2014.
  • Deductive Research (Top-Down Approach)
    • Theory-Based: Starts with existing theories to develop scientific answers to research questions.
    • Hypothesis Development: Hypotheses are specified and then empirically examined.
    • Empirical Examination: Involves a thorough empirical analysis of hypotheses using sound methods.
    • Theory Refinement: Results can refine existing theories or contribute to new theoretical insights.
    • Application: Preferred when existing theories relate to the research question.
    • Example Projects: Usually explanatory projects asking ‘why’ questions to uncover relationships.

Explanatory and Interpretative Research Designs

  • Explanatory Research
    • Definition: Explanatory research aims to explain the relationships between variables, often addressing ‘why’ questions. It is primarily concerned with identifying cause-and-effect dynamics and is typically quantitative in nature. The goal is to test hypotheses derived from theories and to establish patterns that can predict future occurrences.
  • Interpretative Research
    • Definition: Interpretative research focuses on understanding the deeper meaning or underlying context of social phenomena. It often addresses ‘how is this possible’ questions, seeking to comprehend how certain outcomes or behaviors are produced within specific contexts. This type of research is usually qualitative and prioritizes individual experiences and perceptions.
  • Differences Between Explanatory and Interpretative Research
    • Type of Research Questions.
      • Explanatory Research: Poses ‘why’ questions to explore causal relationships and understand what factors influence certain outcomes.
      • Interpretative Research: Asks ‘how is this possible’ questions to delve into the processes and meanings behind social phenomena.
    • Use of Theories
      • Explanatory Research: Relies on established theories to form hypotheses about causal relationships between variables. These theories are then tested through empirical research.
      • Interpretative Research: Uses theories to provide a framework for understanding the social context and meanings. The focus is on constitutive relationships rather than causal ones.
    • Number of Cases Studied
      • Explanatory Research: Often involves studying multiple cases to allow for comparison and generalization. It seeks patterns across different scenarios.
      • Interpretative Research: Typically concentrates on single case studies, providing an in-depth understanding of that particular case without necessarily aiming for generalization.
    • Generalization of Findings
      • Explanatory Research: Aims to produce findings that can be generalized to other similar cases or populations. It seeks universal or broad patterns.
      • Interpretative Research: Offers detailed insights specific to a single case or context. These findings are not necessarily intended to be generalized but to provide a deep understanding of the particular case.

Qualitative, Quantitative, and Mixed-method Projects

  • Qualitative Research
    • Definition: Qualitative research is exploratory and aims to understand human behavior, beliefs, feelings, and experiences. It involves collecting non-numerical data, often through interviews, focus groups, or textual analysis. This method is ideal for gaining in-depth insights into specific phenomena.
    • Example in Education: A qualitative study might involve conducting in-depth interviews with teachers to explore their experiences and challenges with remote teaching during the pandemic. This research would aim to understand the nuances of their experiences, challenges, and adaptations in a detailed and descriptive manner.
  • Quantitative Research
    • Definition: Quantitative research seeks to quantify data and generalize results from a sample to the population of interest. It involves measurable, numerical data and often uses statistical methods for analysis. This approach is suitable for testing hypotheses or examining relationships between variables.
    • Example in Education: A quantitative study could involve surveying a large number of students to determine the correlation between the amount of time spent on homework and their academic achievement. This would involve collecting numerical data (hours of homework, grades) and applying statistical analysis to examine relationships or differences.
  • Mixed-Method Research
    • Definition: Mixed-method research combines both qualitative and quantitative approaches, providing a more comprehensive understanding of the research problem. It allows for the exploration of complex research questions by integrating numerical data analysis with detailed narrative data.
    • Example in Education: A mixed-method study might investigate the impact of a new teaching method. The research could start with quantitative methods, like administering standardized tests to measure learning outcomes, followed by qualitative methods, such as conducting focus groups with students and teachers to understand their perceptions and experiences with the new teaching method. This combination provides both statistical results and in-depth understanding.
  • Making the Choice: The choice among these methods should be guided by:
    • Research Questions: What kind of information is needed to answer the questions? Qualitative for “how” and “why”, quantitative for “how many” or “how much”, and mixed methods for a comprehensive understanding of both the breadth and depth of a phenomenon.
    • Nature of the Study: Is the study aiming to explore a new area (qualitative), confirm hypotheses (quantitative), or achieve both (mixed-method)?
    • Resources Available: Time, funding, and expertise available can influence the choice. Qualitative research can be more time-consuming, while quantitative research may require specific statistical skills.
    • Data Sources: Availability and type of data also guide the methodology. Existing numerical data might lean towards quantitative, while studies requiring personal experiences or opinions might be qualitative.

References:

Blair, G., Coppock, A., & Humphreys, M. (2023). Research Design in the Social Sciences: Declaration, Diagnosis, and Redesign. Princeton University Press.

Panke, D. (2018). Research design & method selection: Making good choices in the social sciences. Research Design & Method Selection, 1-368.