The Mechanisms of Peer-Effects in Education: A Frame-factor analysis of Instruction
Pontus Bäckström, Jönköping University
In the educational literature on peer effects, attention has been brought to the fact that the mechanisms creating peer effects are still to a large extent hidden in obscurity. The hypothesis in this study is that the Frame Factor Theory can be used to explain these mechanisms. At heart of the theory is the concept of “time needed” for students to learn a certain curricula unit. The relations between class-aggregated time needed and the actual time available, steers and hinders the actions possible for the teacher. Further, the theory predicts that the timing and pacing of the teachers’ instruction is governed by a “steering criterion group” (SCG), namely the pupils in the 10th-25th percentile of the aptitude distribution in class. The class composition hereby set the possibilities and limitations for instruction, creating peer effects on individual outcomes.
To test if the theory can be applied to the issue of peer effects, the study employs multilevel structural equation modelling (M-SEM) on Swedish TIMSS 2015-data (Trends in International Mathematics and Science Study; students N=3761, teachers N=179). Using confirmatory factor analysis (CFA) in the SEM-framework, latent variables are specified according to the theory, such as “limitations of instruction” from TIMSS survey items.
The results indicate a good model fit to data of the measurement model. The SEM-model verify a strong relation between the mean level of the SCG and the latent variable of limitations on instruction, a variable which in turn has a great impact on individual students’ test results. Thus, the analysis indicates a confirmation of the predictions derived from the frame factor theory and reveals that one of the important mechanisms creating peer effects in student outcomes is the effect the class composition has upon the teachers’ instruction in class.
How to avoid v-hacking in organizational psychology research
Sofie Holmquist, Umeå University
Identifying, defining, and measuring critical factors involved in job-related motivation and well-being has been a fundamental part of organizational research over the past 50 years. Valid measurement is a prerequisite to trustworthy claims of psychological phenomena. If the measurement of a theoretical construct (e.g., leadership style) is questionable, then so is every subsequent claim of the theoretical construct. In recent years, considerable research has focused on measurement issues in organizational psychology research. Hussey and Hughes (2020, p. 181) introduced the term v-hacking to describe ”cherry-picking those validity metrics that provide the most favorable impression of their measures (…) to the potential detriment of the validity of their conclusions.” Scholars call for improved measurement practices in organizational psychology research and psychological research in general. A standard procedure in organizational psychology research is to use surveys and self-rating scales to collect data on the constructs of interest. Methodological concerns regard the research practices surrounding psychometric investigations of structural validity. Much of the critique can be summarized as a lack of theoretical grounds for claims about validity. Although validity is a central aspect in any research setting, validity theory frameworks seem to be underreported in scale evaluation studies. Thus, the purpose of the proposed presentation is to provide practical guidance for building validity arguments related to self-rating scales in organizational psychology research. From a theoretical and a practical perspective, an overview of the validity concept will be given in the proposed presentation. Theoretically, the approach emphasized is based on modern validity theory, as described by Kane (2013); validity evidence is limited to a specific interpretation or use of test scores. Concepts discussed include interpretative arguments, validity arguments, and validity evidence.
Testing measurement invariance across alternative test forms
Daria Gracheva, Higher School of Economics, Moscow
The use of alternative test forms has become a standard practice in education. Various forms are required to test large samples at different times avoiding practice effects or to enhance test security. However, test results can be considered valid only in case test forms are comparable. The present paper is focused on the comparability of different forms of computerized performance-based tasks (CPBT). The CPBT includes a variety of task types and has already been stated as a promising tool for measuring complex constructs (for example, 21st-century skills). We propose to analyze the comparability of CPBT forms using the methodology of measurement invariance in confirmatory factor analysis. This methodology is usually applied to investigate how an instrument is functioning across different samples or populations. We decided to shift from the traditional approach and use measurement invariance methodology within a single group taking both test forms. This approach allowed to explore the comparability of test types with complex structures such as CPBT and to investigate the comparability of forms regardless of test-takers personal characteristics. Moreover, we applied measurement invariance with ordered-categorical data using the WLSMV estimator while most studies treated categorical data as the interval scale in multigroup confirmatory factor analysis. The paper included the empirical example exploring the compatibility of CPBT aimed at measuring 21st-century skills (critical thinking, creativity, communication, cooperation) on a sample of more than 400 four graders from Russian schools who take both forms.
The geography of educational capital: a comparison of grade averages between urban and rural areas in the years 1998 to 2017
Tommie Petersson, Uppsala University
The French sociologist Pierre Bourdieu investigated different forms of capital, power resources, that works together to form a social hierarchy that denotes different social status to different geographical locations. He described this hierarchization as “socially ranked geographical space” (Bourdieu 1979). One of the most determining factors of social hierarchization is the amount of cultural and social capital accumulated by individuals, and one of the most important aspects in these types of capitals are educational capital (Broady 1998).
This study will present preliminary findings from a comparison between academic achievement or outcomes between urban and rural areas. By comparing grade averages in urban areas with the ones in rural areas, one can get an initial sense of how different parts of Sweden is ranked in an academic context. The data used in the present study is from the Gothenburg Educational Longitudinal Database (GOLD) and spans over the years 1998 to 2017.
The analysis of academic outcomes in grade 9 in compulsory school will also be performed separately for the different years and geographical areas. Descriptive statistics for grade averages will be presented alongside some introductory thoughts on how to interpret the results.
Bourdieu, Pierre. La Distinction [Distinction]. Transl: Tony Bennett. London: Routledge, 2010 (orig. 1979).
Broady, Donald. Kapitalbegreppet som utbildningssociologiskt verktyg. Uppsala: Uppsala universitet, 1998.
Working memory and its mediating role on the relationship of math anxiety and math performance
Jonatan Finell, Umeå University
It has been well established that math anxiety has a negative correlation with math performance (Hembree 1990; Namkung et al., 2019). Various theories have provided explanations for this relationship. One of them, the Processing Efficiency Theory (PET), suggests that worry can reduce both storage and processing capacity in the working memory system, that is accessible for concurrent task (Eysenck & Calvo, 1992). Within the PET framework a model where working memory mediates the relationship of math anxiety predicting math performance, is examined through a systematic literature study to gain deeper insights of the mechanisms of the math anxiety – math performance relationship. The aim of this systematic literature study is to synthesize published research that have analyzed such a model. As the research that includes this specific model is scarce, a meta-analysis with a fixed effect model approach will be used and function as a descriptive statistic. The second aim of this study is to synthesize the math anxiety – working memory correlation. As there is more available research that provides results for the second aim, a meta-analysis with a random-effects approach will be performed. Through data base searches with pre-determined search strings, 1357 unique records were identified. Abstracts of these studies were read by two researchers and an inter-rater reliability coefficient was calculated. After excluding non-relevant studies, data from 77 studies was extracted. 148 effect sizes were found for aim two of the study, 8 were found for aim 1 of the study.