Examination of Internal Reliabilities of BIMAS-2 SEL Scales

In April 2024, our student Sequoyah Bell, project coordinator Allison Dembowski, and Dr. Christy Walcott (not pictured) presented a poster at the Southeastern School Behavioral Health Conference in Myrtle Beach, SC, describing an examination of the social-emotional learning subscales of the Behavior Intervention Monitoring Assessment System, Second Edition (BIMAS-2). The contents of that poster and supplemental materials are below.

INTRODUCTION

The BIMAS-2 (McDougal et al., 2011) is a robust universal behavior screener and progress monitoring tool. Five norm-referenced scales are derived on the Standard Scale: Conduct, Negative Affect, Cognitive/Attention, Social, and Academic Functioning.

Creation of SEL Scale

The BIMAS-2 creators mapped items from the standard scale onto the five domains of social-emotional learning (SEL) in the CASEL framework (CASEL, 2024) (see graphic) to form the BIMAS-2 SEL Scales: Self-Awareness, Self-Management, Responsible Decision-Making, Relationship Skills, and Social Awareness (EduMetrisis, 2021).

Present Study

The original SEL scales were never empirically tested. We examined internal reliability of the original BIMAS-2 SEL Scales. Additionally, many Items load onto multiple SEL scales rendering intercorrelated scales. An exploratory factor analysis (EFA) of all 33 BIMAS-2 SEL items could reveal the factor structure. Research questions were:

•Are the original BIMAS-2 SEL scales reliable?

•Does EFA discover factors representing all 5 CASEL SEL areas consistent with the original BIMAS-2 SEL scales?

•Do the 33 items fit well on a single factor?

METHODS

•Teachers completed the BIMAS-2 for all students in their class, over 3 consecutive semesters (4th & 5th grade), at 16 schools in NC and SC as part of a larger ongoing study.. The data from Spring ‘23 was used in the present study (n = 923).

•Reliability was based on the full sample. Then, we randomly split the sample in half, and one half was used for the EFA (n = 463).

RESULTS

Estimates of internal reliabilities for the orginal BIMAS-2 SEL scales are in Table 1.

Table 1. Internal Reliabilities of the Original BIMAS-2 SEL Scales

Extraction method was principal axis factoring with oblique rotation (Promax with Kaiser Normalization). Factor loadings above .30 presented. Two items from BIMAS-2 SEL scales did not adequately load onto any factor ("Spoke Clearly to Others" and "Shared What He/She was Thinking About"). Results presented in Table 2.

DISCUSSION

Internal Reliability

We found strong internal reliabilities for each of the original BIMAS-2 SEL scales (α = .80 -.92). Similar ranges were noted across all three measurement occasions.

Exploratory Factor Analysis

Our EFA identified 5 distinct factors; 4 mapped well conceptually onto the CASEL-5 framework, but one area (Social Awareness) was not represented. Many of the original BIMAS SEL items for Soc.A. fell on the new EFA Relationship Skills factor and a few others on Responsible Decision-Making.

Changes from the original SEL scales to the new EFA scales were:

•RDM went from 15 to 8 items, and 88% of new factor items were also on the original RDM scale.

•SM went from 14 to 7 items, and 100% of new factor items were also on the original SM scale.

•Self.A went from 8 to 5 items, and 80% of new factor items were also on the original Self.A scale.

•RS went from 9 to 4 items, and 100% of new factor items were also on the original RS scale.

•Self-destructive Behaviors was a new factor that doesn’t map onto a distinct area of CASEL-5 Framework.

The EFA component correlation matrix is available via the QR code below. Correlations between the new factors ranged from r = -0.05 to 0.61.

Overall, the original SEL Scales have strong internal reliability, but contain many overlapping items. The EFA discovered 5 factors, with 4 aligning well with CASEL framework. Thirty-one of the 33 items fit well onto a single scale; two items were dropped due to poor loadings.

Future studies will use CFA to confirm the factor structure suggested here, using the other half of the sample. We will also test for invariance across time and across racial & gender groups.

REFERENCES

Collaborative for Academic, Social, and Emotional Learning.Fundamentals of SEL. Retrieved January 25, 2024, from https://casel.org/fundamentals-of-sel/

Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297-334. https://doi.org/10.1007/BF02310555

EduMetrisis. (2021, March 16). The BIMAS-2 and the CASEL SEL Framework. Retrieved April 18, 2024, from https://edumetrisis.com/social-emotional/the-bimas-2-and-the-casel-sel-framework/

McDougal, J. L. D., Achilles, N. B., & Meier, S. T. (2011). Behavior Intervention Monitoring Assessment System (BIMAS ) Technical Manual. Multi-Health Systems, Inc.

ACKNOWLEDGEMENTS

The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through Grant R324A210179 to East Carolina University. The opinions expressed are those of the authors and do not represent views of the Institute or the U.S. Department of Education. Special thanks to Dr. Alex Schoemann (statistician) at ECU.