DESCRIPTION
In this reflective article, the evaluation team for the NASA-funded SciAct STEM Learning Ecosystems project discusses the process we used to study four exemplar STEM learning ecosystems in regions across the US using culturally responsive evaluation (CRE) methods. We expected to follow a linear path for each of the study’s three inquiry cycles but found that this process needed to be fluid because of the culturally responsive evaluation methods employed. These methods meant we shared control with the project team in ways we had not before and had to rethink study validity and rigor. Beyond the evaluation team, this article shares reflections from the projectteam as well as the study’s external evaluation advisor about what it was like to participate in a study that used CRE methods.
Suggested citation:
Kollmann, E. K., Atwood, A., Anderson, A., Ostman, R., Bledsoe, K. L., Buffington, C., Cass, M., DeLisi, J., Jackson, A., Leavell, C., Mannis, K., Martin, P., McCarthy, C., Neff, R., Peake, L., Sparrow, E. B. (2025). Implementing Culturally Responsive Evaluation Methods: Reflections on Challenges to Traditional Understandings of Power, Validity, and Rigor. Visitor Studies, 1-21.
https://doi.org/10.1080/10645578.2025.2494485
DESCRIPTION
In this reflective article, the evaluation team for the NASA-funded SciAct STEM Learning Ecosystems project discusses the process we used to study four exemplar STEM learning ecosystems in regions across the US using culturally responsive evaluation (CRE) methods. We expected to follow a linear path for each of the study’s three inquiry cycles but found that this process needed to be fluid because of the culturally responsive evaluation methods employed. These methods meant we shared control with the project team in ways we had not before and had to rethink study validity and rigor. Beyond the evaluation team, this article shares reflections from the projectteam as well as the study’s external evaluation advisor about what it was like to participate in a study that used CRE methods.
Suggested citation:
Kollmann, E. K., Atwood, A., Anderson, A., Ostman, R., Bledsoe, K. L., Buffington, C., Cass, M., DeLisi, J., Jackson, A., Leavell, C., Mannis, K., Martin, P., McCarthy, C., Neff, R., Peake, L., Sparrow, E. B. (2025). Implementing Culturally Responsive Evaluation Methods: Reflections on Challenges to Traditional Understandings of Power, Validity, and Rigor. Visitor Studies, 1-21.
https://doi.org/10.1080/10645578.2025.2494485
Credits
© 2025 The Author(s). Published with license by Taylor & Francis Group, LLC.Visitor Studies
This material is based upon work supported by NASA under cooperative agreement award number 80NSSC21M0007. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the view of the National Aeronautics and Space Administration (NASA).
This linked product was created by another institution (not by the NISE Network). Contact owning institution regarding rights and permissions.
NISE Network products are developed through an iterative collaborative process that includes scientific review, peer review, and visitor evaluation in accordance with an inclusive audiences approach. Products are designed to be easily edited and adapted for different audiences under a Creative Commons Attribution Non-Commercial Share Alike license. To learn more, visit our Development Process page.