Evaluation

hand writing evaluation sticky notes

 


The NISE Network Evaluation group is a multi-organizational group dedicated to informing and improving the work of the Network by conducting evaluation studies that closely examine the public, professional, and field-wide impacts of NISE Net activities.


Evaluation Tools


Team-Based Inquiry

Team-Based Inquiry (TBI) is a practical approach to empowering education professionals to get the data they need, when they need it, to improve their products and practices and, ultimately, more effectively engage public and professional audiences. The TBI process involves an ongoing cycle of inquiry: question, investigate, reflect, and improve.

  • The Team-based Inquiry Guide explains each step of the TBI process and features ways TBI is used in the NISE Network to improve educational experiences and professional practice. 

  • NISE Net Guidelines for Collecting and Handling Data:  This document outlines general guidelines for collecting and handling Team Based Inquiry (TBI) data. The guide was created for the NISE Network's Institutional Review Board (IRB) protocol as a part of the Human Subjects Research training for NISE Net partners.


Examples of Evaluation Studies

Access to many NISE Net evaluation reports is available through the Research & Evaluation section of the catalog: http://www.nisenet.org/catalog/eval   Below are some examples of the kinds of studies conducted by the NISE Net Evaluation group:

Front-End Evaluation:

Formative Evaluation:

  • Intro to Nanotechnology 2006 Formative Evaluation
    This formative evaluation tested 3 prototypes for the Introduction to Nanotechnology in March 2006. Of the three prototypes tested Self Assembly continued forward to a final exhibit: Creating Nanomaterials. The concepts in Colored Glass prototype informed the development of Unexpected Properties.
  • Who Decides Forum 2000 Formative Evaluation
    This study was conducted as a part of the formative evaluation of the NISE Network forum “Nanotechnology: Risks, Benefits, and Who Decides?”
  • Spiral Zoom Nasturtium Leaf 2009 Formative Evaluation
    This formative evaluation was conducted to see how the addition of an interactive media piece enhanced visitors' understanding of Nasturtium, a life sciences exhibit that demonstrates the water repelling properties of nasturtium leaves. The media piece allows the visitor to view leaf structures at progressively higher magnifications to better illustrate their scale and function.
  • Exploring Measurement Stretchability 2010 Formative Evaluation
    The Science Museum of Minnesota conducted the StretchAbility program on January 25th, and February 1st, 2010, and the Children’s Museum of Houston conducted the program on November 10th, 14th, and 25th, 2009. A total of 20 paired adult and child groups provided feedback through a survey designed to measure their engagement with and comprehension of the activity. After the activity, evaluators targeted participating children 8 or younger who were verbal for the interview, and gave a survey to the child’s parent to complete. Paired surveys were used due to the lower verbal nature of the younger audience; by also collecting data from the children’s parents, the evaluators were able to better assess the young visitors’ possible connection to the story.
  • Treating Tumors with Gold 2009 Formative Evaluation
    This report evaluates the program entitled “Treating Tumors with Gold” by looking at visitor feedback in an attempt to assess the success with which the presentation was able to educate the public on a particular study using nanotechnology.
 
Summative Evaluation
  • Overview of NISE Network Evaluation Year 4 Summative Evaluation
    The documents that comprise the Inverness Research Summative Report provide a comprehensive and systematic review of the progress made in developing a network organization capable of supporting nanoscience education for the public on a national scale
  • Museum Public Awareness of Nanotechnology Year 3 Summative Evaluation
    This 2008 summative evaluation is a post-survey design with nonrandomized comparison groups, looking at the impact of nano-topic deliverables on public awareness at four NISE Net museums.
  • Museum Public Awareness of Nanotechnology Year 4 Summative Evaluation
    The Nanoawareness Study is designed to answer the question "What, if any, impact do NISE Net activities delivered at Tier 1 and Tier 2 institutions have on the nanoawareness of the public audiences that experience those activities?" The Nanoawareness Study was initially conducted in Year 3 and then replicated in Year 4 with some methodological changes and a different sample of participants. The following report describes the Nanoawareness Study findings from Year 4 in comparison to findings from Year 3.
  • Public Impacts: Use of NISE Net Products Year 4 Summative Evaluation
    This study looks specifically at the activities of the Tier I, II and III institutions as a way of determining whether it is likely that NISE Net will have an impact on the public through the NSET public outreach activities of those institutions. The main question driving this study is the following: To what extent is NISE Net reaching the public through the different tiers of the Network? This study presents preliminary findings from the Study 2 investigation, looking specifically at the actions of the professionals who have come into contact with NISE Net (including those who have attended conference sessions, signed-up for nisenet.org, attended the NISE Net annual meeting, and signed-up for NanoDays) and whether those professionals are delivering programs to the public that are likely to have an impact on public awareness and understanding of NSET.
  • Public Impacts: The Counting Study Year 4 Summative Evaluation
    This study looked specifically at one question: Approximately how many people participated in NISE Net public outreach activities during NanoDays 2009?
  • Exhibits and Programs Year 4 Summative Evaluation
    Summative evaluation of four programs created by the NISE Network.
  • Delivery and Public Reach Year 5 Summative Evaluation
    The 2010 Delivery and Reach study documented the delivery of nano education activities at NISE Network partner institutions and estimated the public reach of those activities.
  • Exhibits and Programs Year 5 Summative Evaluation
    As part of the overall summative evaluation of the first five years of the NISE Network, the Year 5 Exhibits and Programs Study examines the measurable impacts of these public products on museum visitors.
  • Review of NISE Net Evaluation Findings: Years 1-5 Summative Evaluation
    The Review of NISE Network Evaluation Findings: Years 1-5 seeks to investigate the work of the NISE Network since its inception in 2005 and provide an overarching summary of NISE Net Public Impacts evaluation efforts to the NISE Network and the broader ISE field
  • Network Communication Study Year 6 Developmental Evaluation
    A Study of Communication in the NISE Network (Network Communication Study), conducted during the sixth year of the grant, sought to learn how the four primary communication components that were developed in the first 5 years of NISE Net (NanoDays, face-to-face meetings, the regional hub structure, and the nisenet.org website) are functioning within the Network. In particular, the study explored how these components communicate information, ideas, and practices related to NISE Net between and within the three Network tiers.
  • Public Impacts Mini-Exhibition Study Year 8 Summative Evaluation
    The Summative Study of the Nano Mini-exhibition took place during the spring and summer of 2012. After being observed during their Mini-exhibition experience, 455 visitors across six different partner institutions participated in surveys and interviews with NISE Net evaluation team members. This report begins by describing the key findings of the study in detail, with additional information about study methods, instruments, and two exploratory sub-studies found in the Appendices.
     

Evaluation team members

During the project, the evaluation team has included staff members from the following organizations:

In addition, the public impacts summative evaluation is overseen by an external Committee of Visitors comprised of the following members:

  • Bruce Lewenstein
  • Saul Rockman
  • Frances Lawrenz

NISE Network's online surveys are powered by SurveyGizmo.