A Transactionional and Participatory Approach to Program Evaluation

In my graduate course on program evaluation, I put together a proposal plan for the evaluation of a sports program for ages 3-6.  The evaluation proposal that I put together was created to address conflict that was present in the program, as there were various stakeholder groups all of whom had a vested interested and ownership of the program.  In this case the stakeholders would be the sports club organization, each individual community club (6 in total), parents, and coaches/conveners. My plan used an evaluation model that used a participatory and transactional approach.  The model is described below.

The evaluation process will be ongoing every season for two years.  The evaluation will follow a modified version of Robert Rippey’s transactional evaluation model and will include elements of a participatory evaluation model (Taylor-Powell, 2006) which includes a collaborative focus that encourages participant focus and ownership.  Part of this approach includes the creation of an advisory group that will work with the evaluator to analyze data and make program recommendations. This combination is being used as there are a variety of stakeholders who all represent important parts of the program.  It is important to establish that input and communication from each group is important within this evaluation to have all parties invested in the process of improving this program.

• Roles
  • Evaluator:  The evaluator will administer the entire evaluation.  The evaluator will be responsible for designing data collection instruments, collecting data, analyzing data, sharing and presenting data to the advisory group, making program recommendations with the advisory group, and creating a final report summarizing the evaluation.
  • Advisory Group:  The advisory group will consist of the evaluator, and at least one representative from the sports club, and each community centre.  Current parents and coaches will also be invited to be part of the group.
• The structure will follow Rippey’s (1975) outline that will be broken into phases and will take place over a two year period.
  • The first phase will begin in April/May, where the evaluator will begin the process of gathering of information by the evaluator who meet with the advisory group and have discussions with stakeholders who are known to have concerns, conflicts, or issues.  The evaluator and advisory group will conduct a document review. The evaluator will create a survey.
  • In early June, the survey will be sent out to all parents, volunteers, and organizers to complete.
  • In mid to late June, further data will be collected using a variety of methods to including interviews, observations, and focus groups.  
  • In late summer/early fall the data will be discussed and examined by the advisory group that will include the evaluator, various stakeholder representatives, as well as content experts.  Data collected through instrumentation and analysis of old documents will be reported to the advisory group by the evaluator. A list of recommendations for the next season will be made.
  • In late fall/winter, recommendations will be presented to the sports club and community centres, so that administrators can make program adjustments to address concerns.  
  • The process will repeat for two seasons so that additional data can be collected during the season.
  • At the end of the second season, a final report will be prepared by the evaluator that will provide a summary of the work, the recommendations, and their status of implementation.

Although program evaluation is not one of my duties or responsibilities in my role in the school division, having an understanding of these models and processes are important as a teacher as part of personal reflective process of evaluation of my own teaching practice, as well as working with others in theirs as it helps get a understanding of different perspectives and methodologies that can be accessed and used to improve a variety of factors that influence educational programs.

Rippey, R. (1977). Transactional Evaluation and the Improvement of Instruction. Educational Technology, 17(2), 7-11. Retrieved from http://www.jstor.org.cyber.usask.ca/stable/44421056

Taylor-Powell, E. (2006).  Evaluation Tips #35: Participatory Evaluation.  University of Wisconsin Extension:  Program Development and Evaluation.  Retrieved from https://fyi.uwex.edu/programdevelopment/files/2016/04/Tipsheet35.pdf.