Margaret Potter, Director of the School Psychology Program, prepared this assessment report. The School Psychology Coordinating Committee (SPCC), which is the program advisory committee, reviewed the draft and suggested changes. The SPCC also used information contained in this report to make one immediate policy change and to identify several areas where further discussion of program policy and practice is needed.

The report primarily covers the 1998-1999 academic year, but also includes available data from Fall semester 1999. Our assessment plan that was approved in May, 1998, included four Assessment Components: (a) the Master's degree (M.S.) written comprehensive examination; (b) the Master's degree thesis and oral examination; (c) the NTE School Psychologist Specialty Area Exam administered by ETS; and (d) the Specialist degree portfolio and oral exam. Data from these components will be summarized below and the implications discussed for the School Psychology Program.

As a small graduate program, we have a limited number of students for whom we can report data. As will be seen, the Ns in most cases are approximately six students. Therefore, data reported will be descriptive in nature with means and frequencies reported as appropriate.

M.S. Written Comprehensive Examinations. Six students took the written comprehensive exam in the Fall of 1998 and another six took the exam in the Fall of 1999. Each student responds to six questions and three readers rate the responses on a 0 to 5 point scale with 0 representing a Failing response and 5 representing an Excellent response. Ratings are then averaged for each question across the three readers. The student must receive an average rating of at least 3 in order to pass the question.  Five of the six questions must be passed in order to pass the exam. Students may re-take the exam once.  

Average ratings earned across students are presented in Table 1.  As may be seen, average ratings per question ranged from 3.5 to 4.0 in 1998 and 3.3 to 4.3 in 1999. These ratings represent performances primarily in the Good to Very Good range. All students taking the examination in the Fall of 1998 passed the exam. Two students in the 1999 group failed more than one question, and therefore, will be re-taking the exam in the spring of 2000. The difficulties these two students had with Areas I, II, and IV, in particular, brought down the average ratings in these areas for the 1999 group.  

Master's Degree Thesis and Oral Examination.  At the request of the Assessment Committee in 1998, we created a rating form to be completed at the final oral examination for Master's degree students. Unfortunately, because of a change in program director for the 1998-1999 academic year and lack of awareness among thesis committee chairs of this new form, it was not completed for the students taking their Master's oral exams last year. Therefore, while seven students successfully completed theses and passed their final oral exams, we do not have ratings sheets for them. 

NTE School Psychologist Specialty Area Exam. The NTE exam is a national multiple choice exam for school psychologists with items clustered in six areas that closely parallel our Master's written comprehensive exam areas. The total score on the exam ranges from 250-990. The average range of 650-750 includes the middle 50% of scores in the normative group.  Our passing score is set at 630. 

Seven students submitted scores for the NTE exam during this evaluation period. Scores ranged from a low of 690 to a high of 830. The mean of these seven scores was 753 with median and modal scores of 750. Thus, the performance of our students on this objective standardized national exam was at approximately the 75^th percentile as compared to scores of other school psychologists and advanced school psychology students taking this exam.  The lowest score earned by our students was still in the middle of the average range with all other scores at the high end of the average range or above the average range. Standard scores are not available for the six sub scales of the test, however, information is provided by ETS on the student's raw score in each area and the average range for raw scores in that area. None of our students earned raw scores below the average range in any of the sub areas; 63% of the raw scores earned were either above the average range, or the top score in the average range. The one area in which no student earned an above average score was the Applied Education Foundations area that taps knowledge of “exceptionalities, instruction, schools.” Still, this data indicates that our students are coming out of our program with a solid knowledge base across all basic areas of school psychology, at least relative to this national standard.

Specialist Degree Portfolio and Oral Exam. We have rating sheets for five students completed as part of the Specialist oral exam and portfolio review. A copy of the rating sheet with frequency counts for each area is attached. Most committees consider a rating of “Adequate” to represent a performance level that is appropriate for a professional with the level of experience that the student has had. A rating of “Strong” is used in those areas where the committee believes the student has relatively stronger skills that go beyond the basic skills expected of a new professional.  As may be seen in Table 2, in no case was the student rated deemed to have less than adequate skills in any area. All students received ratings of Strong for assessment skills and four of five received a Strong rating for effective writing skills.  Relatively weaker areas were in regard to the effective use of technology and the design and implementation of interventions.

The present results suggest that, overall, our curriculum is providing our students with an appropriate knowledge base that is consistent with the program goals and objectives. Two areas that may be relatively weaker than other areas are the effective use of technology and knowledge and use of interventions. Our current students are increasingly using technology, but this will be an area where we need to continue focus on upgrading both faculty and student use of available and emerging technology. 

We have long been aware that we need to place increasing emphasis on the knowledge and use of both academic and social/emotional interventions. We have made improvements in this area, but it is an area of the curriculum that continues to need strengthening and we will look at this as we consider possible modifications to our curriculum.

Our original passing criteria of a minimum score of 630 on the NTE School Psychology exam was set several years ago when few of our students had taken the exam. Based on the scores reported in this assessment report, it appears that our students have little difficulty meeting this criterion. Since the cutoff score for the national certification system is 660, the SPCC decided to raise our criterion to 660 to be consistent with the national standards.

The SPCC is concerned that the results of the Master's degree written comprehensive exams are not as reflective of the knowledge base of our students as they could be. The SPCC currently is in the process of considering potential modifications in the structure of the exams, as well as the need for more comprehensive instructions for the faculty who write questions for the exams and who serve as readers. 

Budgeting

One budgetary implication is in regard to continuing to acquire software for such needs as computer based observation systems and assessment scoring/interpretation programs. We have been fortunate to have several of our more immediate needs have been met through special source funding in the past year (including a Strategic Budget grant for laptop computers and videoconferencing equipment, and university purchase of a teleconferencing phone compatible with the videoconferencing system), however, there will continue to be the need to upgrade the current technology. 

A second major budgetary implication is in regard to maintaining the adequacy of our training in assessment methods. We will need to spend a significant amount of money on revisions of assessment instruments in the near future. Under our current budgetary model, as a program within the Psychology Department, needs of the School Psychology Program need to be balanced with the needs of the undergraduate program. Since assessment instruments have become very costly, it is becoming increasingly difficult to meet these needs while addressing the priorities of the department as a whole.

Our assessment data indicate that we are meeting the educational needs of our students.  This year we had several students withdraw within the first few weeks of school, but that was somewhat unusual. Once they have started the program, we have an almost 100% retention rate, at least through the Master's and most of the Specialist coursework. It is at the point of completing the thesis requirement that we have lost some students, and a number of others complete all requirements except the final portfolio and oral for the Specialist degree. Our data indicate that the general knowledge base of our students is strong, and they are successful in field placements. We need to continue to search for ways to increase the actual degree completion rate for these students. 

Application numbers have been down during the past few years. Although we have taken some steps this year to increase the visibility of the program through a web site, temporary on-campus displays, and regional mailings, we need to do better job of communicating the strengths of our program. The department chair has submitted a grant proposal to cover the cost of a display case so we can permanently display information about the program on-campus. A new version of the APA guide to graduate study in psychology will be appearing this spring with updated information about the program. After the university name change, we will update information to be disseminated to prospective students and will look at how to incorporate more information about the strengths of our program and our students. Updating our web site (www.mnstate.edu/gradpsyc) to include more data about the strengths of our program is another way to communicate to prospective students and the public.

Since the assessment plan is still new, we are not making significant changes at this point.  Instead, we will focus on being more consistent in data collection, especially in regard to completing the ratings sheets for the Master's and Specialist degree oral exams. We are also looking at more effective ways to do on-going evaluations of current students, and may consider adding that component to the assessment plan in the future. We also are considering adding an assessment component that tracks rates of admission, acceptance, enrollment, and retention.

Numbers under each rating area represent the number 
of students (out of the 5 rated) who received that rating.

5 (Excellent):                

The response attended to all of the main points expected.  The response was well-written, well-integrated, and well-conceptualized. Student showed evidence of an excellent knowledge base in the area being rated.

The response was better than average but not quite good enough for a 5. Student showed evidence of a very good knowledge base in the area rated.