School Psychology
Assessment Report
2000-2002

Margaret Potter, Director of the School Psychology Program, prepared this assessment report. The School Psychology Coordinating Committee (SPCC), which is the program advisory committee, reviewed this document and provided input. 

The report primarily covers Spring 2000 to Spring 2002. Our assessment plan (approved in May 1998) includes four Assessment Components: (a) the Master’s degree (M.S.) written comprehensive examination; (b) the Master’s degree thesis and oral examination; (c) the NTE School Psychologist Specialty Area Exam administered by ETS; and (d) the Specialist degree portfolio and oral exam.  Data from these components will be summarized below and the implications discussed for the School Psychology Program.

Our university 6-Year Review self-report and the external evaluator’s visit were completed in the Spring of 2002. Concurrently, we were completing the comprehensive review folio required for maintaining approval by the National Association of School Psychologist, an NCATE affiliate. The national training standards recently have been modified and now require considerable outcomes assessment data. As part of that review process, we conducted a survey of recent graduates from the past five years. We also compiled summary data from internship supervisor ratings. Summaries of those data are not included in this report since they are not part of our current assessment plan, however, they are available upon request.

Our assessment data for this report is primarily summarized by Assessment Component, however a summary of Assessment Outcomes organized by the desired student outcomes identified in our original Assessment Plan is included in Table 1.

Summary of Data
As a small graduate program, we have a limited number of students for whom we can report data.  Therefore, data reported will be descriptive in nature with means and frequencies reported as appropriate.

M.S. Written Comprehensive Examinations. Four students took the written comprehensive exam in the Fall of 2000 and another seven took the exam in the Fall of 2001.  Each student responds to six questions and three readers rate the responses on a 0 to 5 point scale with 0 representing a Failing response and 5 representing an Excellent response. Ratings are then averaged for each question across the three readers. The student must receive an average rating of at least 3 in order to pass the question. Five of the six questions must be passed in order to pass the exam.  Students may re-take the exam once.  

Average ratings earned across students for each of the six areas of the M. S. comprehensive exam areas are presented in Table 2. As may be seen, average ratings per question ranged from 3.08 to 4.17 in 2000 and 3.57 to 4.09 in 2001. These ratings represent performances primarily in the Good to Very Good range.  While one student in 2001 failed to achieve a rating of 3.0 on one question, all students taking the examination in 2000 and 2001 passed the exam. 

Master’s Degree Thesis and Oral Examination. At the request of the Assessment Committee in 1998, we created a rating form to be completed at the final oral examination for Master’s degree students.  Since various faculty chair the M.S. committees, occasionally completion of the form is forgotten.  Forms were completed for 5 of 6 students completing theses in 2000 and 2 of four students completing theses in 2001. All students passed the oral exam. Six of seven students needed minor changes only in their written papers with one student needing to make moderate changes before receiving final approval for the paper. 

NTE School Psychologist Specialty Area Exam. The NTE exam is a national multiple choice exam for school psychologists with items clustered in six areas that closely parallel our Master’s written comprehensive exam areas. The total score on the exam ranges from 250-990. The average range of 650-740 or 660-750 (it varies slightly from exam period to exam period) includes the middle 50% of scores in the normative group. Our passing score is set at 660. This is the same minimum score as that required for national certification as a school psychologist.

We have exam scores for eight students who took the NTE exam in 2000 or 2001. Scores ranged from a low of 730 to a high of 790. The mean of these eight scores was 754 with median and modal scores of 740.  Thus, the performance of our students on this objective standardized national exam was at approximately the 75^th percentile as compared to scores of other school psychologists and advanced school psychology students taking this exam. The lowest score earned by our students was still well above the middle of the average range with all other scores at the high end of the average range or above the average range. 

Standard scores are not available for the six sub areas of the test, however, information is provided by ETS on the student’s raw score in each area and the average range for raw scores in that area. It should be noted that the specific area categories changed between 2000 and 2001 so area scores are not comparable across administrations. One of our students earned a raw score that was one point below the average range in one area (Applied Education); otherwise all area scores were within or above the average range. The Professional Practices/Ethics area seemed to be the area in which students were most likely to score above the average range. It was noted in our last report that the Applied Education area was the one area in which no one scored above average. This time 4 of the 8 students earned above average scores in this area.  Overall, these data indicate that our students are coming out of our program with a solid knowledge base across all basic areas of school psychology, at least relative to this national standard.

Data reported in our 2000 Assessment report had the mean total score at 753. While this is essentially identical to the mean total reported here, the range in 2000 was much larger (690-830) and the exam scores were likely to be reported to us only by those students motivated to complete all requirements for the Specialist degree rather than just licensure requirements. In the present reporting period, we had no one score below 730. We are also now getting scores reported by a higher percentage of students completing the program (100% of students completing course requirements in 2001 reported exam scores). Thus, it is encouraging that all of our most recent students are consistently doing very well on this national exam. 

Specialist Degree Portfolio and Oral Exam. We have rating sheets for 9 students completed as part of the Specialist oral exam and portfolio review between Spring 2000 and Fall 2001. A copy of the rating sheet with frequency counts for each area is attached (see Table 3). Most committees consider a rating of “Adequate” to represent a performance level that is appropriate for a professional with the level of experience that the student has had. A rating of “Strong” is used in those areas where the committee believes the student has relatively stronger skills that go beyond the basic skills expected of a new professional. Based on the materials presented and the oral exam, all students were judged by their committee to have passed the oral and to have “demonstrated the ability to integrate knowledge and skills to effectively and positively serve children and youth as a school psychologist.”

As may be seen in Table 3, in no case was the student rated deemed to have less than adequate skills in any area. Committees were particularly inclined to rate students as strong in the area of consulting effectively with teachers, parents and other professionals, and for demonstrating the ability to design and implement interventions. The positive ratings in the intervention area are in contrast to ratings reported in our last assessment report where the interventions item was much less likely to be rated as “strong” as compared to other items rated.

Curriculum
The present results suggest that, overall, our curriculum is providing our students with an appropriate knowledge base that is consistent with the program goals and objectives. Two areas that were noted to be relatively weaker in our assessment report of 2000 were the areas related to a) the effective use of technology and b) knowledge and use of interventions. 

We have increasingly incorporated the use of technology into the curriculum to the point where it is more of an integral part of the curriculum than it used to be. While this continues to be an area to work on, it is a stronger area than it used to be.

We have long been aware that we need to place increasing emphasis on the knowledge and use of both academic and social/emotional interventions. We seem to have made some progress in this area, but it is an area of the curriculum that continues to need strengthening and we will look at this as we consider possible modifications to our curriculum.

In the last assessment report, it was noted that the SPCC was concerned that the results of the Master’s degree written comprehensive exams were not as reflective of the knowledge base of our students as they could be. The SPCC did consider potential modifications in the structure of the exams, as well as the need for more comprehensive instructions for the faculty who write questions for the exams and who serve as readers. We did not change the structure of the exams, however, we have improved communication among writers of exam questions and are now providing more guidance to exam readers.

Although it did not specifically come up in assessment data, an area of the curriculum that we have made concerted efforts to strengthen over the past two years has been in regard to cultural diversity issues. Here, again, we believe we have made some progress, but we are struggling with not only how to improve our curriculum in this area, but also how to measure outcomes.

Budgeting
As also noted in our last report, one budgetary implication is in regard to continuing to acquire software for such needs as computer based observation systems and assessment scoring/interpretation programs. We have been able to acquire the necessary hardware and software to meet present needs, however, the computer in the School Psychology library, the video-conferencing equipment and the laptops we purchased with Strategic Initiative money are now several years old and will likely need to be upgraded or replaced in the not-too-distant future.

A potentially even more critical budgetary issue is in regard to maintaining the adequacy of our training in assessment methods. We will need to spend a significant amount of money on revisions of assessment instruments in the near future. Under our current budgetary model, as a program within the Psychology Department, needs of the School Psychology Program need to be balanced with the needs of the undergraduate program, as well as with the needs of the College as a whole. Since assessment instruments have become very costly, it is becoming increasingly difficult to meet these needs while addressing the priorities of the department much less the College. This was an issue noted in our last assessment report. We have been fortunate that we have had to purchase relatively little in the way of major assessment instruments in the past two years, however, a number of key intelligence tests will have revisions published in the next two years. These are materials that will be vital to maintaining the quality of our training and the purchase of which cannot be delayed from year to year.

Retention
Our assessment data indicate that we are meeting the educational needs of our students. Once they have started the program, we have an almost 100% retention rate, at least through the Master’s and the Specialist coursework. A long standing difficulty was getting students to not only complete the coursework and internship requirements needed for state licensure, but to also complete the actual Specialist degree. This was a need noted in our last assessment report. We did dramatically increase the rate of degree completion among our students completing their internship in 2001 by providing increased structure and incentives for completion of the Specialist portfolio during the internship year. All of the 2000-2001 interns and the 2001-2002 interns did complete the Specialist degree. 

Public Relations
Application numbers have been down during the past few years.  We have taken steps over the past two years to increase the visibility of the program through a web site, temporary and permanent on-campus displays, regional mailings, visits to area campuses, as well as visits to classes/groups on our own campus. Still, we need to do better job of communicating not only the strengths of our program, but positive aspects of the field of school psychology. The department chair submitted and received a grant proposal to cover the cost of a display case so we can permanently display information about the program on-campus (this is located on the third floor of Bridges). A new version of the APA guide to graduate study in psychology will be appearing this spring with updated information about the program. After the university name change, we updated information to be disseminated to prospective students incorporating more information about the strengths of our program and our students. Regularly updating our web site (www.mnstate.edu/gradpsyc) to include more data about our program is another way to communicate to prospective students and the public.  Feedback from prospective students and the amount of correspondence we now get via the internet suggests that these have been useful steps.

Visibility of our program regionally and nationally has been strengthened over the past two years through numerous presentations and inservices done by program faculty. We also have had current students and recent graduates present papers/posters at national conferences each of the last three years.

Revisions to the Assessment Plan
While we have not revised our university assessment plan within the past two years, our current national approval review process and our university 6-Year review may lead to changes in our assessment procedures and in our university assessment plan. For example, we may include the 5-year alumni survey as a standard part of the assessment plan. This survey has included questions related to effectiveness of advising that may serve well in providing data relevant to institutional effectiveness.

Table 1
Assessment Outcomes by Desired Outcomes

Table 2
Master’s Comprehensive Exams
Average Ratings by Area

Note: Responses for each student are rated by 3 readers. The average of the ratings across the 3 readers is the student’s score for that Area. Students must earn a rating of at least 3.0 to pass the Area and must pass 5 of 6 Areas to pass the Comprehensive Exam as a whole.

Table 3
School Psychology Program
Specialist Portfolio Rating Sheet

  Summary Ratings for Spring 2000 through Fall 2001 (N = 9)

Note: Most committees consider an Adequate Rating to represent the level of skills expected of a student at the end of his/her training. Strong ratings are given to acknowledge a student’s particular strengths.  

Overall:  Has the student demonstrated the ability to integrate knowledge and skills to effectively and positively serve children and youth as a school psychologist?    
    9 YES         0  NO

A member of the Minnesota State Colleges and Universities System
an equal opportunity educator and employer | Accessibility Questions? | Contact assess@mnstate.edu | Updated 1/11/06
MSU Moorhead | 1104 7th Ave South | Moorhead, MN 56563 | 1.800.593.7246
home | msum | committee members | faqs | forms | grants | learning improvements
plans and reports | policies and procedures | resources | schedule status