The report covers Fall 2001 through Summer 2003.  This report overlaps slightly with our previous report that covered Spring 2000 to Spring 2002 (actually submitted Fall 2002).  Although the dates overlap, none of the student data has been duplicated.

Our assessment plan (approved in May 1998) includes four Assessment Components: (a) the Master’s degree (M.S.) written comprehensive examination, (b) the Master’s degree thesis and oral examination, (c) the NTE School Psychologist Specialty Area Exam administered by ETS, and (d) the Specialist degree portfolio and oral exam. Data from these components will be summarized below and the implications discussed for the School Psychology Program.

Our assessment data for this report is primarily summarized by Assessment Component, however a summary of Assessment Outcomes organized by the desired student outcomes identified in our original Assessment Plan is included in Table 1 [PDF].

Summary of Data
As a small graduate program, we have a limited number of students for whom we can report data. Therefore, data reported will be descriptive in nature with means and frequencies reported as appropriate.

M.S. Written Comprehensive Examinations. Five students took the written comprehensive exam in the Fall of 2002 and another three took the exam in the Fall of 2003. Each student responds to six questions and three readers rate the responses on a 0 to 5 point scale with 0 representing a Failing response and 5 representing an Excellent response. Ratings are then averaged for each question across the three readers. The student must receive an average rating of at least 3 in order to pass the question.  Five of the six questions must be passed in order to pass the exam. Students may re-take the exam once.

Average ratings earned across students for each of the six areas of the M. S. comprehensive exam areas are presented in Table 2 [PDF]. As may be seen, average ratings per question ranged from 3.40 to 3.80 in 2002 and 3.50 to 4.55 in 2003. These ratings represent performances primarily in the Good to Very Good range. All students taking the examination in 2002 and 2003 passed the exam the first time they took it. 

Master’s Degree Thesis and Oral Examination. At the request of the Assessment Committee in 1998, we created a rating form to be completed at the final oral examination for Master’s degree students. Forms were completed for 6 students completing theses in 2002 and 5 students completing theses in 2003. All students passed the oral exam. Ten of the eleven students needed minor changes only in their written papers with one student needing to make moderate changes before receiving final approval for the paper. 

NTE School Psychologist Specialty Area Exam. The NTE exam is a national multiple-choice exam for Specialist and Doctoral degree school psychologists. It has items clustered in six areas that closely parallel our Master’s written comprehensive exam areas. The total possible score on the exam ranges from 250-990. The national average range for the 2002 and 2003 exam periods was 660-750. This represents the middle 50% of scores earned by all examinees in the previous three years.Our passing score is set at 660. This is the same minimum score as that required for national certification as a school psychologist.

We have exam scores for the twelve students who completed their Specialist degrees in 2002 or 2003. Ten of these took the NTE exam in 2002 or 2003; two had taken the exam earlier.  All students taking the exam passed it on their first try. Scores ranged from a low of 670 to a high of 790. The mean of these 12 scores was 734 with a median score of 730. Scores of all but one student were 720 or higher.

Standard scores are not available for the five sub areas of the test, however, information is provided by ETS on the student’s raw score in each area and the average range for raw scores in that area. We do not require that students provide the subtest information and are missing this data for 2 of the 12 students. Two of the 10 students for whom we have data earned a raw score that was one point below the average range in one area each; otherwise all area scores were within or above the average range (96% of all possible area scores). The “Diagnosis and Fact Finding” area appeared to be the one in which our students performed the best (40% earning scores above the average range); otherwise performance was quite consistent across the categories.Overall, these data indicate that our students are coming out of our program with a solid knowledge base across all basic areas of school psychology, at least relative to this national standard.

Data reported in our 2000-2002 Assessment report had the mean total score at 754 with a median score of 740 and no one scoring below 730 (N = 8). The mean for this reporting period (mean = 734; N = 12) is somewhat below the prior mean, however the median score is very similar and the lowest scores earned (720; except for one student) are quite similar. This is also the first reporting period in which we have had 100% of students completing course requirements for the Specialist degree reporting exam scores. Thus, it is encouraging that our graduates are continuing to consistently do very well on this national exam. 

Specialist Degree Portfolio and Oral Exam. Since various faculty chair the orals committees, occasionally completion of the form is forgotten. We have rating sheets for 9 of the 12 students completing the Specialist oral exam and portfolio review between Spring 2002 and Summer 2003. While we only have the specific item ratings for 9 students, all 12 successfully passed the oral. A copy of the rating sheet with frequency counts for each area is attached (see Table 3 [PDF]). Most committees consider a rating of “Adequate” to represent a performance level that is appropriate for a professional with the level of experience that the student has had. A rating of “Strong” is used in those areas where the committee believes the student has relatively stronger skills that go well beyond the basic skills expected of a new professional. Based on the materials presented and the oral exam, all students were judged by their committee to have passed the oral and to have “demonstrated the ability to integrate knowledge and skills to effectively and positively serve children and youth as a school psychologist.”

As may be seen in Table 3 [PDF], in no case was the student rated deemed to have less than adequate skills in any area. Committees were particularly inclined to rate students as strong in the area of consulting effectively with teachers, parents and other professionals, and for effectively using technology. The lower rating on “knowledge base in education” relative to the other areas seems to reflect the fact that all of our graduates during this reporting period had an undergraduate background in psychology rather than education and had not had the opportunity to develop a strong background in this area. For some of the students there was also an issue with the usefulness of the content of one course taken in this area (this issue has since been resolved).

It should be noted that this is the first reporting period in which all students who completed all program coursework requirements also completed all additional degree requirements (the portfolio, orals and national exam) and earned their Specialist degrees.

Use of Assessment Data
The data collected for our prior reports were used in our university 6 year report (Spring 2002) and our folio submitted to the National Association of School Psychologists (NASP) for renewal of our status as a Nationally Approved Program (in partnership with NCATE). Additional data collected as part of these two major evaluations included a survey of graduates from the previous 5 years and a visit by a site evaluator. Our site evaluator met with students, faculty, administrators and local field supervisors.  In addition, a sub-committee of the Graduate Studies Council reviewed our 6 year report and the report of our evaluator.

The results of all of the evaluations done during the past two years, confirms that overall, our curriculum is providing our students with an appropriate knowledge base that is consistent with the program goals and objectives. Some issues were raised as part of the assessment process relevant to this report:

Knowledge education and use of interventions: Although our students demonstrate adequate skills in this area, it continues to be one we wish to strengthen. Three of the courses students take in this area are taught through the Special Education Department. We have consulted with the instructors of these courses and have been part of committees reviewing these courses in the Special Education Department. Based on these discussions, changes have been made in these courses that will make them more useful to our students. We have already received positive feedback from students on these course changes.

Technology: Use of technology had been a weakness noted in previous assessments. We have improved in this area and now have some graduating students where technology use is noted as a strength. Several of our reviewers encouraged us to explore new ways of using technology, particularly in course delivery. One way this has been addressed is through exploring several on-line course meeting techniques in one of our advanced courses (Psy 728). These approaches not only make the class more accessible to our students doing internships at a distance, but give them experience with various on-line communication skills.

Assessment and data-based decision making skills: Our students consistently have demonstrated strong skills in the area of assessment and data-based decision making. However, many of the instruments they must learn to use have been recently revised, and if we were not able to upgrade this equipment, the quality of our students’ education in this area would be seriously jeopardized. This issue of maintaining current instructional equipment was addressed with academic administrators throughout the assessment process in 2002. As a result of subsequent budgetary support at the department and academic affairs levels during the past two years, we have been able to obtain needed equipment. We also made enough profit from a workshop we sponsored to add to this equipment and to replace the computer dedicated to student use/instruction that we noted in our last report needed replacement (this also addresses the Technology component discussed above).   

Revisions to the Assessment Plan
We developed three effectiveness goals to add to our Assessment Plan:

• To provide a positive and collegial atmosphere that facilitates student learning and professional development.

• To provide support systems (materials, information, etc.) that facilitate student learning.

• To provide a high quality total educational experience that efficiently and effectively meets the needs of students.

These goals will be assessed through a survey administered to our advanced graduate students at the end of their third year in the program, most likely in their Systems Intervention course. 

A completed Assessment Grid for these goals is attached along with a copy of the proposed survey.

A member of the Minnesota State Colleges and Universities System
an equal opportunity educator and employer | Accessibility Questions? | Contact assess@mnstate.edu | Updated 1/11/06
MSU Moorhead | 1104 7th Ave South | Moorhead, MN 56563 | 1.800.593.7246
home | msum | committee members | faqs | forms | grants | learning improvements
plans and reports | policies and procedures | resources | schedule status