Assessment of Student Learning - Minnesota State University Moorhead

Assessment data from spring and summer 1999 includes the National Examination in Speech-Language Pathology and Audiology (NESPA) (See summary of scores); faculty rating of performance during oral examination for the master's degree (See score form/summary); faculty and off-campus clinical supervisor rating on Evaluation of Clinical Competency (See score form/summary); and student self-rating.

Curriculum
There are two direct measures of student knowledge in our graduate assessment plan the nationally standardized NESPA test and an oral examination.

NESPA Test
This test is usually taken by beginning professional speech-language pathologists as one of the requirements for the Certificate of Clinical Competence from ASHA. Our graduate students take this test as their comprehensive exam for the Graduate School In 1999, 13/13 students who took the test received a standard score of 600 or above which is the level required for ASHA certification For assessment of student learning purposes, 9 students volunteered to report their scores on three separate areas of content (See summary): Basic professional knowledge, speech-language disorders and assessment, therapy, and regulations. The first two areas relate to our knowledge goals (goal #1,2) while the third relates to clinical skills competency (goal #3). Our students performed very similarly to the national average in each area. For basic knowledge, the national average was 18-21 while the MSU average was 21.2 (SD 5.09). For speech-language disorders, the national average was 41-52 while the MSU average 50.1 (SD 7.89). Finally, for assessment, treatment, and regulation, national average was 17-22 while the MSU average was 20.8 (SD 3.33). These scores indicate that our students demonstrate the whole range of basic knowledge and clinical knowledge expected for beginning professionals in speech-language pathology. See attached summary of national score results and MSU graduate student results.

Oral Examination
Four faculty members rate graduate students as they answer four questions during an oral examination. Each question is rated on five criteria (see attached form summary). Results are reported for eight students who took this exam during spring semester 1999. This year's data indicates that students are scoring at an adequate level (mean 15, possible =20 points per question). However, it is difficult to assess specific learning outcomes using the present rating form. The SLHS department plans to revise the rating form to include two sets of scoring criteria. The first set will include organization, relevance to issue; and clarity and specificity which relate to learning outcome #4 (demonstration of skill in oral language). The Second set will include support of viewpoints, and answering impromptu questions which relate to learning outcome #2 (synthesis and evaluation of specialized information). Data will be gathered beginning spring semester (2000) using the revised form.

Clinical Competency rating
The Evaluation of Clinical Competency rating is completed by both on campus and off campus supervisors during our students' clinical experiences. Individual students are assigned numbers so that it will be possible to track their competency across experience levels from entry to advanced. There were 9 graduate students rated so far this year. Four students at the intermediate level (35-120 clock hours), and five students at the advanced level (120+ clock hours). Supervisor ratings of assessment and treatment competencies for both intermediate and advanced groups were more than adequate (range of means 5.5(1.22) assessment skills to 6.6(0.90) interactive skills).

We have little data yet to use for tracking students across all three levels of clinical experience. However, of the 34 different students followed so far, 5 have ratings reported across all experience levels. It is interesting to see that with this preliminary data, students are improving mean scores across level of expectation as well as meeting the new requirements for increased independence at each successive level. Initial interpretations of such trends should be possible by fall 2000.

Graduate student exit questionnaires.
There were only two graduate student exit questionnaires submitted during spring 1999.
This year we will try to increase this rate of response by included ft on a checklist of
procedures necessary for graduation.

Budgeting: There are no budgetary implications from these initial results. However, continuing the assessment process as it was approved will require continued support.

Retention/Public Relation: The initial data show positive movement. Students are achieving at adequate or above levels on all of the assessment measures at the graduate level We should be able to disseminate positive information on student success both locally and compared to national results.

Possible assessment plan revision: We will revise oral examination form as discussed above so that ft better reflects specific goals in our plan. We should also consider defining target values for mean ratings and their deviations.
SLHS-Assessment of Student Learning-NESPA Scores-1999

The NESPA is a nationally standardized test usually given to professional speech language pathologists as one of the requirements for the Certificate of Clinical Competence from ASHA. OUT graduate students take this test as their comprehensive exam for the Graduate School In 1999, 13 students took the NESPA and passed. For assessment of Student learning purposes, 9 students volunteered to report their scores on three separate areas of content: evaluation skills, plan/implementation remediation skills, and administration skills.

Total Score Comparisons
National Averages
1995-1998 Total Test Takers: 17,390
Median Score: 670
Average Range of the Middle 50%: 630-710
SEM: 24

MSU Average
1999 Total Test Takers: 13
Median Score: 690
Average Score: 688
SD: 58

Students will Initially answer four questions. Please record points (1-5) (see below) for each individual question. With four committee members voting, there are 20 points possible per question. Ten points is passing. If one question is failed, the fifth question will be asked immediately and scored. Four questions must be passed to successfully complete the oral examination.

Please consider these criteria when you assign a point value for each question:
Organization; relevance to issue; clarity avid specificity; ability to support viewpoints; and ability to answer impromptu questions.

Range of Total Scores: 40.5-72.5
Average Total: 59.25
SD: 11.06
(Total Possible 80)

SLHS Assessment of Student Learning-Evaluation of Clinical Competencies Score Form-l999

There were 12 student clinicians rated using competency forms during the 1999 academic year (so far). The categories reviewed include assessment skills, treatment skills, interactive skills, professional characteristics, application of current technology, and clinical writing. Each category has specific subcategories on which the clinician is assessed. The clinician's assessment is also based on higher level of experience-entry (0-35 clock hours), intermediate (35-120 clock hours), and advanced (120+ clock hours). There were no entry-level clinician competencies, there were five intermediate, and there were four advanced clinicians.

Adequate progress in skill development is judged to be a minimum average score of 5 at each level of experience. The range, mean and SD for each level are listed below.

It is of interest to observe individual student improvement over their educational clinical experience. Of the 37 student's data reports collected over four semesters, we have three different clinical competency reviews for 5 students. The following table summarizes the average gain for these students across skill areas

Student #	Assessment	Therapy	Interactive	Professional	Technology	Writing
11	+1.1	+3.3	+4.1	+3.3	NA	+3.9
12	+2.0	+3.6	+3.4	+3.2	+4.0	+4.0
13	+0.3	+1.3	+1.8	+3.0	+2.5	+1.8
14	+0.9	+1.6	+2.5	+2.8	+2.0	+2.8
22	+0.1	+0.0	+1.0	+0.0	+0.0	-0.6