Assessment of Student Learning - Minnesota State University Moorhead

Physics and Physics Education
Assessment Report
January 2004

Assessment measures in the liberal arts astronomy and physics courses for non-majors

One of the major changes to the curriculum made since the last review was to add a 70-minute activity period in the liberal arts courses in introductory astronomy, AST 102 and AST 104. This 70-minute activity period takes the place of a third hour of lecture for a week. So far, it has been a challenge to assess the effectiveness of the activities in a reliable manner. Each activity requires students to answer a series of summary questions, to determine understanding of the core concepts. The ultimate goal is to represent these questions on final examinations. All instructors of AST 102 and AST 104 will ask a subset of questions that are identical, and tie back to key activities done during the semester. The statistics on this subset will then be tracked.

The other liberal arts courses that we teach also have an activity component, including PHY 101 (introductory physics) and PHY 105 (Physics of Music). PHY 101 includes a project, in which written reports are assessed. The activities in PHY 105 are also assessed via written reports.

Assessment measures in the physical science course

The Physical Science Course, PSCI 170 has been steadily evolving over the past four years. It has changed from PHY 120, a course taught solely by physics faculty, to PSCI 170, a course taught by both physics and chemistry faculty. Changes have occurred in response to changes in the MN Board of Teaching standards for science teaching. The course now reflects the standards for physical science at the junior high school level in MN public schools.

The one factor that has remained the same concerning the course is the hands-on, activity-based nature of the course. We have also maintained the requirement for completing a group project, in which a group of two to three students develops a scientific question, investigates the question, and communicates their findings orally (during a poster session) to the rest of the class, and via a written report to the instructor. The quality of these reports has been steady over the past 6 years. However, we have added a peer-evaluation component, in which classmates rate each presentation as if they were judging entries in a science fair.

Assessment measures in the introductory physics sequences

We have been continuing to use nationally known tests in the sequences PHY 160/161 (algebra-based physics) and PHY 200/201 (calculus-based physics). In Fall semester, both sequences involve lecture and hands-on learning of mechanics, including kinematics, dynamics, rotational motion, and conservation laws. The students are pre-tested at the beginning of the semester using the Force Concept Inventory (FCI), and the Force and Motion Concept Evaluation (FMCE).

The FCI results are easier to compare to national data. Final scores on the FCI range from 47-79% in algebra-based physics courses. The average in Physics 160 over the four years from 2000 through 2003 was 50%, near the bottom of the nationally reported range. However, the pre-test score for MSUM students, averaged over the two years for which we have pre-test data, is 26%, somewhat lower than the national range of 30-42%.

A more useful measure of student learning on the FCI is the normalized gain (NG), defined by NG=(Post-Pre)/(100-Pre). The normalized gain measures the percentage improvement in student scores. The normalized gain at MSUM, averaged over the years for which we have pre- and post-test data, is 0.33. This compares favorably with results seen in the literature[1], where the range in NG is 0.2-0.65. Clearly our students benefit from Physics 160, though there is room for improvement.

For the FMCE we have four years of post-test data and three years of pre-test data. Over the years for which we have pre-test data the average score was 17%. The average post-test score was 58%, a clear improvement in performance. Unfortunately it is impossible to compare this data with national data presented in the literature. The data in the literature[2] is presented in terms of scores on selected groups of FMCE questions, data that we have not accumulated. Nationally, pre-test scores on the groups of questions range from 10-20%, and scores on the post-test range from 20-90%. Given the difference between the scores we have accumulated and the scores that are presented in the literature, the safest conclusion is simply that students have shown a clear increase in their performance on this test.

The Conceptual Survey of Electricity and Magnetism (CSEM) has been used to assess student learning in Physics 161 and most recently, in PHY 201. Pre- and post-test data for students at other institutions are available in the literature[3] for algebra-based physics courses. The table below summarizes the average performance of MSUM students over the period 2000-2003 with 262 students in a similar course at other institutions.

	CSEM Pre	CSEM Post	Normalized gain
Physics 161	24%	36%	0.16
Other institutions	(25±8)%	(44±13)%	0.25

Students show a clear gain in performance, but there is room for improvement.

We have not tracked the pre and post-test results for as long in PHY 200. Another aspect of this course is that, due to the 20 to 30 student size of the class, statistics are not as reliable as for the PHY 160/161 sequence.

About the largest change we have made during the past six years is a scaling back of the Workshop Physics approach to teaching these courses. We have returned to a more standard lecture-lab format, due to requests from other departments whom we serve. Our labs have remained very hands-on and inquiry-based. In PHY 160 we have been basically modeling our activities after the Real Time Physics curriculum by Thornton, Sokoloff, and Laws. We believe that more hands-on experience with electricity and magnetism may improve student learning, especially for the PHY 161 students. Current direction in our department is to develop activities in electricity and magnetism for both PHY 161 and PHY 201 students, as well as improve assessment of student learning in the areas of DC circuits and optics.

Assessment measures of physics majors

We require of all physics majors the completion of a Senior Project. All department faculty are responsible for evaluating the oral and written reports that result from the project. In addition to the Senior Project, we also ask physics majors to complete a portfolio and submit to the department. However, since we have not enforced the portfolio requirement, only a fraction of our majors have completed the portfolio.

We have also tried giving seniors a set of post-test exams the semester prior to graduation. The post-tests include the FCI and CSEM, exams given to students in PHY 160/161 and PHY 200/201. So far we have gotten results for only three majors, so it is difficult to draw conclusions. However, the physics majors performed slightly better than students in PHY 200/201 on both exams.

We also wrote a post-test exam testing students’ problem-solving skills in fundamental areas of physics: modern physics, thermodynamics, electricity/magnetism, and mechanics. The same subset of three students were given the exam. Experience with the three students was rather disappointing, as the results show the questions were not taken seriously. These efforts (post-test, portfolio) point out that in order to assess our students, we have to develop tools in which the students have some investment. We have discussed tracking GRE scores on the physics portion of that exam, for instance

Some of the best information we have obtained concerning our physics major is from graduation interviews conducted by our Academic Vice President, Bette Midgarden. Dr. Midgarden asks for feedback from each of our seniors on an individual basis. From these interviews, we have learned that our majors are overall satisfied with the education they received, and report good rapport with the faculty. They have been most dissatisfied with the equipment available for classes and research at the advanced level, and have also felt that learning would be improved by having more physics majors with whom to interact.


[1] “Interactive-engagement versus traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses,” Richard R. Hake, Am. J. Phys. 66, 64 (1998); "Interactive-engagement methods in introductory mechanics courses," Richard R. Hakes, submitted on 6/19/98 to the "Physics Education Research Supplement to AJP"(PERS) [apparently never published, but available as a preprint from http://www.physics.indiana.edu/~sdi/IEM-2b.pdf; contains the underlying data for the results presented in the previous article published in the American Journal of Physics]

[2] “Assessing student learning of Newton's laws: The Force and Motion Conceptual Evaluation and the Evaluation of Active Learning Laboratory and Lecture Curricula,” Ronald K. Thornton and David R. Sokoloff, Am. J. Phys. 66, 338 (1998)

[3] “Surveying students' conceptual knowledge of electricity and magnetism,” David P. Maloney, Thomas L. O'Kuma, Curtis J. Hieggelke, and Alan Van Heuvelen, Am. J. Phys. 69, S12 (2001)