OutcomesAssessment

Department of Materials Science and Engineering

Virginia Polytechnic Institute and State University

College of Engineering
Virginia Polytechnic Institute and State University

OUTCOMES ASSESSMENT REPORT
Undergraduate and Graduate Programs

Submitted to the State Council for Higher Education in Virginia (SCHEV)

Spring 2003

Prepared by
S.L. Kampe, MSE Curriculum Committee Chairman
W.T. Reynolds, Jr. Graduate Program Chairman
David E. Clark, Department Head

The Materials Science and Engineering (MSE) Department offers the Bachelor of Science degree in Materials Science and Engineering, as well as the Master of Science, Master of Engineering, and PhD graduate degrees in MSE. The present-day MSE Department comprises the disciplines previously served at Virginia Tech by the Departments of Metallurgy or Metallurgical Engineering (1906-1964), of Ceramics (1928-1964), of Metals and Ceramic Engineering (1964-1976), and of Materials Engineering (1976-1992). The expanding identity and scope of the Department reflects the profession's prominence within, and importance to, the engineering community and society. The discipline of Materials Science and Engineering is central to several identified strategic college, university, state, and national initiatives, notably biotechnology, nanotechnology, information technology, vehicular transportation systems, national defense and security, and environmental stewardship. MSE graduates at Virginia Tech receive formal education relevant to all conceivable engineering material forms in support of these and other areas, including (but not limited to) metals, ceramics and glasses, polymers; electronic, magnetic, and photonic materials; composites, and biomaterial forms. MSE's graduates receive instruction in the structure, properties, processing, manufacturing, and application of materials for engineering systems.

Additional details and information regarding the MSE program at Virginia Tech can be found in the University's Undergraduate and/or Graduate catalogs, and at the MSE Department's website: http://www.mse.vt.edu

MISSION
The Department is guided by its Program Objectives, which are:

provide a broad, innovative, and quality education to men and women in all aspects of materials science and engineering; and
ensure that our graduates are well-rounded, competitive, and prepared for either careers in industry or advanced studies in materials science and engineering or other graduate programs

EDUCATIONAL OUTCOMES AND BASES FOR ASSESSMENT
The Department's mission is executed by ensuring that certain outcomes are achieved within the curriculum. The desired outcomes form the bases for the Department's assessment strategies; that is, they provide goals capable of being measured quantifiably as part of a regular and ongoing assessment program. Specifically, the MSE Department strives to graduate professional, competent scientists and engineers who are:

proficient in the fundamental use of mathematics, physics, chemistry, and engineering science;
proficient in the use of the basic concepts of materials science and engineering, covering all classes of materials, including ceramics, composites, electronic/optic/photonic/magnetic materials, metals, and polymers;
capable of studying materials on scales ranging from the electronic and atomic through the microscopic and macroscopic;
capable of developing new materials, improving traditional materials, and developing new methods to produce and manufacture materials;
capable of characterizing a material's properties using modern measurement techniques;
able to predict and evaluate the performance of real materials as functional elements of engineering systems;
able to select, specify, and design materials that are appropriate for broad ranges of application;
able to express their thoughts, ideas, and results through oral, written, and computer-based communications;
able to demonstrate an appreciation for the humanities and the responsible role of technology in society;
appreciative of the value of diversity and other cultures;
able to work effectively in interdisciplinary teams and demonstrate problem solving, leadership, and organizational skills.

ADVISORY BOARD
The MSE Department maintains an active Advisory Board to assist and provide guidance to the Department in meeting the Program Objectives and Education Outcomes as summarized above. The Board is comprised of representatives from several constituency groups, each having an active interest in the success of the Department. These include potential employers of the undergraduates and graduate students, alumni of the Department, faculty and potential graduate school advisors from peer university institutions, and representatives from national scientific and technology laboratories. The Board's membership was restructured in 2001, and now consists of fourteen members. The new board met in Blacksburg for the first time in April 2002.

ASSESSMENT STRATEGIES
Undergraduate Program: The MSE Department conducts regular and periodic assessment of its success in meeting the educational outcomes outlined above. The assessment program consists of

senior exit survey (MSE assessment committee)
senior exit interview (MSE Department Head)
surveys of recent alumni (of classes graduating 2, 4, and 6 years previously)
ongoing student performance tracking, sophomore through graduation (MSE assessment committee)
overall quality and relevance of program (MSE Advisory Board)
accreditation by the Accreditation Board for Engineering and Technology (ABET) for the undergraduate program
other indicators of program quality

As shown, a variety of assessment techniques are utilized and implemented at a variety of times throughout the undergraduate program and following graduation. The surveys focus on several topics, including the responder's sentiments regarding the Department's success in meeting the desired educational outcomes, a variety of administrative issues, and issues of current interest and relevance to the program, faculty, or students. The Department considers results from alumni surveys to be representative of the opinions of actual and potential employers since alumni are well-positioned to offer meaningful perspective with respect to their level of preparedness relative to that of their industrial peers. Likewise, the alumni surveys also capture the opinions of students who have subsequently attended graduate school following their studies at Virginia Tech. Ongoing student performance in the classroom is monitored in traditional ways by individual faculty and academic advisors, and through a quantitative summary of overall class performance.

Graduate Program: Assessment tools for evaluating the effectiveness of the Department's Graduate Program include

an exit survey of students leaving the department with an advanced degree (Graduate Administrator)
a survey (by mail) of Graduate Program alumni (primarily from the past 15 years)
an annual feedback meeting with current students (MSE Program Administrator and Chair)
perceptions of the MSE Advisory Board
student performance on benchmarking exams (the PhD Qualifying Exam and the Preliminary Exam, and the Final Examinations for each of the three graduate degrees)

ASSESSMENT RESULTS
Undergraduate Program
a. Senior exit survey
Results from the most recent senior exit survey (class of 2003) provides the following responses to questions aligned with the program's desired education outcomes as presented above. The percentages given result from a 100% response rate (15 out of 15 graduating seniors responding):

95% of the students are satisfied that they are proficient in the fundamental use of math, physics, chemistry, and engineering science.
93% indicated satisfaction that they are proficient in the use of the basic concepts of materials science and engineering, covering all classes of materials, including ceramics, composites, electronic/optic/photonic/ magnetic materials, metals, and polymers.
97% felt that they are capable of studying materials on scales ranging from the electronic and atomic through the microscopic and macroscopic.
86% felt that they are capable of developing new materials, improving traditional materials, and developing new methods to produce and manufacture materials.
92% of students were satisfied that they are capable of characterizing a material's properties using modern measurement techniques.
86% were satisfied that they are able to predict and evaluate the performance of real materials as functional elements of engineering systems.
88% indicated that they are able to select, specify, and design materials that are appropriate for broad ranges of application.
100% indicated satisfaction with their ability to communicate effectively.
95% indicated that they have an appreciation for the humanities and the responsible role of technology in society.
97% were appreciative of the value of diversity and other cultures.
97% feel that they able to work effectively in interdisciplinary teams and demonstrate problem solving, leadership, and organizational skills.

In addition, when asked about specific administrative and programmatic functions:

77% were satisfied with the quality of the classroom instruction that they received
93% were satisfied with the quality of the laboratory instruction
97% were satisfied with the competency of the faculty
97% were satisfied with the advising by faculty
50% were satisfied with the quality of the laboratory facilities
100% were satisfied with the quality of the curriculum advising received

The results indicate an overall high level of satisfaction with the quality of the MSE educational program by graduating seniors. However, perception of the quality of the laboratory facilities continues to be of concern to the students and thus to the department. It should be noted that this has been an ongoing focus area and priority for the Department for several years and that considerable progress has been made. In particular, several laboratory areas have been cleaned and renovated, obsolete and broken equipment removed, an effective maintenance plan established, and the purchase and installation of various new pieces of equipment (e.g., scanning electron microscope, tape casting unit).   A relatively low satisfaction rate (77%) with classroom instruction was unexpected and inconsistent with rates reported from recent, previous years. The low value may be in response to university budget cuts affecting the number of sections and offerings of certain courses within the university and department. Future student response in this area will be closely monitored in future surveys.

b.     Senior exit interviews
One-on-one discussions with exiting seniors provides a means to receive candid assessment of a broad range of Departmental activities. Based on comments received during exit interviews, the Department has implemented a variety of improvements to the curriculum and to the quality of life for the MSE students. Examples of changes made in response to exit interviews include: i.) the creation of an undergraduate computing laboratory providing on-campus access to computers for homework and projects, ii.) modifications to the curriculum checksheet regarding the chemistry requirements (removal of analytical and physical chemistry in favor of a new MSE foundation course, Elements of Materials Science); iii.) the consolidation of the thermodynamics and phase equilibria sequence to minimize topic redundancy; and iv.) the expansion of the capstone design experience to include more instruction of traditional engineering design.

c.     Alumni Survey
The results from the most recent graduating class (class of 2001) polled as part of the yearly alumni survey provide the following:

a 63% response rate (15 out of 24 graduates responding)
53% of the respondents are attending, or have attended, graduate school following their program at Virginia Tech
60% of the respondents have been employed in industry following graduation. Of these, 100% were/are employed within the discipline of MSE.

With regards to the desired educational outcomes defined by the faculty:

95% of the students are satisfied that they are proficient in the fundamental use of math, physics, chemistry, and engineering science.
94% indicated satisfaction that they are proficient in the use of the basic concepts of materials science and engineering, covering all classes of materials, including ceramics, composites, electronic/optic/photonic/ magnetic materials, metals, and polymers.
98% felt that they are capable of studying materials on scales ranging from the electronic and atomic through the microscopic and macroscopic.
92% felt that they are capable of developing new materials, improving traditional materials, and developing new methods to produce and manufacture materials.
96% of students were satisfied that they are capable of characterizing a material's properties using modern measurement techniques.
92% were satisfied that they are able to predict and evaluate the performance of real materials as functional elements of engineering systems.
91% indicated that they are able to select, specify, and design materials that are appropriate for broad ranges of application.
97% indicated satisfaction with their ability to communicate effectively.
95% indicated that they have an appreciation for the humanities and the responsible role of technology in society.
90% were appreciative of the value of diversity and other cultures.
100% feel that they able to work effectively in interdisciplinary teams and demonstrate problem solving, leadership, and organizational skills.

With respect to the programmatic criteria:

97% were satisfied with the quality of the classroom instruction that they received
78% were satisfied with the quality of the laboratory instruction
100% were satisfied with the competency of the faculty
84% were satisfied with the advising by faculty
50% were satisfied with the quality of the laboratory facilities
69% were satisfied with the quality of the curriculum advising received.

The results of the alumni survey indicate a high level of satisfaction with the education that they received from MSE Department. The 50% satisfaction rate with the quality of the laboratory facilities shows consistency with the impressions provided by the most recent seniors, above.

d.     Ongoing student performance tracking, sophomore through graduation
This is a recently-initiated assessment tool and, as such, contains insufficient data to provide meaningful assessment results at this time. The tool attempts to associate student course grades to a quantitative success matrix as a means to track student progress toward meeting the educational outcomes described above. It is anticipated that results from this assessment mechanism will be included in the next assessment report.

e.     Overall quality and relevance of program, MSE Advisory Board
The Board continues to provide strong support for, and endorsement of, the undergraduate curriculum and the faculty. In particular, the Board identified the MSE Communications Program (a unique and novel across-the-curriculum program aimed at providing students with instruction and guidance in all communication forms and as a means to stimulate discussion on a variety of professional and societal issues) as notably useful and valuable. The board met with a group of students and was impressed with their level of perceptiveness, quality of character, and value to the Department. The Board identified several areas for improvement relevant to the quality and continued improvement of the undergraduate program, including a need for additional faculty, a need to decrease the average faculty teaching load, a suggestion that the Department utilize the enthusiasm of present students to assist in the recruitment of new students, and the development of a revised space plan for the Program.

f.      External accreditation by ABET
The Accreditation Board for Engineering and Technology (ABET) visited Virginia Tech in October of 2001 to evaluate all degree granting departments in the College of Engineering. ABET is the recognized accreditation-granting entity for all engineering degree programs nationally. To become accredited, a program is required to show compliance with a specific set of criteria as established by ABET and its board of practicing professional engineers.

As a result of the 2001 visit, the MSE Program received a six-year duration of accreditation, the highest level of endorsement from ABET.

g.     Other measurements of program quality and strength
Other indications of program success that are non-recurring include the following, confined to the current population of undergraduates in MSE:

Two Barry M. Goldwater Scholars; these awards were established by the U.S. Congress to honor former Senator Goldwater and highly qualified scholars of science, mathematics, and engineers.
A National Fulbright Scholar.
The 2002-2003 Outstanding Senior in the College of Engineering
A Hoffman Scholar, the highest award given by the Electronics Division of the American Ceramics Society,
An H.H. Harris Scholar, given by the American Foundry Society for students in the metallurgical and metal casting field
A Foundry Education Scholar, given by the Foundry Education Foundation for students interested in the metalcasting industry
A Morgan L. Williams Scholar, awarded by the Washington, D.C. Chapter of ASM
Seventeen College of Engineering Scholarships awarded, representing approximately 30% of the undergraduate constituency of the department
18% of the undergraduate constituency are University Honors students
Approximately 43% of the students graduating in the last 3 classes (2000, 2001, 2002) are attending graduate school at first or second quality-tier institutions

Graduate Program
a. Graduate alumni survey

A random sample of 51 people who graduated with advanced degrees from the MSE Department (or the now defunct MESc PhD Program) between 1990 and 2001 were mailed a survey designed to elicit their impressions of the MSE Graduate Program. We received 17 responses for a 33% response rate. The survey was conducted in February 2002, and its questions sought to probe alumni opinions about administrative aspects of the Graduate Program (the recruitment experience, program policies, and procedures), the graduate curriculum, and opportunities for professional development during their graduate studies. In addition, respondents were asked to rank the quality of several facets of their graduate education experience. Roughly half of the respondents earned Masters degrees from the Department and half earned Ph.D. degrees.

On the whole, alumni impressions of the graduate program administration and curriculum were positive. The few negative comments tended to arise from isolated events and did not share any common threads. Some of the complaints were offset by positive impressions of the same course or requirement. Thus, no serious problems requiring corrective actions were identified by the survey. Nevertheless, the strength of this conclusion must be tempered by the fact that the survey represented a small sample of opinions. Satisfaction rankings (1 to 10 with 10 being the greatest satisfaction) from the survey are:

What is your level of satisfaction with each of the following?

the curriculum (13 Responses, Mean: 7.3, Median 8)
the experimental equipment and facilities (14 Responses, Mean: 6.9, Median 8)
the amount of technical writing (15 Responses, Mean: 6.5, Median 7)
the amount of technical presentations (14 Responses, Mean: 6.8, Median 7)
the amount of mentoring available from your advisor (15 Responses, Mean: 8.7, Median 10)
the amount of mentoring available to you at Virginia Tech (14 Responses, Mean: 7.5, Median 8)
the ability of the program to prepare you for a broad range of careers (14 Responses, Mean: 6.9, Median 7)
MSE Graduate Program administration (11 Responses, Mean: 7.9, Median 8)
the Virginia Tech Graduate School (15 Responses, Mean: 7.2, Median 8)
the overall graduate experience in the MSE Department (13 Responses, Mean: 7.9, Median 8)

There was somewhat less satisfaction with the amount of technical writing and technical presentations in the graduate program, and there was comparatively less satisfaction in the program's ability to prepare graduates for a broad range of careers. Mentoring from graduate student advisors is one area the alumni thought the MSE Department is doing relatively well. It is not entirely clear what the absolute values of these satisfaction rankings mean because the number of responders to the survey questions is small, and the range of years over which the alumni graduated ensured they were not all ranking the same graduate program (the faculty, program administration, and policies have evolved over the years).

b. Annual Graduate Student feedback meeting
The most recent meeting between current graduate students and the Graduate Chair and Graduate Administrator was held on 14 January 2003 to register complaints about how the MSE Graduate Program is working and to solicit suggestions about what might be changed to improve it. A little more than a quarter of the graduate student body attended. The issues discussed are listed below followed by itemized summaries of the student comments.

Seminar:
* Inconsistently run from year to year.
* Make it two separate seminars one for outside speakers, one for students.
* Increase faculty involvement.
* Dislike required attendance/doesn't necessarily mean students will attend.
* 1 required presentation for Masters students, 2 presentations for PhD students is acceptable
* Once credits are fulfilled, no need to sign up for the course.
* Dislike the write-ups at the end of the semester (a particular instructor's requirement) not needed.
* Schedule talks in advance and make sure the students have enough time to prepare as well as have enough research to present.
* Have a class/presentation for presentation skills. Consider videotaping graduate presentations and offering instruction

Qualifier:
* Schedule it in June/August before school starts so people taking it a second time have at least a semester to fulfill remedial requirements (e.g. additional course work) before taking it a second time.
* Make sure students taking it realize they need to know the fundamentals; small seminar (class on proposal writing) give out copies of good examples from previous exams before the qualifier is to be taken.
* Have the committees reflect on research related to what each student is doing (for example, don't put all polymers faculty on the committee of a metallurgy student).
* Format seems fine.

Courses:
* Perhaps create an Introduction to Grad School class (1 credit), Art of the Seminar class (possibly a couple of the seminars) that includes topics like proposal writing, presentations, and software.
* Put 'Full' in the timetable so sessions of courses don't get dropped and student tuitions don't get messed up.
* The number of courses per semester is not satisfactory, and the courses don't match student interests
* Characterization class (distance learning or online classes not good). Could a "hands on" class be developed?
* Can graduate classes be made available in the summer?

Labs:
* There is concern about equipment being moved too frequently (quit moving items from one place to another). This makes it difficult to keep some equipment calibrated and working properly. Also, make sure equipment remains working.
* Have a list of labs and equipment available with contact information. This would be helpful for departmental labs as well as campus wide. A tour of facilities for new students to introduce them to what is available would be nice.
* Improve equipment availability/cooperation to allow usage not only to professor's own students who are "in charge" of a piece of equipment. Several cases were cited of departmental equipment becoming unavailable once it was moved to a professor's lab.
* Assign a technician to help maintain equipment and train the students on it.

Students would like:
* Personal mailboxes
* Signed ISR and Bookstore forms available in Collegiate Square (MSE Annex site)
* Streamline purchasing procedures
* Poster printer, (compare costs with using the one in Biology)
* Have computer technician available for small fixes.
* Use of "Undergrad" Computer Lab (1st year students and students who don't have an advisor should have access to this lab). Ask Dave Berry to add SS#'s to the door lock.
* Students would like to meet with Susette once a year to double check that they are staying on track with what is needed to graduate. Make sure everyone meets all requirements and turns in forms that are needed.

General:
* More faculty members are needed; at the moment, there are not enough to assemble an advisory committee for some students.
* Not enough funding for students
* Administrative procedures within the Dept's Grad Program are fine

c.        Perceptions of the MSE Advisory Board
A subcommittee of the Advisory Board met in April 2002 to evaluate the MSE Graduate Program. This Graduate Review Board noted the MSE Graduate Program is working effectively. They also noted the graduate students display a strong interest in the health of the program and a high degree of satisfaction with their formal course work and research training. The Board found the large ratio of domestic to international students (compared with other MSE programs across the country) particularly noteworthy. Issues that raised some concern among the reviewers included the loss of key faculty and the small number of MSE faculty in recent years, both of which make it difficult to maintain a healthy selection of graduate courses and a stable student population. The Review Board noted efforts should be made to recruit more graduate students from other departments and other schools to increase the breadth of graduate student backgrounds in the Graduate Program.

d.       Student performance on benchmark exams
During the last few years, this assessment has been based upon the results of the PhD qualifying exam. Faculty who have created and graded this examination have expressed general dissatisfaction with students' mastery of fundamentals, their familiarity with general materials topics, and their ability to develop research proposals. Although these issues are always topics for debate, the faculty concluded part of the problem probably derives from the faculty and students having different ideas about what is expected of the students on the Qualifying Exam. Suggestions for improving the curriculum, program policies, and program procedures are discussed among the faculty following each Qualifying exam, and potential improvements tested during the next exam cycle (the Qualifying Exam is offered annually.

RECENT RESPONSES TO ASSESSMENT RESULTS
Undergraduate
The assessment results given above indicate an extremely high level of satisfaction with the MSE undergraduate program among all identified constituency groups. Thus, the faculty are presently in the favorable position of having to make relatively small changes in response to occasional issues which may arise. A series of curriculum changes were recently implemented in response to the results listed above and obtained from recent surveys. The changes include:

creation of new course, MSE 2054, Elements of Materials Science, as a foundation to higher level courses and in response to student difficulties associated with level of preparedness and scheduling of the traditional Physical Chemistry course;
redefining the second-year mathematics sequence in response to elimination of the previous course by the Mathematics Department;
removal of MSE/ECE 2214, Semiconductor Processing Laboratory, in response to state-mandated budget cuts;
removal of the course on Fortran in favor of a new Program Elective option that provides students with a choice of courses which emphasize a variety of computational or numerical processing skills;
removal of a lecture in Manufacturing Processes from the required curriculum in response to it being relevant only to a relatively small minority of the undergraduate constituency; and
the insertion of an additional technical elective in response to student and alumni request for additional choice and versatility within the curriculum

Graduate
Many of the problems identified by the assessment tools above require little more than administrative adjustments. Most of these minor problems were corrected as they came to light. Some of the more important areas of concern, such as availability of graduate courses, coverage of additional skills in the curriculum, and the availability of assistantships, are linked to the MSE Department's size and resources. These are more difficult to change, but efforts are underway to affect improvements. A few of the recent Graduate Program changes include:

The number of undergraduate sections was reduced to lessen teaching loads and enable faculty to teach a few more graduate courses.
The MSE graduate curriculum underwent a complete overhaul during the Spring 2003 semester. Revisions included the addition of new courses (some of which are distance courses delivered to Blacksburg from other locations), the elimination of infrequently offered courses, and the addition of flexible syllabus courses for advanced graduate courses. The core curriculum was also simplified to a set of 5 courses.
Graduate administrative duties were assigned to one staff member with expanded responsibilities for student recruiting and tracking.
Instituted a Departmental policy of requiring GRE scores from Virginia Tech applicants to the MSE Graduate Program to help diversify the graduate student body.
Established a meeting between Graduate Chair and students preparing for the PhD Qualifying Exam to convey procedural information and expectations.