By Vlad Krotov, PhD
Client
A young, large private university in the Middle East undergoing initial accreditation with WASC and AACSB.
Client Requirements
Designing a simple and robust Assurance of Learning system for a Doctor of Business Administration (DBA) executive doctoral program offered by a large private university in the Middle East. The system had to meet the following requirements:
a) Compliant with AACSB Standards. The College of Business where the program was offered was going through initial accreditation with AACSB; therefore, the system had to meet all the AACSB requirements in relation to AoL
b) Simple. The system had to be simple enough so that the DBA faculty could quickly understand and contribute to the continuous quality improvement program based on this AoL system. This was important to accommodate changes in the faculty roster teaching in the DBA program as well as general changes in the policies and the curriculum of this newly established DBA program
c) Reliable. The system had to produce reliable, useful results. It was important for the system to have a “pre-test” and then a “post-test” to produce meaningful results in relation to program learning goals. Also, the measurement tools had to incorporate both quantitative and qualitative results for further improvements.
Solution
As the first step, the curriculum of the DBA program was aligned along the five Program Learning Outcomes (PLOs). The results of the curriculum alignment process are provided in the Course Alignment (CAM) matrix below (see Figure 1):
The extent to which Doctoral of Business Administration (DBA) students have mastered the learning outcomes of the program is assessed at 3 strategic points: METH 1 “Introduction to Business Research” course (Assessment Point 0), RSCH 1 “Research Proposal” (Assessment Point 1), and RSCH 2 “Dissertation” (Assessment Point 2) (see Figure 2 below).
At each of the assessment points, the Research Evaluation Rubric is used to assess student performance. The rubric relies on two other rubrics developed by the College: General Analytical Writing Assessment Rubric and Oral Presentation Rubric. In METH1 course, the basis for assessment is student “mock” dissertation proposal – an exercise where students, based on their limited knowledge of their research domain and high-level understanding of research methods, describe and present a rough plan for their possible future dissertation research. The assessment is done by the instructor teaching this course. This assessment point is used to establish a baseline for student knowledge and skills in relation to the program learning outcomes shortly after they join the DBA program. In RSCH the basis for assessment is the actual research proposal document and presentation. Finally, in RSCH 2 the basis for assessment is the final dissertation document and the final dissertation defense. In RSCH 1 and RSCH 2 assessment is done by the dissertation committee chair. In both cases, assessment results are submitted to the Program Director for analysis. Subsequent changes in the curriculum are subjected to the standard curriculum revision process implemented by the College and presided over by the College Curriculum Committee.
Results
With this simple, robust AoL System, the college was able to “close the loop” in relation to the DBA program in just one year (see Figure 3) below:
The results of the newly designed AoL system indicate a noticeable growth in the level at which doctoral students master the PLOs across semesters (see Figure 4 below). These improvements are largely a result of the recommendations for curriculum and policy changes submitted by the DBA faculty participating in the AoL process using the rubrics used in assessment.
The College believes that this assessment method allows for a closer monitoring of individual students with respect to their achievement of the learning goals of the program. The AoL system was received positively by AACSB Review Team.