Institutional effectiveness is the term used at Eastern Mennonite University to refer to the collection of processes and policies that together assure that the university is successful in achieving its mission and vision and meeting the needs of its stakeholders (students, employees, as well as the churches and communities it serves). Institutional effectiveness might just as easily be viewed as a widely-held commitment to continuous improvement on the part of all areas of the university.
Planning at EMU takes place at a variety of levels within the organization. All planning simultaneously informs and is shaped by the university’s strategic plan. A formal strategic planning process is undertaken every three years, and concludes with an updated three-year plan for the institution. The three-year plan is adjusted as-needed between major updates. Each year departments and programs within the university develop or revise their own multi-year plans, all of which are linked to the priorities of the university’s three-year plan.
Assessment efforts at EMU are built around student learning outcomes and operational outcomes that academic programs and departments articulate. Each year, faculty and staff review data according to their own assessment cycles. The findings from assessment work inform plans for growth and improvement. Additionally, program review cycles provide an opportunity for more in-depth review of program effectiveness, viability and future direction.
In order to ensure the EMU’s commitment to principles of continuous improvement, based on a systematic and documented process of assessing institutional performance with respect to mission in all aspects of the institution, the university as a whole and its various programs and services engage in the following processes:
These processes inform decision-making at all levels and provide a sound basis for budgetary decisions and resource allocations.
Lest they devolve into bureaucratic exercises, EMU’s institutional effectiveness processes should be participative, flexible and responsive, relevant, and closely linked to university planning. To this end, we strive to maintain an institutional effectiveness framework that adheres to these principles:
PACE is the process which facilitates the implementation of the university’s institutional effectiveness policy. The PACE process relies on a web-based system, Strategic Planning Online (SPOL) that tracks planning objectives, division- and institution-level priorities, student learning outcomes, operational outcomes, and much of the data associated with assessment of these elements.
EMU uses a three-year time horizon for its strategic plan, making minor adjustments as needed for the plan’s second and third years based on campus input and environmental considerations.
The three-year plan is developed via the focused work of a representative six- to eight-member task force over a six- to nine-month participatory process. The task force gathers input from key campus constituencies via surveys, focus groups and interviews. This input is combined with an environmental scan in order to develop university priorities for the next three years.
The strategic planning task force is comprised as follows:
Though the specifics of the planning process will be adapted as needed for each cycle, the basic planning process may include all or some portion of the following steps:
At the conclusion of the process, the task force forwards its recommended strategic plan to the president’s cabinet for approval. Finally, the plan is approved by the board of trustees.
Upon approval, each of the plan’s priorities is assigned to a member of the president’s executive team who will serve as its point person. As point person for a priority, the executive team member will coordinate efforts across the university in support of the priority (in some cases, in conjunction with another member of the team). The point person is responsible for developing a measurement and evaluation plan for the priority (metrics, data collection plan, etc.) in collaboration with the institutional research office.
Each year the point person is responsible for preparing progress reports to the president and board of trustees for the priority. These progress reports often are incorporated into regular (three times per year) reporting that is incorporated into the docket of materials for each board of trustees meeting. At a minimum, annual reports on each of the strategic plan priorities are completed over the summer, distributed to campus in the early part of the fall semester, inform the president’s annual report, and are included for review by the board of trustees at its November (annual general) meeting.
Three times each year, usually at the conclusion of the fall, spring and summer terms, the president’s cabinet also formally meets to evaluate progress on the priorities and to consider whether adjustments to the plan are necessary based on internal or environmental considerations. Any adjustments to plan are shared with the campus community at the opening of the fall semester.
In the third year of the plan, the president’s cabinet convenes a new strategic planning task force to begin preparing the next three-year plan.
Each year, individual academic programs and administrative departments engage in a localized planning process that considers two academic/fiscal years simultaneously and consists of the following components:
Program- and department-level plans are, whenever possible, directly linked to university strategic plan priorities in the planning system (SPOL). Point persons for strategic priorities are responsible to review and coordinate efforts across the university linked to their priority/ies.
During the academic year, supervisors of academic program directors and administrative unit heads check-in on a regular basis regarding the status of planning objectives of the various academic and administrative units. These updates can be recorded in SPOL to facilitate ease of end-of-year and strategic planning reporting.
Each summer, deans and VPs review progress reports from their academic programs and administrative departments and provide feedback to inform future direction. VP review of program/department progress reports also inform the PACE Executive Summaries (further described below) they prepare for the president and for the board of trustees annual general meeting each November.
See Appendix 1 for a list of EMU's planning units as identified for PACE planning purposes.
Each year in December the chief financial officer kicks off the formal process for development of the university operating budget and capital expenditure priorities for the next fiscal year. Prior to this formal launch, academic programs and administrative departments can begin preparation of budget requests and capital requests. These requests will likely have emerged through the program’s/department’s local planning processes, and can be submitted via the planning system (SPOL).
The basic sequence of the annual budget process is outlined in the table below.
Month | Process Steps/Milestones |
Sep-Nov | Programs and departments develop budget and capital requests independently, based on their local planning; these requests are submitted via the SPOL planning module |
Dec/Jan | Core baseline budget parameters:
|
Jan/Feb | Budget worksheets to VPs and Deans:
Capital budget meetings with primary stakeholders
|
February | Budgets returned to VP for Finance
|
March | Presentation of preliminary budget outlook to Board of Trustees, |
April-May | Revisions to preliminary budget; enrollment adjustments |
June | Final Budget worksheets to VPs and Deans:
Board of Trustees, Institutional Resources & Sustainability Committee:
|
July | Final Operating and Capital budget based on actual enrollment
Preliminary “final” budget loaded into ERP for reporting and tracking |
November | Final approval by Board of Trustees, Institutional Resources & Sustainability Committee
|
Outcomes Assessment is the means by which the university’s programs and departments hold themselves accountable to their own aspirations. The SACSCOC principles of accreditation offer helpful guidance with regard to the scope and nature of the university’s assessment efforts.
Academic programs maintain student learning objectives that describe the knowledge, skills, and attitudes/responsibilities/dispositions their graduates will demonstrate upon program completion. Academic and student services identify outcomes of their work that support student success. Section 8 of the 2018 SACSCOC principles articulates the following requirements:
8.2 The institution identifies expected outcomes, assesses the extent to which it achieves these outcomes, and provides evidence of seeking improvement based on analysis of the results in the areas below:
Particularly important in the SACSCOC language above is the requirement that programs “[provide] evidence of seeking improvement based on analysis of … results.” The PACE process and the tools of SPOL are designed to scaffold this work and support departments in maintaining relevant data and documenting their efforts.
Administrative departments must likewise articulate the expected outcomes of administrative support services. Section 7 of the 2018 SACSCOC principles articulates the following requirements:
7.3 The institution identifies expected outcomes of its administrative support services and demonstrates the extent to which the outcomes are achieved.
It is worth noting that the SACSCOC principles do not require that administrative departments provide evidence of seeking improvement. Administrative support services may find success in articulating standards of performance/service (outcomes) and consistently achieving them. Likewise, administrative departments may demonstrate compliance with SACSCOC expectations for outcomes assessment via well-designed planning objectives with clear outcomes (via PACE planning) and/or performance-focused outcomes (via PACE assessment).
See Appendix 1 for lists of EMU’s academic programs and services and its administrative departments as identified for PACE assessment purposes.
The outcomes assessment work of academic programs and administrative departments will generally take place late in the academic/fiscal year. By design, this work should be completed prior to taking up planning work for the upcoming and future years and can inform budget requests and planning objectives for the program or department. A basic timeline:
It is important to note that academic programs and services and administrative departments need not collect and analyze data for each of its outcomes every year. A program/department may develop a multi-year (ideally 2-4 year) cycle over which it assesses each of its outcomes. Such a cycle allows for data collection over multiple years, and allows programs/departments to provide more focused attention on a subset of their outcomes in any given year. Beginning with the rollout of the updated academic program review cycle in 2020-21, academic programs are expected to have collected and analyzed data for each student learning outcome at least twice between each program review. See below for further details on program review.
Further, in order to ensure the sustainability and validity of assessment, small academic programs may opt to extend assessment data collection over several years in order to achieve adequate sample sizes to support analysis. Programs that opt for this approach should collect data for all outcomes each year, but may still focus on analysis of results for a subset of student learning outcomes each year.
At the conclusion of each academic year (usually over the summer and the early fall), deans and vice presidents prepare a PACE Executive Summary for the school or division. The executive summary is based on the dean’s or vice president’s review of their program/departmental activities (i.e., planning and assessment results). The 2-5 page summary addresses the following points:
In addition to the PACE Executive Summaries prepared by VPs and deans, the VP for Institutional Effectiveness prepares annually a University-Wide PACE Report. This report highlights:
In addition to the PACE process, all academic programs that do not undergo an external accreditation review are reviewed on a six-year cycle (adapted as-needed to specific program considerations). See Appendix 2 for more details on the review process. See the Program Review Cycle for details on when each program is reviewed. This comprehensive review is conducted by a faculty task force, overseen by the provost council and includes consideration of:
The review process consists of the following steps:
Possible outcomes of the review are:
For the purposes of institutional effectiveness work, EMU organizes its academic programs, academic and student services, and administrative support services into planning units and academic programs. The configuration of these programs and units largely aligns with the university's Organizational Chart. However, in some cases--due to uniqueness in focus or scope of impact--organizational units are excluded from PACE processes. For example, PACE processes exclude academic and public service centers that (a) do not have associated academic programs, and (b) have a small scope of influence within the university as evaluated by considerations such as direct impact on student-facing educational activities, external focus, fiscal footprint, and/or governance.
The tables below outline the organizational units (i.e., academic programs, academic and student services, and administrative services) that are incorporated into the PACE processes.
Undergraduate Planning Units and Academic Programs | ||
Biology & Chemistry AS Health Science BS Biochemistry BS Biology BS Chemistry BS Clinical Lab Science BS Environmental Science Nursing BS Nursing BS RN to BS Nursing Mathematical Sciences BS Computer Science BS Engineering BS Mathematics Psychology BS Psychology Visual and Communication Arts BA Art BA Digital Media and Communication BA Photography | Bible, Religion & Theology | Applied Social Sciences |
Graduate Planning Units and Academic Programs | ||
Doctor of Nursing Practice MS Biomedicine MS Nursing | Eastern Mennonite Seminary MA Church Leadership MA Religion Master of Divinity | Center for Justice & Peacebuilding MA Conflict Transformation MA Restorative Justice MA Transformational Leadership MA Counseling MA Education Organizational & Leadership Studies MA Org. Leadership Master of Bus. Admin. |
Academic and Student Support Units | Administrative Support Units |
---|---|
Academic Success Center | Advancement Admissions - Undergraduate Auxiliary Services Business Office EMU at Lancaster Facilities Management Financial Assistance Graduate & Professional Recruitment Human Resources Information Systems Institutional Research Marketing and Communications Registrar's Office Safety and Security Title IX Washington Community Scholars' Center |
Basic Structure:
Evaluation Team:
The evaluation team will typically consist of two external peer reviewers within the discipline(s) of the program cluster to be reviewed, and one internal peer review. Ideally, one external reviewer will be from an institution that is comparable to EMU and a second reviewer will be from an “aspirational” institution. The internal peer review will typically be a tenured faculty member from a different program cluster; the internal peer reviewer will be offered some measurable reduction in workload during the academic year when the review is conducted (such as release from a committee or a dean’s hour). The program cluster should contact nominees for the evaluation team prior to submitting names to determine interest and availability. The program cluster should submit the names of at least three potential external reviewers, along with vitae, to the dean, with additional comments or a ranking if desired.
The program should also identify an upper-level student currently enrolled in the program to support the work of the evaluation team. The student will assist in gathering input from current students in the program (likely by assisting with focus groups) and will meet with the evaluation team at least once to synthesize student input into the final review report.
Outline of Academic Program Self-study ReportA. Academic Program
B. Faculty
C. Students
D. Resources
E. Summary and QuestionsProvide any concluding statements that may be helpful for the review team, along with any specific questions that you would like the team to consider in their evaluation. |
Academic Program Review Process revised and approved by Provost's Council June 29, 2021