Guidelines on How to Prepare an Institutional Effectiveness Report


© 1999 by Dr. Thomas W. MacFarland -- All Rights Reserved




 
General format for an Institutional Effectiveness (IE) report


1  Goal and context

   (A) What is the primary goal for participants in this major
       and is this goal articulated in the catalog and other
       public documents?
     
   (B) What is the target population for participants in this 
       major, specifically in terms of age, employment and/or
       other prior expectations, proximity to campus, etc?

   (C) Are GPA, SAT or GRE scores, or portfolio submissions used 
       in the admission review process?  Are there other admission 
       requirements for participants in this major and do these  
       processes allow for consideration of applicants who do not 
       meet these requirements?  

   (D) Is it possible to identify tracking information on 
       inquiries and applicants?  

2.  Expected results of core topics to address for each major 

    Graduation Rates and Persistence

    (A) What are the overall graduation rates for students in 
        this major?

        Additionally, what are the graduation rates for breakout
        subpopulations in this major:

        (1) Graduation rates by race/ethnicity.

        (2) Graduation rates by gender.

        (3) For majors that offer instruction through distance
            education modalities, it is also useful to examine:
            graduation rates by place of class attendance (i.e.,
            campus-based students v. their distance education 
            counterparts), graduation rates by teaching modality
            (i.e., students who generally attend courses offered 
            by lecture v. students who generally attend courses
            offered by compressed video v. students who generally
            attend courses offered through Internet-based 
            utilities).

    (B) Related to graduation rates, what are the rates of 
        persistence for students in this major, overall and by
        breakout groups? If you use a cohort methodology to track 
        students from program entry to graduation, it is 
        important to note students who are still persistent but 
        not yet graduated.

    Learning Outcomes
 
    (A) Does the major have a declared terminal knowledge base, 
        set of competencies, and skills for its graduates and 
        did students gain these constructs by graduation?

    (B) What evidence will you use to demonstrate these outcomes?

    (C) If you use a survey or some other similar instrument, 
        what is the composition of the sample and is the sample 
        representative of the overall population?  Sample size (N)
        is important, but it is also important to consider sample
        representation.

    (D) Is it possible to identify tracking information on other
        indicators of student success that relate to demonstrated
        learning outcomes: results of standardized tests 
        administered by professional associations and other 
        external constituencies, results of government and other 
        licensing examinations, employer evaluations, alumni 
        evaluations, evaluations by students engaged in practica 
        and/or internships at external institutions.

    Continuation After Graduation

    (A) What happens to students in the major after they graduate
        and do administrators have an adequate tracking system to
        maintain occasional communication with graduates?

    (B) What number and percent of total of all graduates gain
        "in-field" employment within a reasonable period of time
        after graduation?

    (C) What number and percent of total of all graduates go on
        to advanced study within a reasonable period of time
        after graduation?

    (D) Is there evidence that the major is structured so that 
        students have the professional and other competencies 
        needed for career success and advancement?

    Note:  For each expected result, there should be a formal
           statement of assessment method and the results of
           this research-based assessment activity.  The 
           assessment method(s) and subsequent result(s) should 
           follow standard practice in terms of social research
           in terms of sample size and representation.

           #1 Expected Result 

              #1 Assessment Method 
              #1 Data Resource 
              #1 Result of Assessment 

           #2 Expected Result 

              #2 Assessment Method 
              #2 Data Resource 
              #2 Result of Assessment 

           #3 Expected Result 

              #3 Assessment Method 
              #3 Data Resource 
              #3 Result of Assessment 

3.  How assessment results were used to structure changes that
    provide improvements to the major
       
    (A) What is the outcome of the findings gained from the above
        section?

        Did you determine that the overall graduation rate is
        acceptable, but that there are breakout groups (such
        as distance education students, minority students, 
        etc.) who are not in parity with summative results?  If 
        so, how can you work with other administrative departments,
        faculty, and students to restructure the major and remedy 
        this concern?

    (B) Did the assessment process identify strengths and 
        weaknesses in how the major is structured?  If so, what
        can you do to build on the strengths and minimize the
        weaknesses?

    (C) Did you determine that external factors, such as 
        professional accrediting associations, have an increasing
        impact on students as they matriculate through the major?
        Discuss the impact of external influences and how the
        curriculum is changed, or needs to be changed, to reflect
        these external groups.

    (D) Is it possible that you found that changes are not needed?
        If so, be sure to highlight assessment findings that
        support this position.

Let me know how I can assist you in the areas of institutional research, statistics, and computer science education.


Please send comments or suggestions to Dr. Thomas W. MacFarland

There have been [an error occurred while processing this directive] visitors to this page since February 1, 1999.