Program Review Option (per state partnership agreement)
CAEP Program Review with National Recognition (SPA)
1. Based on the analysis of the disaggregated data, how have the results of the specialty licensure area or SPA evidence been used to inform decision making and improve instruction and candidate learning outcomes?
In general, the analysis disaggregated data from SPA evidence has been used to change course descriptions and foci, address issues with assessments, revise assessment rubrics to more clearly convey the criteria that constitutes corresponding scores, and to evaluate the sequencing of courses within a program. For example, the Math program improved and expanded use of rubrics to score lesson plans in the Math Methods courses (elementary and secondary) and made content changes to the Math 200 Geometry course by requiring lesson plans to include math specific technology as well as being scored using a SPA rubric aligned to NCTM Standards. Another example of course revision was the addition of restricted electives to the US History and the Behavioral Sciences blocks, including updated courses in anthropology, women’s history, African-American history, and 20th Century history. Candidates’ content GPAs and Praxis sub-scores are being followed carefully to determine any effect of these changes.
The Physical Education program shared its e-portfolio layout with the PEUC early in the Fall 2016 semester and invited PEUC members to attend the training sessions they provided for their students and faculty members. This sharing led to a discussion within the PEUC about the use of portfolios across program areas. All programs require a portfolio of some sort, and portfolio submission and review is part of the Juncture process. However, it became apparent that the content requirements, format, delivery method, and assessment criteria are vastly different across program areas. The PEUC determined a more purposeful and consistent development, submission, assessment and use of portfolios should be developed and an ad hoc committee was formed to more deeply review portfolios across programs and bring suggestions for unit-wide implementation back to the group for discussion and ultimately a motion.
The revision of rubrics to more clearly define the expectations and criteria for assessments across specialty licensure areas sparked discussion in the PEUC regarding assessment quality and validity. One outcome across programs was a deeper examination of the unit-wide ST-11, with a norming a feedback session for all student teacher supervisors at the end of the Fall 2016 semester and a follow up session prior to the Spring 2017 semester.
Across programs, to systems are used for data collection and report pulling, TK20 and FileMaker Pro. Discussion among members of the PEUC, many of whom are specialty area coordinators, about the issues and concerns faculty have around each of these systems has led to a more concerted review and understanding of the data that is needed for individual program review as well as unit-wide data analysis needs. Many issues with the way these systems are set up, particularly the way data reports are generated and the ensuing need to “reconstruct” the data presentation to facilitate meaningful and efficient data analysis, has led to the collection of ideas and suggestions for system improvement to facilitate the needs of all program areas moving forward. The outcome of these discussions has been the scheduling of a consultation with TK20 for April 2017 to revise and revamp the way data is organized and reported in the future.
2. Based on the analysis of the specialty licensure area data, how have individual licensure areas used data for change?
Each program area has made changes based on the data analysis completed for their individual program areas. Some of these changes are highlighted below:
Has worked diligently to create a clear pathway to success for all areas of Juncture 1, 2, and 3. They created a Sakai link for their majors with multiple resources, require a well-organized e-portfolio, hosted training workshops, and meets regularly to better the program. Additionally, some of the changes and improvements the PE program has made based on data analysis include:
Working to ensure that instructors to have a degree, be it bachelors, masters, or terminal, with a strong emphasis in physical education pedagogy
Specialization coordinator is establishing Transfer Articulation Agreements to increase size and diversity within the program
Working to minimize reliance on adjunct professors teaching courses where content is assessed on the PRAXIS II and instead have full-time physical education faculty teach in these content areas
Establishing peer-tutoring positions to assist students in classes where data has shown that performance is in-need of improvement. Faculty complete an additional Pro 05 for any TCs where there were concerns about their pedagogical and professional knowledge, skills, and dispositions. Physical Education faculty meet regularly to discuss any concerns faculty have with a TCs’ disposition or any apparent existing trends within the TCs knowledge, skills, and dispositions as a whole. Faculty have invested significant time in developing clinical teaching and volunteer experiences in the community where TCs interact further and in a meaningful manner with peers, professors, k-12 children, and parents. This is believed to be a major enhancement of TCs’ “interpersonal skills”, “engagement in critical discourse”, “appreciate diversity and dignity of individuals”, and “willingness to commit to professional development”.
The PETE faculty organized annual tennis instruction workshops, archery certification workshops, adaptive physical education workshops, fundraising events to sponsor purchasing bicycles for socio-economically-challenged children in the community, organized fundraising for Relay for Life, initiated and are maintaining the Phi Epsilon Kappa physical activity fraternity, provided sponsorship for excelling TCs to attend state and national physical education conferences, have TCs present at the state-level physical education conference, and several other clinical service experiences that promote physical activity.
Shifting PHED 410 from the spring semester of sophomore year to the fall semester of junior year. This change was implemented in the fall of 2016. The rationale is that TCs will most benefit by completing their Building a Unit of Instruction (Key Assessment 5) in PHED 410, while also gaining competence in Standard 5: Impact on Student Learning. The PETE faculty are anticipating the TCs to perform strongly in the TWS due to these changes.
Recently, a US History option (HIST 203) was added to the program which approaches the study of history from a global context. Students may opt to take this course in place of HIST 202; the Praxis II sub-scores for US and World History of candidates who do so will be watched carefully to determine any effect on content knowledge.
In 2010-2011, a 120 credit-hour limit was mandated for all university programs. As part of this change, the geography requirement for social studies education students was changed from 3 courses (9 credit hours) to 2 courses (6 credit hours). At this time, the geography sub scores for Praxis II are being closely watched to determine any effect this change is having on candidates’ content knowledge.
(5-9) Removed Math 154
(5-9) Added Math 155
(5-9) Content changes in Math 102: Math for Elementary Teachers – new and expanded history portfolio required that must be scored using SPA rubric aligned to NCTM Standards
(All Programs) Statistics and Calculus course: Improved and expanded use of math specific technology (SPSS, R, and Geometers Sketchpad)
(All programs) ST-11-Addendum used to evaluate student teaching specifically aligned to NCTM standards.
(Math 5-12) Math 414 The History of mathematics – includes concrete models and portfolio requirements that are aligned to NCTM standards.
Curriculum Analysis Form – this analysis is a new requirement from NSTA (National Science Teacher’s Association) to be submitted with the SPA. This curricular analysis is the only true “tool” that has been used to assess the student teacher curriculum. Prior to the analysis exercise, the program was formed on (a) general education/core curricular requirements with respect to science and (b) discussions and curricular retreats between science faculty at Shepherd and veteran teachers with secondary science certifications. The analysis clearly defined areas of content and categorized them in a coherent way so that we could see that all aspects of the curriculum at Shepherd were covered on the analysis forms. Previously, there had been some discussion & observations (as well as student advising) concerning Praxis content, but no alignment has ever been with Shepherd’s curriculum. While we await feedback concerning the curriculum analysis, some discussion and observations has already occurred concerning possible sets of restricted electives and/or course substitutions that could be made that may help interested students to transfer into the science education major without having to add additional coursework and increase time to graduation. Recent changes to the science majors (especially Biology) has been made, partially to address transferability of coursework as well as preparedness for both upper level coursework and careers/graduate school. These changes have also made it less coursework to obtain a dual B. A. degree in secondary science and a B. S. degree in either Biology or Chemistry. Such students should be able to take the same curricula and have the same experiences as science majors, rather than an “equivalent” experience. We have some students already taking advantage of this and it may be a way to recruit and retain science majors into science education.
The curriculum analysis form also informed department heads and instructors who teach secondary science majors of the curricular elements that are considered most important for secondary science education and allowed them to more easily visualize and integrate secondary science education as a possible career goal or graduate school study area for the undergraduate science majors. Prior to the analysis form, specific science curricular content needs, goals and objectives for science education was less obvious – especially since at Shepherd, the students earn a B. A. degree from another department (Education) in another school (Education and Professional Studies) as opposed to a science department within the School of Natural Sciences and Math.
Advising for Elective content – advising for elective courses became more apparent with the curricular analysis form, this is especially relevant for the general science education program, but also impacted the Chemistry education program since both have electives.
Assessment revision – the original submission of this SPA was the first SPA where science specific assessments (with rubrics) for the Unit Plan, Teacher Work Sample, Student Teacher Evaluation science addendum, and Safety Module were submitted with data. These assessment rubrics were developed following the previous SPA submissions prior to 2010. The current SPA submission did not include the Science Research assessment (which has been terminated since grades & GPAs are used), yet the Science portfolio has been retained (even though it is not required by the SPA or NCATE) in an effort to ensure that students have an understanding of science research and basic principles and practices in science and to give the student and advisor a ‘tool’ for developing skills in lesson and unit plan writing. As soon as NSTA 2012 standards were released, Shepherd developed a Professional Development assessment and rubric and began collecting data immediately such that the data could be analyzed and presented with the initial SPA submission (in 2014). From our first round of feedback, we have revised the professional development assessment to be certain to include both teaching/pedagogical and science research/process of science professional development activities.
3. For Program Review with Feedback only: How does the specialty licensure area data align with and provide evidence for meeting the state-selected standards? N/A
4. For National Recognition only: How are SPA reports that are not Nationally Recognized being addressed?
Programs that have recently (February 2017) come back with conditions, those Specialty Area Coordinators are working to revise and update their reports to address the concerns identified. Those SPA reports will be resubmitted by the appropriate deadline. One SPA, Elementary Education, came back without recognition, meaning the current cycle of recognition will expire in February 2018. A new SPA, focusing on the concerns presented via the National Feedback Reports, will be submitted in September 2017. The feedback report should be received by February 2018. While we are aiming for National Recognition, even getting recognized with conditions will help ensure we retain national recognition at this point.