February – July 2012
Notified of the award after the spring 2012 term had started, time for professional development during the project’s first semester focused on assessing student, instructor, and math mentor perceptions regarding what supported learner success in passing Basic and Intermediate Algebra. From multiple focus groups and end of term surveys, their initial input established the professional development process using Participatory Action Research methods for the two year project (Flood, 2010; Greenwood, Whyte, & Harkavy, 1993; Hoedebeck, 2011; Kemmis, McTaggart, & Nixon, 2014; McIntyre, 2008; McTaggart, 1991; Reason & Bradbury, 2001; Somekh, 2005; Stringer, 2013).
Summer 2012 was spent analyzing early quantitative and qualitative data to prepare the professional development training for August prior to beginning the second semester of the project. From this initial data analysis, we recognized the need for consistent procedures to collect valid and reliable data to evaluate model impact on student success. As importantly, reflecting on data analysis with the instructors was established as a critical component of the professional development process. In addition, initial perception data suggested confusion and inconsistencies across students, instructors, and math mentors regarding use of evidence-based practices to improve student success. As a result, we spent our first summer preparing for the professional development retreat planned for August 20, 2012.
August – December 2012
To launch the professional development process, the research and development team hosted an instructor retreat for a day of training in scientifically-based research on learning mathematics and its relationship to the multi-tiered system of supports (MTSS) model for adult learners prior to beginning the fall 2012 term (Adelman, 2004; Attewell, 2006; Batsche et al., 2005; Batsche et al., 2005; Gersten, 2009; Jimerson, Burns, & VanDerHeyden, 2007; Mellard & Johnson, 2008; Pashler et al., 2007; VanDerHeyden & Burns, 2010). At this time, each instructor received a three-ring binder titled Engaging Developmental Math Students in Learning as the INSTRUCTOR RESOURCES manual for the retreat, and to add on-going training materials as we implemented and evaluated the MTSS model throughout the two year project. Their initial manuals included copies of the MTSS model, the conceptual framework for teaching and learning developmental mathematics, the student attribute measures, procedures for referring students to the Math Assistance Center, the essentials of explicit instruction, course syllabi, roles & responsibilities for instructors, students, and math mentors, and the MTSS model evaluation plan with data collection forms.
During this instructor retreat, the research and development team presented an overview of the NSF grant proposal and its use of evidence-based practice to improve student pass rates in developmental mathematics. We discussed what we were learning about our students when assessing math anxiety, math confidence, math usefulness, and academic motivation as well as what we were learning from them when they responded to “what supports success in passing Basic and Intermediate Algebra” at the end of the spring 2012 term. We concluded this session of the training with early descriptive and inferential statistics showing promise that ENGAGED STUDENTS SUCCEED.
At this point in the training, we introduced explicit instruction as a way to structure interactive learning while directly teaching fluency with numbers, procedural knowledge, and conceptual understanding (Archer & Hughes, 2011; Marchand-Martella, Slocum, & Martella, 2004; Pashler et al., 2007; Rosenshine, 1997). To direct implementing the MTSS model, we reviewed the HelpCard procedure to prompt students who fail a quiz to access Tier II level of instructional support in the Math Assistance Center.
After lunch, we focused on measuring student engagement in learning when evaluating MTSS model impact. We discussed the purpose of formative assessment to monitor daily academic progress and refer students for help in the Math Assistance Center, as well as the importance of consistency in recording instructional methods used for each class session throughout the two year project.
Finally, the research and development team recognized that implementing the model was dependent upon all eight instructors’ willingness to use scientifically-based research from the education sciences, especially explicit instruction during the two-year project. And as such, we asked the instructors to:
- discuss our “commitment to student success” on the first day of class when reviewing the course syllabus and signing teaching-learning contracts outlining instructor, student, and math mentor roles and responsibilities;
- plan classtime for researchers to seek student consent to participate in the NSF grant, administer student attribute measures, and complete automaticity pre-test;
- record daily student attendance;
- monitor daily progress to direct students who failed a quiz to seek Tier II support in the Math Assistance Center;
- self-report time spent using traditional instruction, explicit instruction, or test preview during 50 minute class session each day;
- work with math mentors assigned to each Tier I class 2 times a week and Tier III class 5 times a week in the classroom and discuss their in-class observations of instructional methods used during the 50 minute session (e.g., traditional instruction, explicit instruction, or assessment/review); and,
- participate in on-going instructor-research team workshops and follow along bi-monthly meetings throughout the two year project.
Our follow-along meetings during the second semester of the project established a trusting, collegial relationship for the professional development process to continue. The research and development team collaborated with these dedicated instructors to bring the MTSS model for students enrolled in Basic and Intermediate Algebra to life.
January – July 2013
Prior to beginning the spring 2013 term, the research team met with the developmental mathematics instructors to review first year results and recognize how significant the initial five instructors’ commitment to student success was in implementing the multi-tiered system of supports model by the end of the project’s first year. We also used this professional development day to analyze student and instructor survey data gathered at the end of the second semester and respond to this feedback regarding how the model was impacting students passing Basic and Intermediate Algebra.
To address instructor concern for students who refused our help, the research team prepared an instructor guide to prompt and reinforce student development of cognitive-behavioral learning skills necessary to pass Basic and Intermediate Algebra classes (Bandura, 1986; Bai, Wang, Pan, & Frey, 2009; Betz & Hackett, 1993; Deci & Ryan, 1985; Ironsmith, Marva, Harju, & Eppler, 2003; Langenfeld, 1993; Levine-Brown, Bonham, Saxon, & Boylan, 2008; National Math Advisory Panel, 2008; Pajares & Miller, 1994; Vizek Vidovic, 1999; Saxon, Levine-Brown, & Boylan, 2008). Added to their Instructor Resources manual, these materials reviewed how the multi-tiered system of supports model integrates student development of such critical self-directed learning skills by:
- administering student attribute instruments for students to self-assess “who and how they are” which may increase individual awareness of how personal strengths and weaknesses could influence academic success;
- using student/instructor/math mentor contracts for students to sign a commitment to actively engage in learning at the beginning of each semester;
- structuring classtime for students to practice think aloud, talk it through to solve learning outcome problems with a classmate, Math Mentor or instructor for interactive engagement in learning;
- completing daily homework assignments for independent practice after each class;
- self-monitoring learning success in passing quizzes and unit exams; and finally,
- self-evaluating academic progress and seeking help from the Math Assistance Center if failed quiz, and intensive instruction offered in Tier III if failed unit exam.
Finally, this presentation included the cognitive - behavioral research embedded in the MTSS model (Andriessen, Phalet, & Lens, 2006; Bandalos, Yates, & Thorndike-Christ, 1995; Barkoukis, Tsorbatzoudis, Grouios, & Sideridis, 2008; Betz & Hackett, 1993; Boylan, 2009; Brown-Chidsey, 2005; Cassady & Johnson, 2002; Cauley & McMillan, 2010; Cokley, Bernard, Cunningham, & Motoike, 2001; Dowling, 1978; Fennema & Sherman, 1976; Garavalia & Gredler, 2002; Hustinx, Kuyper, Van der Werf, & Dijkstra, 2009; Preckel, Holling, & Vock, 2006; Schloemer & Brenan, 2006; Vallerand et al., 1992). We then introduced a checklist to observe student and instructor behaviors when explicit instruction engages students in interactive learning (Archer & Hughes, 2011; Deschler et al., 2003; Gersten, 2009; Hollingsworth, & Ybarra, 2008; Loveless, 2011; Marchand-Martella, Slocum, & Martella, 2004; National Mathematics Advisory Panel, 2008; Pashler et al., 2007; Payzant, & Wolf, 1993; Woodward et al., 2012). Our checklist was adapted from “Record, Replay, Reflect” published by www.learningforward.org, suggested by co-author and our grant evaluator when we discussed using video recording Basic and Intermediate Algebra classes for continued professional development training during the second year of the project.
Subsequently, our professional development process integrated review of video recordings of Basic and Intermediate Algebra classes to reinforce faculty use of evidence-based practices employed in the MTSS model during the second year of the project. At bi-monthly follow along meetings, the research team and developmental math instructors reviewed class videos to further examine, define, and refine student engagement in interactive learning while using explicit instruction with adult learners. We piloted the video review framework during the spring 2013 semester to structure discussing developmental math class videos, and then re-evaluated continuing its use at our faculty development training prior to beginning the fall 2013 term.
The final professional development training was held on August 20, 2013 before fall classes began. At this time, the research team met with four of the 5 instructors involved in the first year of the project and three new faculty members who replaced two of the developmental math instructors who left the university at the end of the third semester of our 2 year project. During this instructor training, we summarized the NSF project goals and expected outcomes, reviewed how the multi-tiered system of supports model works, and presented student progress over the past three semesters.
We examined new and edited materials to be added to the Instructor Resources manual as we reviewed our work over the past two years. In addition, we discussed continuing the professional development approach using the video review framework at follow-along meetings during the last semester of the project. Instructor remarks were positive about this professional development process as they reflected on their individual videotaping experiences. Their engagement in using the “record, replay, reflect” as our professional development process was so supportive and encouraging for each other, one new faculty volunteered to video one of her classes before the training day ended!
From then on, we focused on the MTSS model’s referral procedures for Tier II and Tier III supports, and related data collection methods to evaluate the model’s impact on passing Basic and Intermediate Algebra. This aspect of the professional development session was led by the four seasoned faculty to share their knowledge and experiences implementing the MTSS model with the three new faculty members. This collegial relationship with each other provided peer support during the new faculty’s first semester involved with implementing and evaluating the MTSS model.
As we concluded the last training session, plans were finalized to review course syllabus, sign teaching-learning contracts, and complete automatically pre-tests, as well as seek student consent to participate in the project and administer the student attribute measures during the first three days of class. Lastly, dates were set for the follow-along instructor meetings to review class videos through the end of the fall semester as we completed the final semester of the two year project.
Monthly follow-along meetings continued to review class videos of instructors teaching and students learning Basic and Intermediate Algebra during the final semester of the project. These video tapes added new opportunities for developmental math instructors to discuss, refine, and reinforce the use of an explicit instruction routine to teach and engage students in interactive learning with the research and development team. Finally, we celebrated our success at the December meeting!
Lastly, instructor interviews were completed when gathering their final survey responses. Individually, each instructor expressed their appreciation for participating in the professional development process and being a part of improving pass rates in Basic and Intermediate Algebra. Likewise, we learned that new faculty may have benefited from additional orientation training in the referral procedure to prompt students for Tier II assistance. It is interesting to note that our students provided this training for new faculty!
Math Mentor Training
As importantly, Math Mentor training was offered by research team members and the Tier III instructor who provides oversight of the Math Assistance Center (MAC) each fall term during the two year project. At these sessions, we presented the MTSS model and the nature of our students; the referral and documentation procedures to access Help in the MAC; and, praised the math mentors for changing the culture in the Math Assistance Center to invite and help students enrolled in developmental mathematics.
Lastly, we reviewed the use of explicit instruction in the classroom and in the MAC. We then answered questions, and further clarified the math mentor role in the classroom to assist the instructor in providing performance feedback to students as they practice “think aloud and talk it through” as well as record the instructional methods they observe used when assigned Tier I and Tier III classes (i.e., traditional lecture, explicit instruction, assessment or review). All math mentors attended the 7am meeting and the pizza parties with instructors! It is important to note that half of the second year’s math mentors were new. And for the first time in MAC history, all mentors were math majors or math education majors. Likewise, the application pool had grown greater than the number of positions for the first time ever!