This week, a question was raised regarding progress monitoring and goal setting. The context here is that the school is using a hybrid attendance model where students attend two to three days per week. Typical COVID precautions such as distancing, masks, and limited movement through the school are also in use. Considering the current situation, ideal instruction is difficult to impossible to provide even in schools that are still 100% in person. My first question is how close to 100 percent of core instruction are students receiving at this time? I’m not pointing fingers but trying to gauge how much growth is realistic to expect from students, especially our most vulnerable learners? There are “best practices” for goal setting but what are the best practices for goal setting during a pandemic, when instruction is part-time, and students have not been in class for six months? That Google Scholar search turned up little for me.
First of all, what does the research indicate is best practice when delivering interventions? When seeking to obtain maximum effects of an intervention, students should receive the intervention the following elements (Gersten et al., 2011; Gersten et al., 2009; Gersten, Newman-Gonchar, Haymond, & Dimino, 2017)
· Instruction is provided in addition to the general curriculum (30 to 60 minutes, 4 to 5 days a week)
· Instruction is provided in small groups or individually (6 or less)
· Targeted instruction focuses on 3 or fewer skills
· Ample practice and feedback is provided when learning new skills
· Monitoring of progress occurs regularly
The skill deficits students have, have not decreased, yet the amount of instruction students receive may be reduced drastically. If a district is providing 100% virtual and online instruction, there is the potential that an intervention could be delivered with adequate dosage (≥ 30 minutes/4-5 times per week). The mode of delivery would change from in-person to online which may have some effects but the dosage could be close to what the student would receive if he or she was attending in person.
However, if hybrid attendance models are used where students attend partial days or 2-3 days per week, teachers are working 100% of the time but with only half of the students. In this way, providing optimal dosage is not likely or even possible. Operating procedures may be similar for teachers but students are on a drastically different schedule, influenced by what they can accomplish at home, and they missed 3 or more months of school last spring.
The goal-setting question was raised within a hybrid attendance model where students attend 2-3 days a week. Some weeks, the student attends twice and some weeks the student attends a third day. If the student is provided an intervention, the best-case scenario is 3 sessions a week. In-person with adequate attendance, the student would receive 16 sessions during a 4-week time frame. In the hybrid attendance model, the student would likely receive 10 sessions. That is roughly a 40% loss of sessions or minutes of instruction. Setting an ambitious goal is normally completed assuming instruction would be in addition to the core curriculum, delivered daily.
I am a supporter of automation for assessment and progress monitoring. Even before COVID-19, finding time to assess, monitor, and score routine CBMs was challenging for most school teams. The more the online system could do for us, the better and likely leads to increased measurement fidelity. But, this might be a time to pull out the norms charts and adjust goals based on what makes sense considering the current situation.
On the other hand, I have seen a loss in the understanding of how and why we set goals and the rationale for tailoring ambitious goals for students. Due to automation, many teachers will enter a fall baseline score and then use a slider bar to set a goal that turns a certain color or is labeled “ambitious” when a certain score is reached. These calculations have been automated for some time but “scoring by hand” sometimes has the benefits of slowing things down and forcing us to think about what makes sense for a student.
A couple of methods could be considered when setting goals using assessments with established, research-based, rates of improvement.
Option 1, move from one quadrant to the next. This method would involve looking up your CBM or monitoring system’s norms tables to do some estimation to set and review goals. Using the student’s benchmark data, look at which quadrant the initial score was in. If the student was in the 10th to 25th percentile range, look at whether moving from that quadrant to the next (25th to 50th) is realistic.
If a student begins the year with 17 words per minute, moving to 81 would require growth of 64 words per minute over the 30 weeks we mentioned previously. That is a gain of 64 words per minute. Using the formula of (end-beginning= growth/weeks) would result in:
1. 81-17= 64
2. 64/30= 2.1 ROI per week
By checking the norms chart, an ROI of 2.1 is likely very ambitious and could be demotivating to a student. If instruction is only provided 2-3 times a week, expecting ambitious growth even during normal times is ambitious. Option 1 could be considered for students on the upper end of the band for those students who are likely to cross into the next quadrant. For example, a student with a fall benchmark of 33 might be a good candidate to use Option 1.
This may be a year where maintenance goals are the best we can set. If a school is in an area with high levels of COVID, keeping students on track to maintain and not lose, may be justified. Use the ROI to set a goal to keep the student at the same level throughout the year and reinforce and celebrate when the student beats his or her goal.
Option 2- Modified Goal Setting: Dr. Shapiro (2008) authored a clear and sequential method of goal setting when using CBMs. When setting goals, the suggestion was to establish goals with 150 to 200% growth. For example, with a 2nd-grade student, the weekly rate of improvement (ROI) on Oral Reading would be approximately 1.2 words per week, using a benchmark score between the 10th to 25th percentile with a little rounding. Using growth of 150% would result in an ROI of approximately 1.8 (1.2x 1.5= 1.8). Using 200% growth would result in an ROI of 2.4. If a student had a fall benchmark in 2nd grade of 17 and there were 30 weeks of school left, the following formula was used to calculate a moderately ambitious rate of improvement (150%).
1. ROI: 1.2 x 1.5= 1.8
2. Growth during year : 1.8 x 30= 54
3. Goal= Baseline + Growth : 17+54 = 71
The resulting goal of 71, is closer to the 25th percentile band than the 10th percentile band, which is not ideally the destination we seek for students. However, continuing at least monthly monitoring and encouraging the student to practice and celebrating progress could lead to greater gains. These are not ideal times and there are more factors present that what we likely see.
Self-monitoring and updating students on progress have been indicated to be an effective strategy and motivational for students (Hirsch, Ennis, & Mcdaniel, 2013; Menzies, Lane, & Lee, 2009). I would argue that sharing growth and progress is beneficial although growth targets may need to be adjusted this year. Using relative goals compared to normative would be suggested so the student can see his/her goal of 71 and his or her progress toward meeting the goal. Every bit of gap closure is beneficial and hopefully, we can provide more intensive instruction in the near future.
To answer the original question about how ambitious to set goals, my suggestion would be to set goals with ROIs that are 125-150% above expected. Normally, I would advocate setting a goal closer to 200% above the expected ROI. The best case for many students is that they are suffering from boredom and missing friends due to attempts at social distancing. Gap closure is critical for future success although we are operating in times where we do not know what students are facing outside our schools. Using scaled-back goals provides students with a target and also provides moments to celebrate for students who can crush their goals. For students who are struggling, having a scaled-back goal could provide them with an attainable target and a reason to keep trying. Keeping kids engaged and hopeful is critical during this time and through what could be a challenging winter.
Just out of curiosity, are your schools even setting goals at this time?
Are you using ambitious or CBM system recommendations or scaling back? Please let me know.
References:
Gersten, R., Beckman, S., Clarke, B., Foegen, A., Marsh, L., Star, J. R., & Witzel, B. (2011). Assisting students struggling with mathematics – Response to Intervention (RtI) for elementary and middle schools. A focus on assessment.
Gersten, R., Compton, D., Connor, C. M., Dimino, J., Santoro, L., Linan-Thompson, S., & Tilly, W. D. (2009). Assisting Students Struggling with Reading : Response to Intervention ( RtI ) and Multi-Tier Intervention in the Primary Grades. Assisting Students Struggling with Reading: Response to Intervention and Multi-Tier Intervention for Reading in the Primary Grades. A Practice Guide., 4045, 883–890. https://doi.org/10.1016/j.jhazmat.2011.04.026
Gersten, R., Newman-Gonchar, R., Haymond, K. S., & Dimino, J. (2017). What is the evidence base to support reading interventions for improving student outcomes in grades 1 – 3 ? Ies.Ed.Gov, (April), 1–13. Retrieved from https://ies.ed.gov/ncee/edlabs/regions/southeast/pdf/REL_2017271.pdf%0Apapers3://publication/uuid/0EDAD747-3EB8-44B3-A84E-B83636C1AC66
Hirsch, S. E., Ennis, R. P., & Mcdaniel, S. C. (2013). Student Self-Graphing as a Strategy to Increase Teacher Effectiveness and Student Motivation. Beyond Behavior, 22(3), 31–39. https://doi.org/10.1177/107429561302200305
Menzies, H. M., Lane, K. L., & Lee, J. M. (2009). Self-Monitoring Strategies for Use in the Classroom: A Promising Practice to Support Productive Behavior for Students with Emotional or Behavioral Disorders. Beyond Behavior, 18(April 2020), 27–35. Retrieved from http://ezproxy.umsl.edu/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=EJ869682&site=ehost-live&scope=site%5Cnhttp://www.ccbd.net/content/self-monitoring-strategies-use-classroom-promising-practice-support-productive-behavior-stud
Shapiro, E. S. (2008). Best practices in setting progress monitoring goals for academic skill improvement. In A. Thomas and J. Grimes (Eds.), Best practices in school psychology V. (Vol. 2, pp. 141-157). Bethesda, MD: National Association of School Psychologists.