6-Step Process to Designing Curriculum: Evaluation (Part 6 of 6)

From Kern et al

By Kim Miklusak

Last semester I took a Foundations of Curriculum and Instruction course at UIC. Our textbook, while a medical curriculum textbook, reminds us that curriculum design crosses education fields and that what we are doing in our classes every year has its grounding in research.  Kern, Thomas, and Hughes in their book provide a 6-step approach to curriculum development.  My goal is to share the theory behind our current practices to serve as a guide as design and redesign our courses.  Earlier steps can be found here.


Step 6: Evaluation & Feedback--the final step!

According to Kern (et al), feedback "closes the loop in the curriculum development cycle" (122).  Gathering evaluative data and feedback from participants helps determine the success of the curriculum and bolster support for it moving forward.  Ultimately the question is: did the design of your curriculum meet your intended goals and resolve concerns brought up in Step 1 of the General and Targeted Needs Assessment?

Benefits & drawbacks of various feedback

The feedback could be formative and ongoing, or summative and final.  It could be based on individual participants or the program as a whole.  It could be qualitative or quantitative.  It could be about the students or from your peers.  However, when it comes down to it, the feedback should be accurate.  So the questions become how do we best gather the information?  Are the questions clear?  Are the results useful?  Are they are on some sort of scale?  Are the questions asked in a fair manner?  Are they replicable?  Are they surveys?  Interviews?  Focus groups?

In Practice:

I know many teachers have developed opportunities for feedback through their year and at the end of their year--especially with increased access to tools such as Google Forms or Schoology polls.  For example, in the past I have asked students before and after our AP exam to rate themselves in various skill and habits of work areas as well as to rank the usefulness of activities I have done throughout the year.  I have used this information to guide planning for the following year.  I know other peers at EG and on Twitter give students more open-ended surveys, allowing broader feedback, as well as specific in-the-moment questions at the start of class or at the end of a unit.  A few years ago we even pulled together a focus group of students to discuss the purpose of school!  You can see their insightful comments in our blog post here!  This was probably one of my favorite moments of the past few years.

Yet as we develop and re-develop our courses every year or every few years I wonder if we truly gather the data we need to consider the effectiveness of our courses.  We can use AP and SAT data, grades, test results, etc.  Does that data help us?  Do we use it to guide our instruction and assessment?  Furthermore, do we ask the students?  I recognize that sometimes they aren't aware in the moment of the usefulness of some things we do, so in some cases their feedback isn't quite as valid as we'd like.  But in those moments I wonder if we've done enough to explain to them why we're doing what we're doing, which is feedback in and of itself!  

In the end I encourage everyone to ask their students for feedback at the end of the year and to analyze data as an individual, as a team, and as a subject if possible.  What you ask and how you ask it should be in whatever manner is most comfortable to you.  But you may be surprised at how useful what the data and students have to say is when it comes to redesigning your course!

Please feel free to share other insights and ideas based on your experiences in the comments below!

6-Step Process to Designing Curriculum: Implementation (Part 5)

By Kim Miklusak

Last semester I took a Foundations of Curriculum and Instruction course at UIC. Our textbook, while a medical curriculum textbook, reminds us that curriculum design crosses education fields and that what we are doing in our classes every year has its grounding in research.  Kern, Thomas, and Hughes in their book provide a 6-step approach to curriculum development.  My goal is to share the theory behind our current practices to serve as a guide as design and redesign our courses.  Earlier steps can be found here.


Step 5: Implementation

Effective implementation of your curricular design, according to Thomas (et al), would include addressing all of the elements on this checklist displayed in the image below.  The checklist is broken down into several categories with specific considerations below each.
  
The checklist asks, for example, do we the appropriate faculty available for this course?  Do we have time, finances, and facilities to train them?  Do we have internal support from administration and external support, if it applies?  Do we have appropriate communication in place to facilitate between all personnel and operational support to distribute the materials needed?  Do we anticipate barriers and address them before they affect the curricular design?  And, finally, how do we roll out the curriculum and monitor it for adjustment?  Do we pilot a course, phase it in, or fully implement it?  Ideally, all of these steps would be addressed as we plan through our stages.  Considering them in advance would also help facilitate development and lead to smoother implementation--especially when partnered with a smooth first four steps of this design process.

In practice:

On the ground, on a daily basis, I wonder how many of these elements of the checklist are addressed as we plan.  Is it feasible for a classroom teacher to address any or all of them?  Is it the role of the PLC leader--if that structure even exists at your school?  Is it a department/division chair--is that person an administrator or a teacher-leader with possible release time?  Does that person have the ability to make these decisions, or are they more suggestions?


Please feel free to share other insights and ideas based on your experiences in the comments below!

6-Step Process to Designing Curriculum: Educational Strategies (Part 4)

By Kim Miklusak

I am currently taking a Foundations of Curriculum and Instruction course at UIC.  Our textbook, while a medical curriculum textbook, reminds us that curriculum design crosses education fields and that what we are doing in our classes every year has its grounding in research.  Kern, Thomas, and Hughes in their book provide a 6-step approach to curriculum development.  My goal is to share the theory behind our current practices to serve as a guide as design and redesign our courses.  Earlier steps can be found here.


Step 4: Educational Strategies

In the previous posts, after determining our targeted needs assessment and stating our goals and objectives, we now need to select our content.  For some subjects, the content and objectives overlap.  For others, the content may be more arbitrary.  Regardless of what content is decided upon, it always must build from the objectives and goals.  As stated in Kern, "transformative learning occurs when learners change in meaningful ways" (67). 

The final step in this part of the process is selecting the best educational methods for your content.  The selection must match with goals, yet at the same time be varied to meet the needs of the students, environment, and content.  Additionally, it must adjust to students' learning styles and preferences.  For higher-level, complex skills, oftentimes balancing a few methods works best over the course of a unit or a year.

In practice:

I find this element of curricular design the most interesting as a teacher.  It is true: we all have the methods that we feel most comfortable with day-to-day, and we all have the ones we feel most comfortable using in our content.  However, as Kern's chart shows, not all methods are best depending on our goals and objectives--or our students. 

Are students experiencing trouble reaching their goals?  Are we having classroom management issues?  Perhaps then there are times in our curriculum when we truly want to practice affective/attitudinal goals and could select the appropriate methods to do so.

However, this often requires us to step outside of our comfort zones--especially when coupled with new technology in the classroom or releasing control of our lessons to our students.  One of the ways I have helped work around this in my own teaching is by observing teachers across content areas.  I have learned so much by working with our Collab Lab team as well as learning with and from our peers in learning groups and lesson demos.

Please feel free to share other insights and ideas based on your experiences in the comments below!

6-Step Process to Designing Curriculum (Part 3)

By Kim Miklusak

From Kern, Thomas, and Hughes. See link above.

I am currently taking a Foundations of Curriculum and Instruction course at UIC.  Our textbook, while a medical curriculum textbook, reminds us that curriculum design crosses education fields and that what we are doing in our classes every year has its grounding in research.  Kern, Thomas, and Hughes in their book provide a 6-step approach to curriculum development.  My goal is to share the theory behind our current practices to serve as a guide as design and redesign our courses.  Steps 1 and 2 can be found here.

Step 3: Goals and Objectives


Once we have identified the needs of our learners, we need to clarify our goals and objectives.  While there may be some differences in how these terms are used in our various institutions, on the broader level, goals are the overall purposeful outcomes while objectives are measurable elements.  

When we state our goals, they help us to define and clarify our content, priorities, learning methods, and evaluation/assessment outcomes.  These are not only important for ourselves as we re/design our courses, but they are also important to communicate clearly to all stakeholders--peers, parents, students, administrators, etc.

According to Kern et al. there are five elements to keep in mind when writing a clear and measure objective: who will do how much (or how well) of what by when?  A key here is to use descriptors that are less open to interpretation.  For example, how can we measure the verb "know" as opposed to "identify, list, recite, define, etc."  

In practice:

For this step I wonder a few things, but many depend on the situations at our individual institutions.  For example, do we know the overall purpose of our course?  If there is not a clear "out" such as an AP exam or placement test, why do we cover the content and skills that we do?  Do we consider the general and targeted needs assessments as described in the previous blog posts?

Furthermore, does our course align to and build upon the goals of the courses before/after it in the sequence?  Does it need to?  If we are unable to articulate these goals and objectives, then we often end up duplicating assessments if not content and essential questions.  

Additionally, we may end up over-emphasizing a specific assessment-type when it doesn't really measure the outcome we are looking for.  For example, is the focus of my English class to read a novel or is the focus to practice skills via the novel?  If we are not clear in the focus of materials versus objectives, we may over-assess in some areas (for example plot of a text) when our main goal is something more sophisticated.  We also may misrepresent the total number and weight of questions that are not the focus of our stated objectives.  If we want students to practice more higher-level skills, more of our assessments should be weighted this way.

This further leads to a question of assessing Socio-Emotional skills and other subjective measures.  If we assess objectives like "paying attention," are those elements we instruct and model?  Are they truly the objectives we want to assess in our course?  If they are, we should be clear about how they are instructed and assessed.  If they are not, we should realign our focus to the objectives we do want to measure.

Once these goals and objectives are established, we are then able to move on to Step 4: Instructional Strategies, which I will discuss in the next blog post.

6-Step Process to Designing Curriculum (Part 2)

From Kern, Thomas, and Hughes. See link above.

By Kim Miklusak

I am currently taking a Foundations of Curriculum and Instruction course at UIC.  Our textbook, while a medical curriculum textbook, reminds us that curriculum design crosses education fields and that what we are doing in our classes every year has its grounding in research.  Kern, Thomas, and Hughes in their book provide a 6-step approach to curriculum development.  My goal is to share the theory behind our current practices to serve as a guide as design and redesign our courses.  Step 1, the General Needs Assessment, can be found here.

Step 2: Targeted Needs Assessment.
With the information gathered from the general needs assessment, the curriculum developer will now address their specific learners and learning environment.  This step allows developers to clarify what is already being taught and how it is being taught, including a focus on unmotivated learner.  Furthermore, the developer can set goals for future planning using information gathered.

There are two areas of focus in this stage: 

• The learner: what are the expectations on knowledge and skills?  What previous education do they have?  What characteristics do they have?  What are the perceived deficiencies and attitudes?  What are their preferences to learning strategies?

• The environment: What other curricula exist?  Who are other stakeholders affected?  What resources are available?  What barriers and reinforcing conditions exist that affect the learners (positively and negatively)

From this, the developer needs to consider by what means they will gather this information.  Each method has its own benefits and drawbacks related to resources, time, reliability, and so on.  For example, will the developer use surveys, interviews, questionnaires, tests, observation, etc.?  Will the results be quantitative or qualitative?  Would results be consistent across all developers?  Are the questions geared toward the targeted goals?

In Practice:
For this step I wonder how often we gather information from our learners when we re/design our curricula beyond looking at grades or testing data.  On Twitter I have seen an increasing number of teachers using surveys and questionnaires to gather feedback from students about things such as various attitudes, prior knowledge, or feedback on lessons.  We have tried this with our Senior English course this year, and from the information gathered, we decided we needed a SEL focus built into our course.  However, these methods are, as mentioned above, time consuming and not always accurate.  What would it look like to have one-on-one or small group interviews?  What qualities would we look for if we were observing a course in order to redesign it?  How do we ask the right questions in order to get the information we need?

Additionally, I wonder to what extent we consider the environment when redesigning our curriculum.  Do we consider barriers students have to success?  Do we consider reinforcing conditions that encourage them to succeed or not succeed?  In fact, do we survey our students who are unmotivated or unsuccessful to see how we can better adjust our curriculum to meet their needs--what would that method of information gathering look like? 

If you have samples of ways you have surveyed your learners in order to redesign your curricula, please share examples below!  Thanks!

In the next blog post I will discuss Step 3 where we analyze our goals and objectives.

6-Step Process to Designing Curriculum (Part 1)

By Kim Miklusak

From Kern, Thomas, and Hughes. See link above.

I am currently taking a Foundations of Curriculum and Instruction course at UIC.  Our textbook, while a medical curriculum textbook, reminds us that curriculum design crosses education fields and that what we are doing in our classes every year has its grounding in research.  Kern, Thomas, and Hughes in their book provide a 6-step approach to curriculum development.  I will cover the first step in the first installment.  My goal is to share the theory behind our current practices to serve as a guide as design and redesign our courses.

Step 1: General Needs Assessment
In the general needs assessment you will identify the problem: something simple like "how do we do a better job of teaching X, Y, Z" or a more complex question like "why are students not able to get through the entire curriculum?"  For instance, how do we improve writing of a research paper?  Or how do we engage students in metacognitive reading practice?  The problem may not deal with content at all and instead focus on areas like teacher qualities or student attitudes.  You will analyze whom the problem affects, what it affects, and the quantitative/qualitative importance of these effects.

In the end, you need to consider the current approach as compared to the ideal approach.  That gap between current vs. ideal is your needs assessment and should be investigated from the angle of all stakeholders--teachers, students, administrators, etc.

In Practice:
What does this mean for our day-to-day lives in curriculum design?  First, I wonder when we sit down to redesign our curriculum, if we start with identifying problems.  If there isn't an identifiable problem, why are we redesigning?  Do we have metrics to show something is missing or not working effectively?  Or do we focus on "I'd like to..." or "wouldn't it be fun if..." thoughts, which have their place, but may not be a priority.

I also wonder in our curriculum designs if we take into account teacher qualities or student attitudes.  Do we consider what other factors may be limiting success in our curriculum such as stakeholders' prior knowledge and attitudes, personal skills and environmental forces, and current rewards/punishments?  Some of this is out of our control, but these are areas we should at least investigate as we reflect upon potential problems to address.

In the next blog post I will discuss Step 2--the targeted needs assessments--where we engage other stakeholders in redesigning the curriculum process.

Student Voice: Models of Student Work

By Mark Heintz

I have been going around school stopping students at random to hear what they feel teacher's do that impacts their learning.  It is so important to continually ask the population we serve to get their feedback on what helps them master the content and skills we are trying to teach.  The responses have been so insightful into what works for each of the unique learners that enter our classroom and can continue to drive the methods we use to instruct them.  Once I captured the student's voice, I tracked down the teacher to share and get their input on the practice that was highlighted. 

The student I asked in the video highlights the way Mr. Asmussen uses different levels of student samples to improve students writing ability. 

Here is Mr. Asmussen explaining the impact and process of using student samples in his instruction.  

Sharing Learning Targets with Students

By Melissa Curtis

We’ve been hearing a lot about clear learning targets and meaningful feedback the last couple of years.  Many teachers I know have been using learning targets in their curriculum for years, but maybe not explicitly sharing them with their students or getting any feedback.  I have been using a simple yet effective way to use daily goals or learning targets in my classes that might be helpful to other teachers.

First, I have created a document called Daily Goals that I post in Schoology.

Students open this document in Notability every week.  They can reuse the same one from week to week and just erase last week’s info.  Some choose to upload it from Schoology each week and “add to existing note” so they have one long note instead of 36 different ones throughout the school year.  Each day when they come into class, I have the daily goal projected on the screen for them to write down in the left-hand column.  I post these in my calendar in Schoology so even when students are absent, they can see the daily goal and any handouts/links we used that day.  Here is an example:

At the end of class, I ask students to write a response to the daily goal.  This should be a thoughtful comment about what we learned that day in class.  It guides students and myself to focus on the clear purpose of the lesson.  By Friday, their daily goals document should look like this:

Every Monday, or first day of the week, I walk around the room and check their goals from the prior week.  I give students a small homework grade of 5 points for this.  The real value though is in getting a snapshot of what students learned that week.  In their own words, they are narrowing down my curriculum into the simplest forms.  I usually spot check one or two days for each student and I can comment on any mistakes they have made.  I can also get a sense of the whole class and clear up any misconceptions as a group.  

In my experience, students like the structure and routine of a class.  They know when they come in to take out their iPads and start writing down the daily goal.  At the end of class, it provides closure for the lesson and I can call on random students to help answer it for the whole class.  When students are absent, it provides a quick answer to “What did I miss yesterday?”.  It has become an expectation that most students buy into pretty quickly.  Yes, there are some who don’t do it or copy it from a friend, but you can monitor those individual cases pretty easily.  I have been doing this for years and like the simplicity and accountability of the daily goals.  Now, I will start calling them Learning Targets :)

Feel free to use or modify anything you see here!  

Assessment as an Accurate Reflection of Grades

By Mark Heintz

I missed a few weeks of school due to the birth of my second son.  Meanwhile at school, the students continued their work on their understanding of the Classical Era in history.  Before my leave of absence, the students worked on thesis statements, short answer, and document work.  I had a good understanding on progress towards those skills.  Once I was away, their in class work and homework was geared towards content knowledge and mastery of the objectives of the Classical Era.

Here is a sample of one of the objective:

There is a lot of information about Buddhism in the classical era.  There are books written on just one aspect of the faith in the early years.  However, for the sake of this class, I codified the essentials.

For years, I have debated the best way to assess.  I worry about the validity of the test in its ability to assess a students knowledge of the core objectives.  I worry if it is too easy or too difficult.  But I have to make a decision.  For this assessment, its purpose was to test low level content knowledge.  Its targets were extremely clear.  In its design, I limited the use of academic vocabulary in the question stems.  I also limited the skills involved in the assessment, such as change over time, comparative, cause/effect, and periodization.  I wanted to test their content knowledge and that is what I tested.  Here are two sample questions:

The results were great! Prior to the summative assessment, the students took benchmark tests that assessed their understanding of the content knowledge.  From the student performance on the formative assessments, I thought they would do well.  In the end, the students felt the assessment to be fair and assessed what they were asked.

In the future, I wish to keep this type of assessment and add more essays and stimulus based assessments to test their skills.  I also would like to find a better way to communicate which standards they faltered on.  Also, I need to find a better way to put the score in the grade book.  Do I put the three big standards in the grade book? Do I put each objective in the grade book?   These are things that still need to be addressed.

Author's Purpose and Thesis Development

By Kim Miklusak and Mark Heintz

Bonnie Kale invited us in to see her sophomores analyze each other's work after the three of us had been talking this year about the purpose of our courses.  We quickly realized each of our classes has been getting students to create arguments and defend them with specific pieces of evidence.  Another point of our courses to have students know that authors use different approaches to getting their arguments across to the reader.  So traditionally we rely on the 5-paragraph paper format because it's easy to teach at younger grades, but we realize it's so limiting as students progress.  Bonnie's class was working on getting students to see these patterns and reflect on their own reading and writing.

Prior to the class we saw, the students read a short story and created an outline for their literary analysis.  The students were tasked with two things in their literary analysis. For the first task, the students attempted to identify the author's main idea. For the second task, the students attempted to identify the organizational approach used by the author.

The students had to include both tasks in their thesis for their literary analysis. For the lesson, Mrs. Kale wanted students to identify if students completed both tasks.  She was not having the students provide feedback to other students.  She wanted to see if students could tell when other students completed the task and their effectiveness in the task.  The students walked around the room and attempted to see if other students completed the task using a graphic organizer on their iPads.

Before allowing students to move on to the actual writing of the literary analysis, Mrs. Kale ensured students knew when they were writing to the task at hand.  There has been so much overlap in the skill, the students are seeing that they are writing to a specific audience and for a specific purpose.  Moreover, they are seeing that authors are writing to a specific audience and for a specific purpose in all disciplines.  Some authors are more effective in their argument than others. That same day, all three of us inadvertently focused on the same skill!

Hopefully as this year's sophomores move into juniors, they will have more background in the foundational and cross-disciplinary approach to crafting arguments and supporting those arguments with specific pieces of evidence.  

Lead Learners Wrap-Up

By The Collab Lab Team

Today we had the final Lead Learners meeting of the school year.  We started off by activating prior knowledge and sharing our successes this year:

We then had a series of lesson demonstrations.  The first was from Grove Junior High teachers Tracy Groark and Kerry Frazier who shared the evolution of how they communicate learning targets with their students and new and more intentional ways they connect instruction and assessments more to those learning targets. They both included examples of how they use Google Forms, and even Kahoot to engage students and better understand gaps in their understanding.

Leslie Guimon then shared the evolution of learning targets on the Spanish team.  She shared how the courses were once aligned thematically, then grammatically, and so on.  After discussing at length and agreeing on Enduring Understandings for all levels of Spanish, they developed clear learning targets for each level.

 

The final demonstration was from Tom Boczar.   He discussed how students in Physics classes reflect on their assessments and objectives, using a rubric to determine understanding.  Was there a disconnect or misconception about what they knew?  Was there uncertainty--a lack of confidence or guessing?  Or was there mastery.  The reflection is student-directed, and Tom noticed how important it was to students when he almost skipped the process after a recent unit.  The students objected and insisted he not skip it!

      

Following the lesson demonstrations, teachers engaged in a self assessment activity.  Each participant was asked to bring an assessment they currently use.  Using the self-assessment checklist below, teachers individually assessed the method they were using, where their targets fell, and if their assessment met four conditions for a quality assessment. The group then shared out their "A-Ha" moments about their own assessments, and plans for next steps.

This final EGLLT wrapped up with a year-end evaluation and time for teachers to collaborate in course-alike teams to continue conversations about their next steps. Feedback from staff on the evaluation will be used to plan future professional learning experiences.