6-Step Process to Designing Curriculum: Evaluation (Part 6 of 6)

From Kern et al

By Kim Miklusak

Last semester I took a Foundations of Curriculum and Instruction course at UIC. Our textbook, while a medical curriculum textbook, reminds us that curriculum design crosses education fields and that what we are doing in our classes every year has its grounding in research.  Kern, Thomas, and Hughes in their book provide a 6-step approach to curriculum development.  My goal is to share the theory behind our current practices to serve as a guide as design and redesign our courses.  Earlier steps can be found here.


Step 6: Evaluation & Feedback--the final step!

According to Kern (et al), feedback "closes the loop in the curriculum development cycle" (122).  Gathering evaluative data and feedback from participants helps determine the success of the curriculum and bolster support for it moving forward.  Ultimately the question is: did the design of your curriculum meet your intended goals and resolve concerns brought up in Step 1 of the General and Targeted Needs Assessment?

Benefits & drawbacks of various feedback

The feedback could be formative and ongoing, or summative and final.  It could be based on individual participants or the program as a whole.  It could be qualitative or quantitative.  It could be about the students or from your peers.  However, when it comes down to it, the feedback should be accurate.  So the questions become how do we best gather the information?  Are the questions clear?  Are the results useful?  Are they are on some sort of scale?  Are the questions asked in a fair manner?  Are they replicable?  Are they surveys?  Interviews?  Focus groups?

In Practice:

I know many teachers have developed opportunities for feedback through their year and at the end of their year--especially with increased access to tools such as Google Forms or Schoology polls.  For example, in the past I have asked students before and after our AP exam to rate themselves in various skill and habits of work areas as well as to rank the usefulness of activities I have done throughout the year.  I have used this information to guide planning for the following year.  I know other peers at EG and on Twitter give students more open-ended surveys, allowing broader feedback, as well as specific in-the-moment questions at the start of class or at the end of a unit.  A few years ago we even pulled together a focus group of students to discuss the purpose of school!  You can see their insightful comments in our blog post here!  This was probably one of my favorite moments of the past few years.

Yet as we develop and re-develop our courses every year or every few years I wonder if we truly gather the data we need to consider the effectiveness of our courses.  We can use AP and SAT data, grades, test results, etc.  Does that data help us?  Do we use it to guide our instruction and assessment?  Furthermore, do we ask the students?  I recognize that sometimes they aren't aware in the moment of the usefulness of some things we do, so in some cases their feedback isn't quite as valid as we'd like.  But in those moments I wonder if we've done enough to explain to them why we're doing what we're doing, which is feedback in and of itself!  

In the end I encourage everyone to ask their students for feedback at the end of the year and to analyze data as an individual, as a team, and as a subject if possible.  What you ask and how you ask it should be in whatever manner is most comfortable to you.  But you may be surprised at how useful what the data and students have to say is when it comes to redesigning your course!

Please feel free to share other insights and ideas based on your experiences in the comments below!

6-Step Process to Designing Curriculum: Implementation (Part 5)

By Kim Miklusak

Last semester I took a Foundations of Curriculum and Instruction course at UIC. Our textbook, while a medical curriculum textbook, reminds us that curriculum design crosses education fields and that what we are doing in our classes every year has its grounding in research.  Kern, Thomas, and Hughes in their book provide a 6-step approach to curriculum development.  My goal is to share the theory behind our current practices to serve as a guide as design and redesign our courses.  Earlier steps can be found here.


Step 5: Implementation

Effective implementation of your curricular design, according to Thomas (et al), would include addressing all of the elements on this checklist displayed in the image below.  The checklist is broken down into several categories with specific considerations below each.
  
The checklist asks, for example, do we the appropriate faculty available for this course?  Do we have time, finances, and facilities to train them?  Do we have internal support from administration and external support, if it applies?  Do we have appropriate communication in place to facilitate between all personnel and operational support to distribute the materials needed?  Do we anticipate barriers and address them before they affect the curricular design?  And, finally, how do we roll out the curriculum and monitor it for adjustment?  Do we pilot a course, phase it in, or fully implement it?  Ideally, all of these steps would be addressed as we plan through our stages.  Considering them in advance would also help facilitate development and lead to smoother implementation--especially when partnered with a smooth first four steps of this design process.

In practice:

On the ground, on a daily basis, I wonder how many of these elements of the checklist are addressed as we plan.  Is it feasible for a classroom teacher to address any or all of them?  Is it the role of the PLC leader--if that structure even exists at your school?  Is it a department/division chair--is that person an administrator or a teacher-leader with possible release time?  Does that person have the ability to make these decisions, or are they more suggestions?


Please feel free to share other insights and ideas based on your experiences in the comments below!

6-Step Process to Designing Curriculum: Educational Strategies (Part 4)

By Kim Miklusak

I am currently taking a Foundations of Curriculum and Instruction course at UIC.  Our textbook, while a medical curriculum textbook, reminds us that curriculum design crosses education fields and that what we are doing in our classes every year has its grounding in research.  Kern, Thomas, and Hughes in their book provide a 6-step approach to curriculum development.  My goal is to share the theory behind our current practices to serve as a guide as design and redesign our courses.  Earlier steps can be found here.


Step 4: Educational Strategies

In the previous posts, after determining our targeted needs assessment and stating our goals and objectives, we now need to select our content.  For some subjects, the content and objectives overlap.  For others, the content may be more arbitrary.  Regardless of what content is decided upon, it always must build from the objectives and goals.  As stated in Kern, "transformative learning occurs when learners change in meaningful ways" (67). 

The final step in this part of the process is selecting the best educational methods for your content.  The selection must match with goals, yet at the same time be varied to meet the needs of the students, environment, and content.  Additionally, it must adjust to students' learning styles and preferences.  For higher-level, complex skills, oftentimes balancing a few methods works best over the course of a unit or a year.

In practice:

I find this element of curricular design the most interesting as a teacher.  It is true: we all have the methods that we feel most comfortable with day-to-day, and we all have the ones we feel most comfortable using in our content.  However, as Kern's chart shows, not all methods are best depending on our goals and objectives--or our students. 

Are students experiencing trouble reaching their goals?  Are we having classroom management issues?  Perhaps then there are times in our curriculum when we truly want to practice affective/attitudinal goals and could select the appropriate methods to do so.

However, this often requires us to step outside of our comfort zones--especially when coupled with new technology in the classroom or releasing control of our lessons to our students.  One of the ways I have helped work around this in my own teaching is by observing teachers across content areas.  I have learned so much by working with our Collab Lab team as well as learning with and from our peers in learning groups and lesson demos.

Please feel free to share other insights and ideas based on your experiences in the comments below!

6-Step Process to Designing Curriculum (Part 3)

By Kim Miklusak

From Kern, Thomas, and Hughes. See link above.

I am currently taking a Foundations of Curriculum and Instruction course at UIC.  Our textbook, while a medical curriculum textbook, reminds us that curriculum design crosses education fields and that what we are doing in our classes every year has its grounding in research.  Kern, Thomas, and Hughes in their book provide a 6-step approach to curriculum development.  My goal is to share the theory behind our current practices to serve as a guide as design and redesign our courses.  Steps 1 and 2 can be found here.

Step 3: Goals and Objectives


Once we have identified the needs of our learners, we need to clarify our goals and objectives.  While there may be some differences in how these terms are used in our various institutions, on the broader level, goals are the overall purposeful outcomes while objectives are measurable elements.  

When we state our goals, they help us to define and clarify our content, priorities, learning methods, and evaluation/assessment outcomes.  These are not only important for ourselves as we re/design our courses, but they are also important to communicate clearly to all stakeholders--peers, parents, students, administrators, etc.

According to Kern et al. there are five elements to keep in mind when writing a clear and measure objective: who will do how much (or how well) of what by when?  A key here is to use descriptors that are less open to interpretation.  For example, how can we measure the verb "know" as opposed to "identify, list, recite, define, etc."  

In practice:

For this step I wonder a few things, but many depend on the situations at our individual institutions.  For example, do we know the overall purpose of our course?  If there is not a clear "out" such as an AP exam or placement test, why do we cover the content and skills that we do?  Do we consider the general and targeted needs assessments as described in the previous blog posts?

Furthermore, does our course align to and build upon the goals of the courses before/after it in the sequence?  Does it need to?  If we are unable to articulate these goals and objectives, then we often end up duplicating assessments if not content and essential questions.  

Additionally, we may end up over-emphasizing a specific assessment-type when it doesn't really measure the outcome we are looking for.  For example, is the focus of my English class to read a novel or is the focus to practice skills via the novel?  If we are not clear in the focus of materials versus objectives, we may over-assess in some areas (for example plot of a text) when our main goal is something more sophisticated.  We also may misrepresent the total number and weight of questions that are not the focus of our stated objectives.  If we want students to practice more higher-level skills, more of our assessments should be weighted this way.

This further leads to a question of assessing Socio-Emotional skills and other subjective measures.  If we assess objectives like "paying attention," are those elements we instruct and model?  Are they truly the objectives we want to assess in our course?  If they are, we should be clear about how they are instructed and assessed.  If they are not, we should realign our focus to the objectives we do want to measure.

Once these goals and objectives are established, we are then able to move on to Step 4: Instructional Strategies, which I will discuss in the next blog post.

6-Step Process to Designing Curriculum (Part 2)

From Kern, Thomas, and Hughes. See link above.

By Kim Miklusak

I am currently taking a Foundations of Curriculum and Instruction course at UIC.  Our textbook, while a medical curriculum textbook, reminds us that curriculum design crosses education fields and that what we are doing in our classes every year has its grounding in research.  Kern, Thomas, and Hughes in their book provide a 6-step approach to curriculum development.  My goal is to share the theory behind our current practices to serve as a guide as design and redesign our courses.  Step 1, the General Needs Assessment, can be found here.

Step 2: Targeted Needs Assessment.
With the information gathered from the general needs assessment, the curriculum developer will now address their specific learners and learning environment.  This step allows developers to clarify what is already being taught and how it is being taught, including a focus on unmotivated learner.  Furthermore, the developer can set goals for future planning using information gathered.

There are two areas of focus in this stage: 

• The learner: what are the expectations on knowledge and skills?  What previous education do they have?  What characteristics do they have?  What are the perceived deficiencies and attitudes?  What are their preferences to learning strategies?

• The environment: What other curricula exist?  Who are other stakeholders affected?  What resources are available?  What barriers and reinforcing conditions exist that affect the learners (positively and negatively)

From this, the developer needs to consider by what means they will gather this information.  Each method has its own benefits and drawbacks related to resources, time, reliability, and so on.  For example, will the developer use surveys, interviews, questionnaires, tests, observation, etc.?  Will the results be quantitative or qualitative?  Would results be consistent across all developers?  Are the questions geared toward the targeted goals?

In Practice:
For this step I wonder how often we gather information from our learners when we re/design our curricula beyond looking at grades or testing data.  On Twitter I have seen an increasing number of teachers using surveys and questionnaires to gather feedback from students about things such as various attitudes, prior knowledge, or feedback on lessons.  We have tried this with our Senior English course this year, and from the information gathered, we decided we needed a SEL focus built into our course.  However, these methods are, as mentioned above, time consuming and not always accurate.  What would it look like to have one-on-one or small group interviews?  What qualities would we look for if we were observing a course in order to redesign it?  How do we ask the right questions in order to get the information we need?

Additionally, I wonder to what extent we consider the environment when redesigning our curriculum.  Do we consider barriers students have to success?  Do we consider reinforcing conditions that encourage them to succeed or not succeed?  In fact, do we survey our students who are unmotivated or unsuccessful to see how we can better adjust our curriculum to meet their needs--what would that method of information gathering look like? 

If you have samples of ways you have surveyed your learners in order to redesign your curricula, please share examples below!  Thanks!

In the next blog post I will discuss Step 3 where we analyze our goals and objectives.

6-Step Process to Designing Curriculum (Part 1)

By Kim Miklusak

From Kern, Thomas, and Hughes. See link above.

I am currently taking a Foundations of Curriculum and Instruction course at UIC.  Our textbook, while a medical curriculum textbook, reminds us that curriculum design crosses education fields and that what we are doing in our classes every year has its grounding in research.  Kern, Thomas, and Hughes in their book provide a 6-step approach to curriculum development.  I will cover the first step in the first installment.  My goal is to share the theory behind our current practices to serve as a guide as design and redesign our courses.

Step 1: General Needs Assessment
In the general needs assessment you will identify the problem: something simple like "how do we do a better job of teaching X, Y, Z" or a more complex question like "why are students not able to get through the entire curriculum?"  For instance, how do we improve writing of a research paper?  Or how do we engage students in metacognitive reading practice?  The problem may not deal with content at all and instead focus on areas like teacher qualities or student attitudes.  You will analyze whom the problem affects, what it affects, and the quantitative/qualitative importance of these effects.

In the end, you need to consider the current approach as compared to the ideal approach.  That gap between current vs. ideal is your needs assessment and should be investigated from the angle of all stakeholders--teachers, students, administrators, etc.

In Practice:
What does this mean for our day-to-day lives in curriculum design?  First, I wonder when we sit down to redesign our curriculum, if we start with identifying problems.  If there isn't an identifiable problem, why are we redesigning?  Do we have metrics to show something is missing or not working effectively?  Or do we focus on "I'd like to..." or "wouldn't it be fun if..." thoughts, which have their place, but may not be a priority.

I also wonder in our curriculum designs if we take into account teacher qualities or student attitudes.  Do we consider what other factors may be limiting success in our curriculum such as stakeholders' prior knowledge and attitudes, personal skills and environmental forces, and current rewards/punishments?  Some of this is out of our control, but these are areas we should at least investigate as we reflect upon potential problems to address.

In the next blog post I will discuss Step 2--the targeted needs assessments--where we engage other stakeholders in redesigning the curriculum process.

Matching Assessment Methods: Lead Learners #6

By The CollabLab Team

Yesterday's focus in Lead Learners was based on these two essential questions:

1.  How do we create conditions that maximize mastery of learning?

2.  How do we maximize student learning through sound assessment design?

The majority of our time was spent discussing the concept that assessment methods are not interchangeable.  Thus, we talked about the most relevant assessment method to be used for the type of learning target being assessed.  This helps to ensure accurate assessment results.  The four assessment methods are 1) selected response, 2) written response, 3) performance assessment, and 4) personal communication.

In groups, we did an activity where we were to match the strength of each of these assessment methods for the four types of learning targets.  See the grid below:

Many of us were surprised with the research-based results (provided below).  This spurred very interesting dialogue, especially when looking at different content areas!

Following this dynamic discussion, we had four groups of teachers share lesson demos. The first was our Bio team of Quinn Loch and Krista Glosson who discussed how they restructured their standards to increase a focus on key domains.  They also shared how they report grades both formatively and summatively in a non-standards-based online gradebook to communicate the most accurate information on achievement to students and parents.

Our second lesson demo was by Kim Miklusak, discussing the evolution of changes in the AP English Language grading rubrics from 4-degrees to 3 in order to clarify definitions of mastery and proficiency.  This rubric also mirrors the biology rubric to help increase cohesiveness of skills between content areas. 

Next, Bonnie Kale shared the idea of having one essential question in Honors Written and Oral Communications for a unit.  This unifying element allows for students and teachers to synthesize information and skills in order to maintain a focus throughout an entire novel.  This also allows the team to reloop through writing and reading skills all centered around one theme.

 

Finally, Sandra Lee and Katie Winstead shared how they differentiate formative feedback in Prep World Literature.  They build in elements of a pre- and post-test as well as clear learning targets in order to allow them to modify instruction to meet the needs of a wide range of student ability levels.  Regardless of grammar ability, all students learn the same materials at a different pace and with targeted teacher support.

Thank you to everyone who presented and participated yesterday!  We are looking forward to more conversations in upcoming PLTs and at next month's presentation!  To learn more, follow the #EGLLT hashtag on Twitter or stop into the CollabLab for more conversations!