One Year in AP: Assessment Data (Week Fourteen)

By Mark Heintz

Context

I have two main focuses as I write this weekly blog. Two driving questions that I have in my mind while making decisions.  They are:

• How do I know if my students know? 
• How do I get them to know if they know?  

Whether that is a skill or content, I want to know if they know it.   I no longer think it is acceptable for me to guess or get a feeling on whether or not they know it. Getting the students to know if they know it is downright hard, but I am really attempting to get to a point where the students can recognize their understandings or progress on their skill levels and content knowledge.  Therefore, the purpose of this year of reflection is to see how I make progress towards these two goals and elicit feedback from staff, students, and hopefully people who follow along on the journey.  You can read how last week went here.

Week Fourteen: Answer the Question


This week the content focus was on all of the standards the class has covered this year so far.  The primary focus was the 600-1450 time period.  Here were the standards for this unit:

1. List and locate the major trade routes and major cities during the time period of 600-1450.
2. List and locate the Sui, Tang, Song, Mali, Byzantine, Caliphates, and Mongol Empires.
3. List technologies used and provide one example as to how the technology encouraged interregional trade of luxury items.
4. List five historical example as to how Empires facilitated Afro-Eurasian trade.
5. List four examples as to how empires used conquered people in their economies
6. List three methods used in the expansion of Islam throughout Afro-Eurasia.
7. List one historical example as to how cross-cultural interactions resulted in the diffusion of advancements in Afro-Eurasia.
8. List the crops/diseases diffused and their effects.
9. Provide two examples as to how the Byzantine, and the Sui, Tang, Song reconstituted classical era governments through traditional and innovative sources of power to legitimize their rule.
10. Provide two examples of how new forms of governance emerged in Afro-Eurasia. 
11. Provide one example as to how the Islamic Caliphate and Japan each synthesized local with foreign traditions. 
12. Provide two examples as to how the Inca and Aztec created an imperial system in the Americas. 
13. Provide one example of how contacts or conflicts encouraged significant technological and cultural transfers.
14. The fate of cities: Provide three reasons that contributed to the decline of urban areas between 600-1450.
15. Provide five reasons that contributed to urban revival between 600-1450.
16. Provide one example as to how women increased their status in the Mongol Empire, West Africa, Japan and Southeast Asia.
17. Provide one detail as to how serfdom in Europe/Japan and Mita in the Inca Empire exemplified coerced labor.
18. Provide one example as to how peasants revolted in China and the Byzantine Empire.
19. Explain how the diffusion of Buddhism, Christianity, Islam, and Neo Confucianism changed gender relations. 

This week's skill focus was centered on analyzing charts, maps, and texts and pulling evidence from documents to support a claim.

1. Write one cause/effect, and one comparative short response that reflects the understanding of essential content.
2. Analyze charts, maps, graphs, and texts.
3. Write a thesis statement, contextualize a prompt, and draw evidence from two documents to support the thesis.  

Cite Specific Evidence

First, how do I know that the students know they knew the content and how to do the skills?

I largely focused on this question in last week's blog post. I was pretty confident in their ability on the dbq and the stimulus multiple choice, which is cited in the blog post.  To reinforce and assess content knowledge, I relied heavily on the Schoology checklists that the students complete each week. They couldn't take the assessments until they finished the checklists.  It is a small insurance policy to ensure that the students know they know the content.

How do I know that the students were successful on the big three focuses: content, writing, and analyzing?

The students took three exams.

1. An eighty question multiple-choice content exam.  These are just basic fact recall on all content that has been covered in the year.  The majority of the questions were centered on the current unit, but a fourth of the questions were pulled from previous units.

The students scored an average of 67/80 - garnering an 83%.  This was taken in a 48-minute period.  The students were all able to finish in the time.

2. A document based question writing portion on the role cities played in Muslim society with two documents.

The students were assessed using the following rubric:

The students scored an average of 2.68/4.  The students missed the argumentation point the most.

3. A twenty question multiple-choice stimulus exam.

The students scored an average of 14.1/20.

Explain the Reasoning 

What do all of those numbers mean and tell me?

1. For the eighty question multiple-choice content exam, the students improved from the last content exam by 2%.  Which is great!  We covered 900 additional years of history in this unit and the students are doing better.  While the average has increased, there some students who did not do so well.  I am working on how to better serve those students while maintaining what is working for the majority.  Overall, the content checklists, ongoing formative progress check throughout the unit, seems to be working for the students.

The biggest concepts missed were centered around sinification, the spread of Islam, and the Song Dynasty. I am very fortunate to work in a district with MasteryManager.  The item analysis feature is amazing.  To give you a sample:

The students struggled with how Islam spread into particular areas. I recognize that the spread is nuanced, but the course deals in generalities for these questions.

Also, there were a  few errors on my account - such as weird wording - and a few questions that could pertain to two or more empires. The averages above reflect that change.

2. For the DBQ, a 2.68 is amazing! The students scored a 2.1 on the previous unit's dbq.  It was the same rubric and skills.  A half point of growth is great.  Also, this is the skill we have practiced in this unit day in and day out.  It is great to see it pay off.

3. For the stimulus exam, the students answered on average about two questions more than in the previous unit.  This is a deal breaker.  Two questions on a stimulus exam might be the difference between scoring a 3 or a 4 or a 4 or a 5.  I credit this growth to the document work that we do on a daily basis.  They are reading a ton of primary and secondary sources that a difficult to digest.  Their daily grind appears to have worked!

After the exam, the students went through the tests and recorded what they missed and why they missed the questions. I borrowed heavily from Dan Saken and his Learning Celebrations.   A great tool to get the students to go through the assessments and reflect.

Reflection and Impact

Overall, I am extremely happy with the progress they are making.  There is still a lot to go.  There are three units and multiple more skills to master.  But the daily feedback on their writing and interpreting appears to be working.

I need to do something before the exams to ensure they know the content.  There needs to be a student reflection for them to know if they are ready and if they knew it.  I am not sure what it is yet, but I am working on it for second semester.

Read week fifteen here.