29 Jan Automated Assessment for Introduction to New Testament
Posted at 11:53h in Biblical Studies, Education by Jason A. Staples3 Comments
Two posts ago, I talked about one big change I made to my Introduction to the New Testament class last summer, choosing to take the students through the Synoptic Gospels before teaching the Synoptic Problem itself. That change seemed immensely helpful, as it took an important (but typically uninteresting to the students) subject and forced the students to see the problem before the theoretical solutions. Another change I made was more administrative and took some time to set up but will now be a feature of all my future Introduction to New Testament courses: using Respondus, I established a test bank of around 1,500 questions that I can import into any course management system out there (Blackboard, Sakai, etc.). I used this test bank to create regular online (timed) quizzes on Blackboard to accompany the reading, quizzes that auto graded and instantly gave me what percentage of students got a given question right or wrong, giving some insight into whether I should spend a little extra time on a given point in class.
These questions are largely multiple-choice, but they also include matching, fill-in-the-blank, and other objective question structures. Although in most subjects I am something of a critic of multiple-choice questions, I think they can actually be very effective in New Testament (and Hebrew Bible) introductory courses if written properly. For example, I am a big proponent of “verse identification” questions, which ask students to identify which book a given verse is from. If a student can identify that the verse including “thus he made all things clean” is from the Gospel of Mark, it indicates that the student has actually processed some important thematic issues within the Gospels. Essentially, my goal is to force “essay level reflection for multiple choice questions,” asking questions that force students to think about why a given verse must be from a given book rather than another. I also do thematic questions (e.g. “which Gospel portrays Jesus as especially concerned with the poor?”) and other similar objective questions that require students to have understood the essence of what has been covered in the class. Then of course there are actual historical/data questions, asking about, say, the Pharisees or Alexander the Great. These sorts of questions, taken together, can really give a good picture of whether a student has grasped the material necessary for the course. (That the students came out to an average in the low “B” range with a median in the B+ range—which is about where I as an opponent of grade inflation would generally like them to wind up—was also a pleasant surprise.)
I also used these question banks—which included a pool of essay questions—to construct the midterm and final examinations, which (aside from the essays) auto graded and again gave instant access to student performance data on a per-question basis. Using automated tests both reduced my time grading and gave easier access to better assessment data, a win-win proposition. The students also generally found this arrangement preferable to other testing and assessment options. They did request that the essay portion of the exam be separate from the rest of the questions for the final exam, as the randomized question structure had thrown the essays into the mix at awkward times on the midterm. To address this issue, I simply created two separate exams—one essay, one with objective questions—that together made up the final exam.
The other advantage to putting in the extra time to create these test pools is the reduction in future test and quiz creation time for future courses. Because the test pools are so large and include a range of questions for each section of the course (and because the exams can be randomized), I can give different exams every semester with very little prep time. I do still have some additional work left to polish the pools (I’d like to group and keyword them for adaptive testing in the future), but the time I’ll have to spend on assessment in the future has been greatly reduced. As I teach Hebrew Bible as well, I intend to do the same for that class and ultimately all the introductory courses I teach, effectively automating the bulk of assessment for my introductory courses. This should afford me more time to research and focus on the actual pedagogy in the classroom while also giving better data on student performance. Sometimes the move to computers really does make things smoother.
A few caviats: UNC requires students to have a notebook computer, meaning I could require students to bring a computer to class for these assessments (the quizzes were generally timed quizzes to be taken at home). At institutions where this is not the case, this approach would naturally be more difficult to execute. Learning disabled students also present a special problem in this approach, as separate exams with different timing requirements must typically be created for those students, and it’s a little bit of extra work to get those exams to feed into the right grade column if you use the online gradebook on Blackboard (Sakai’s online assessment and gradebook functions are still pretty limited as well, making this even more difficult on Sakai). Finally, the other potential pitfall is that if you don’t have access to a tool like LockDown Browser (Carolina does not have access, for example), students can potentially use Google or other online tools to cheat rather easily. That’s why I put a time limit on the at-home quizzes (but this is problematic given the advantage LD students have with double the time—typically plenty of time to cheat on these quizzes). In the classroom, I simply require that they keep their browser maximized and open to the test window, while I sit in the back of the classroom—any change of screen should thus stick out pretty clearly. It’s not a perfect system, but I think it’s at least preferable to the old pen-and-paper method.
This guest post is by Rev. Kevin DeYoung (@RevKevDeYoung), senior pastor at University Reformed Church (RCA) in East Lansing, Michigan, and author of Taking God at His Word: Why the Bible Is Knowable, Necessary, and Enough, and What That Means for You and Me (Crossway, 2014).
[See our blogpost, Taking God at His Word: An Interview with Kevin DeYoung]
If you enjoy being challenged by the test below, also try our quiz, Stark, Slytherin, Sauron, or Scripture? Identify These Quotes.
Several years ago our church started offering a leadership training course once or twice a year. The class began as a training seminary for those who had been nominated for elders or deacons. Now the course must be completed before a man can be considered for the office of elder or deacon.
At the end of the 12-week class there is a test. The exam contains a couple longer essays, short answers, and a series of questions testing basic Bible knowledge. We are careful to say that you don’t have to be a brilliant student to serve at University Reformed Church. There is much more to effective ministry than passing a written test. Much more. And yet, we do not want our staff, teachers, and officers to be biblically illiterate. Granted, people aren’t usually too excited about taking a test, but they are almost always glad to have taken the class. And more often than not, the test proves to be an edifying experience.
For fun, and for your own evaluation and learning, I thought I’d post the questions that comprised the knowledge portion of the exam. We have changed the exam in recent years, so these are not the exact questions we currently use. (I’m not going to show everyone the test ahead of time!) But this will give you an idea of the sort of Bible knowledge we want our lay leaders to have. No one gets every question right, but most people get most of them right, and a few erudite members have nearly aced the thing. I have not provided below the answers to test, because, well, it’s a test.
A. Who did the following?
1. Wrote the book of Acts?
2. Appeared with Jesus on the Mount of Transfiguration?
3. Directed the rebuilding of the wall of Jerusalem?
4. Killed a thousand Philistines with a donkey’s jawbone?
5. Led the Israelites into the promised land?
6. Was exiled to the island of Patmos where he wrote Revelation?
7. Was going to curse Israel, but had to bless them?
8. Became the first King of the 10 tribes that broke away?
9. Rescued David from her foolish husband Nabal?
10. Was rebuked by Paul for refusing to eat with Gentiles?
B. Where geographically did the following events take place?
11. God gave Moses the Ten Commandments?
12. A silversmith caused a riot?
13. Elijah had a confrontation with the prophets of Baal?
14. Believers were first called “Christians”?
15. The river Jesus was baptized in?
16. The walls of the city collapsed after the Israelites marched around it?
17. Jesus walked on water?
18. The place where Jonah was supposed to be going when he fled to Tarshish?
19. The place where Paul was heading when he was blinded on the road?
20. The river Ezekiel was at with the exiles when he received a vision from God?
C. In which book of the Bible do you find the following?
21. Peter visits Cornelius where he learns that God accepts Jews and Gentiles?
22. Paul asks a runaway slave to be welcomed back?
23. Israel worships a golden calf made by Aaron?
24. The story of Joseph and he brothers?
25. Twelve men explore the land of Canaan, but only two trust God to give it to them?
26. God’s judgment on Israel is pictured by a prophet as horde of locusts?
27. A description of the armor of God
28. The words “My God, my God, why have you forsaken me?” in the Old Testament?
29. A prophet marries a prostitute?
30. The Magi visiting the Christ child?
D. In which book and chapter(s) do you find the following?
31. God first speaks the Ten Commandments?
32. The call of Abram?
33. The Sermon on the Mount?
34. The Great Commission?
35. The Holy Spirit coming upon the disciples in wind, fire, and tongues?
36. Just as Adam was the head of the old humanity, Christ is the head of the new: “Just as through the disobedience of the one man the many were made sinners, so also through the obedience of the one man the many will be made righteous”?
37. ”But these three remain: faith, hope, and love. And the greatest of these is love”?
38. A religious leader hears “Unless a man is born again he cannot enter the kingdom of heaven”?
39. Satan bound for a thousand years?
40. The three Hebrews saved from the fiery furnace?
E. Give the main topic or event of the following Bible chapters
41. Genesis 3
42. Isaiah 53
43. Romans 4
44. Psalm 119
45. Hebrews 11
46. Acts 15
47. John 17
48. Revelation 21-22
49. Luke 15
50. Exodus 3
F. Who said the following?
51. If I perish, I perish.
52. What is truth?
53. After I am worn out and my master is old, will I now have this pleasure?…Will I really have a child, now that I am old?
54. O Jerusalem, Jerusalem…how often I have longed to gather your children together, as a hen gathers her chicks under her wings, but you were not willing.
55. Grant that one of these two sons of mine may sit at your right and the other at your left in the kingdom.
56. The fire and the wood are here, but where is the lamb for the burnt offering?
57. He must increase; I must decrease.
58. Am I dog that you come at me with sticks?
59. I know my Redeemer lives and that in the end he will stand upon the earth?
60. Give your servant a discerning heart to govern your people and to distinguish between right and wrong.
G. If you encountered the following error, to which book would you turn for help? Choose the book that best addresses the error. Use each of the listed books only once: Genesis, Job, Song of Songs, Amos, John, 1 Corinthians, Galatians, 1 Timothy, James, Revelation.
61. ”As long as you believe the right things, it doesn’t matter how you live your life.”
62. ”I’m sure I don’t have any spiritual gifts. Only special people do.”
63. ”We are saved by Jesus, but we also have to do our part by obeying the law of the Old Testament.”
64. ”If you are sick, you must have sin in your life. Good people don’t suffer.”
65. ”God doesn’t care about the poor and oppressed. That’s the social gospel.”
66. ”I know God promises to bless me, but I can’t really trust him through the hard things in life, like famine, barrenness, and imprisonment.”
67. ”In the end it won’t make any difference who we followed or what we did with our lives. Jesus will treat everybody the same when he comes back.”
68. ”There’s nothing special about Jesus. He’s just one way among many, just another prophet or good moral teacher.”
69. ”The best way to pick your elders is by looking at how successful they are in the business world. Next, consider how many degrees they have. After that, popularity matters most. Finally, if you still can’t decide, go by good looks.”
70. ”The Bible doesn’t say anything about intimacy between a man and a woman. That’s too fleshly for God to care about.”
H. Arrange the following events in proper chronological order.
a. The giving of the Law
b. The atoning death of Christ
c. Malachi prophesies
d. The promise to Abraham
e. creation and fall
g. Exile in Babylon
h. David is King over Israel
i. Paul is shipwrecked
j. The Judges rule over Israel
I. Match the verse with the doctrine it best supports. Each doctrine from the list will be used only once: providence, atonement, election, justification, immutability, sanctification, inspiration, deity of Christ, Trinity, total depravity
81. Just as he who called you is holy, so be holy in all you do. 1 Peter 1:15
82. God demonstrated his love for us in this, while we were still sinners, Christ died for us. Romans 5:8
83. What you meant for evil, God meant for good. Genesis 50:20
84. He chose us in him before the foundation of the world. Ephesians 1:4
85. I the Lord do not change. Malachi 3:6
86. Blessed is the man whose sin the Lord will not count against him. Romans 4:8
87. In the beginning was the Word, and the Word was with God, and the Word was God. John 1:1.
88. Go therefore and make disciples of all nations, baptizing them in the name of the Father and of the Son and of the Holy Spirit. Matthew 28:19
89. For prophecy never had its origin in the will of man, but men spoke from God as they were carried along by the Holy Spirit. 2 Peter 1:21
90. There is no one righteous, not even one; there is no one who understands, no one who seeks God. Romans 3:10-11
J. In which Old Testament book would you find the following Messianic prophecies? Books may be used more than once.
91. The Messiah would be born in Bethlehem.
92. He would crush the head of the serpent.
93. He would come riding on a colt, the foal of a donkey.
94. Born of a virgin.
95. Came to preach good news to the poor, bind up the brokenhearted, proclaim freedom for the captives, release the prisoners from darkness, proclaim the year of the Lord’s favor and comfort all who mourn.
96. Would be a priest forever in the order of Melchizedek.
97. Would be like a sun of righteousness rising with healing in its wings.
98. ”They divided my garments among them and cast lots for my clothing.”
99. Buried with the rich in his death.
100. Like a lion’s cub of the tribe of Judah.
This post originally appeared on Kevin DeYoung’s blog DeYoung, Restless, and Reformed.