Welcome to this Questionmark podcast. Questionmark podcasts bring you news, ideas, and advice about assessments and learning.

Joan Phaup, Questionmark: This is Joan Phaup from Questionmark, and I’m speaking today with Rich Edwards from Kirkwood Community College in Iowa. I want to hear from you about your placement test that you’ve created for science classes. A lot of people are interested in this, and I was interested to find out how that’s benefited Kirkwood Community College to have a test like that.

Rich Edwards, Kirkwood Community College: Well, to specific, Joan, this was a placement test we created to place students either in a basic biological concepts class or the higher level college biology. We started this project about three years ago, and prior to that students could register for college biology even though they did not perform well on a paper and pencil test that was given the first week of classes. And the faculty would tell students based on the results of that paper and pencil test that your chances of success in this college biology course are not very good. Well, knowing students the way we know students, students decided to stay in the course and consequently about 45% of them failed the course or withdrew. So the biology faculty decided we needed to do something about this. So they embarked on a project to create a perception placement test in biology courses. And we developed a test. The faculty wrote questions. We field tested the items. We used the perception item statistics, question statistics, and test reliability reports to validate the test. And finally last fall, we put the test into production. So students now cannot register for the higher level more advanced college biology course until they obtain the cut score on the perception test. We are getting good results out of the placement test. The biology faculty are thrilled that now they have students in their courses that are prepared for the course, for the rigor of the course. They don’t have to teach background knowledge, basic knowledge just at the beginning of the course anymore. And students who are in the course based on their perception placement test are more successful. And this is probably a pretty common situation in a lot of colleges where you don’t have mandatory placement. And students often self-select and register in courses that they’re really not prepared for. So we want to use this model to try to expand it at Kirkwood to other disciplines so that students, even though they’re admitted to the community college on an open door admission policy, are placed in the right courses, placed in the courses that they’re ready for.

Joan: If I were in an organization that wanted to have a placement test, what advice would you give me about how to go about setting that up?

Rich: You can look at the powerpoint on the Perception website from the users’ conference in Memphis. That powerpoint gives you kind of the, a blueprint, a step by step process to go through. But briefly, you need to form a project team, assign responsibilities. You need to have a faculty question rider, so subject matter experts write the questions. And then you need to field test the questions to gather question statistics. The analysis of those question statistics are really important for the validity of the test, so that you have a test that has questions that aren’t too difficult or too easy, that have the right discrimination level, and so that the test has good reliability. So you need to field test the questions, gather the data, analyze the data, reject questions that aren’t performing to the criteria, and administer the test and then gather data on how students do in those courses so that you can set a cut score. So if a student attains a particular score on a test, the criteria we use is that they have a 50% chance or better of earning a C in that particular course. So coming up with a plan to implement a project like this is very important. It’s important to involve all the stakeholders—students, parents, advisers, test proctors, administration deans—to get input from all the stakeholders and buy-in on a placement project like this, because it has implications for students in terms of the curriculum and what courses they can take and what they can’t take. And you need to have buy-in from faculty. So having a carefully laid out plan, assigning roles and responsibilities, and carefully field testing the questions so you’ve got questions that are performing well are important to the success of a project like this.

Joan: How did the students respond to having to take the placement test?

Rich: Well, we sent a letter to all new students and their parents, and we sent an email to all new students informing them of the new placement policy in biology courses. And by and large, students, you know, had no problems taking the test. We did give them an opportunity to retest if they did not obtain the test score. So we had two forms of the test, form A and form B, and what we’re finding is that most students who did not obtain the cut score in form A pretty much accepted the fact that they needed to register for the low level course, understanding that they needed this prerequisite knowledge, they needed this prerequisite course in order to be successful in the higher order(?6:02) course.

Joan: And they’ve accepted that. It’s become part of a deal.

Rich: Yeah. It’s become part of the placement and assessment culture. Students can register online at our college, and if they don’t have the perception cut score, because we import those perception scores into our student information system so if a student tries to register for the high level biology course, they’re blocked from doing so. And they receive a message that they don’t have the cut score, or that they have not taken the test and that they need to go to the test center to take the biology readiness exam.

Joan: Could you touch a little bit on any pitfalls that you would warn people against about embarking on something like this?

Rich: Well, it’s, depending on your organization, you have to have access to a appropriate sized sample. And one of the challenges in college is that we really can only test students at the beginning of the semester. Hence it took us about two years of planning and field testing of questions before we came up with a test that was ready to put into production. If you have an organization where you can get a large enough sample to test your questions and look at the reliability of the question statistics, you can probably shorten the time in production. But that’s, again, it’s part of the whole planning process, because it’s critical that you field test your questions, you analyze the question and item statistics, because if it’s a placement test it’s a relatively high stakes exam. So you’ve got to have questions in a test that is in some cases legally defensible. So that if you have that data, then you have a valid placement test.

Joan: Do you see placement tests like this becoming more commonplace at your college?

Rich: Yeah. We use Perception at Kirkwood for admissions to programs. For example, it’s used for admissions to the veterinary technician program, to the nursing program, and some other health programs. That’s similar to placement testing, but it’s an admissions test. This was the first placement test that we really developed that we think has fairly decent validity, and we hope to use this model and expand it to other disciplines where faculty feel a need to, you know, have students placed in the appropriate courses. (Be)cause again, we’re an open door community college. Students can register, and often they get in courses that they’re really not ready for. And that’s really, presents a problem for retention. It presents a problem for student success, and it presents a problem for faculty. So where disciplines have courses where it makes sense to assess students’ prior knowledge and skills and place them in the appropriate level course, that’s where we want to try to expand this placement model.

Joan: Thanks very much. That was very interesting.

Rich: You’re welcome.

Joan: Nice talking to you.