Editor’s Note: This is the second story in our three-part series on the new GMAT, which makes its official debut on June 5. In this article, we examine the development of the new Integrated Reasoning section.
The mandate was clear. More than 700 business faculty had evaluated the Graduate Management Admission Test (GMAT) and determined that it was time for a dramatic makeover. After almost 60 years, the GMAT needed to test students’ higher-order reasoning, critical thinking, and data analysis skills. To accomplish this ambitious task, which would require a new, 30-minute “integrated reasoning” section, the Graduate Management Admission Council, which administers the GMAT, turned to test developers from ACT, the nonprofit organization that develops the questions for the business school entrance exam.
The task ahead was a formidable one, said Edward Covey, a senior test development associate at ACT who writes questions for the GMAT. “Coming up with a new test and new format, while trying to find effective ways to test other skills that weren’t being tested, was especially challenging,” he said. “It was a long process of invention and experimentation.”
The first step was determining which types of questions and formats would work best in the new section. The “geography of the computer screen” allowed test developers to experiment with ways to incorporate charts, graphics, and tables into the problems, Covey said. The writers came up with 15 different exam question types, and did a soft pilot with students from the University of Iowa’s Tippie College of Business. The students helped them gauge which questions worked, and if they were accurately testing skills MBA students used in the classroom, said Wayne Patience, ACT’s assistant vice president of higher education.
The Tippie MBAs who took part in the pilot offered their thoughts on the questions in a video filmed by ACT. Several students said they liked that they could interact with graphs, move text around, and manipulate data in tables. One Tippie student interviewed in the video said he liked that the question didn’t make test-takers do a thorough analysis, but rather “get in and dig deep but do it quickly, one of the skills we use in business school.”
That was the type of feedback Patience was looking for. “You want the examinees to be engaged and feel that the exam is relevant and pertinent for them, and is not just some esoteric exercise,” he said.
Not all of the new question types hit home, though. Initially, 15 were under consideration. That was whittled down to 10 in the first pilot, and eventually to four by the end of a second pilot, which administered the new section to actual GMAT test-takers, said GMAC’s Ashok Sarathy, vice president of the GMAT program. Some were eliminated either because they proved confusing to test-takers, took too long to complete, or had an inherent cultural bias that would put some students at a disadvantage, Sarathy said. For example, a planned audio section was cut because test developers worried the question was testing examinees’ ability to understand spoken English, rather than their listening comprehension skills, as intended, Sarathy said.
By the end of the pilot process, ACT had come up with the four types of questions that students taking the exam will encounter this June: graphics interpretation, multi-source reasoning, table analysis, and two-part analysis.
That was the start of the fun for the test developers, who got to work developing the dozens of questions for the exam that will appear on the GMAT when the new test launches on June 5. Each question goes through a long and arduous testing and evaluation process by ACT before it makes it to the actual test, Patience said. It takes ACT about six months to develop each question, with as many as a dozen people taking part in the process, he said. Often the writers pull questions from events in the news, or simply “make up things,” Covey said.
However, the writers work hard to ensure that the questions proposed are scenarios that could actually happen in the business world, Covey said. In one of the multi-source reasoning sample questions, test-takers have to read through company e-mails discussing a group of doctors invited to participate in a survey, and infer from those e-mails—all sent by different people at the company and containing various data points—whether certain statements are true or false. For example, the test-taker must determine whether the project coordinator for the survey expects to meet his goal for the number of completed surveys, or if the project’s administrator is willing to exceed the budget he set aside for the task.
“That particular question we just made up out of thin air, but it serves a purpose,” Covey said. “It’s very important for business students to think about what you need to give up in the interest of other things you want to maximize. There’s always a trade-off.”
Coming Next Week: In the third and final part of the series, Bloomberg Businessweek looks at how GMAC and test prep companies have been preparing students to take the exam.