{"id":12630,"date":"2019-06-24T08:23:41","date_gmt":"2019-06-24T12:23:41","guid":{"rendered":"https:\/\/news.elearninginside.com\/?p=12630"},"modified":"2020-09-08T09:14:50","modified_gmt":"2020-09-08T13:14:50","slug":"surgent-boosted-their-cpa-candidates-exam-pass-rate-by-38-with-adaptive-personalized-learning","status":"publish","type":"post","link":"https:\/\/news.elearninginside.com\/surgent-boosted-their-cpa-candidates-exam-pass-rate-by-38-with-adaptive-personalized-learning\/","title":{"rendered":"Surgent Boosted Their CPA Candidates\u2019 Exam Pass Rate by 38% with Adaptive Personalized Learning"},"content":{"rendered":"

Many companies today make claims about how their adaptive algorithms<\/a> personalize the learning experience. Very often, these claims come from the marketing department, rather than the research team. One company, however, has quietly developed some of the most advanced personalized learning<\/a> courseware currently available. What\u2019s more, it has the results to back up their claims. <\/span><\/p>\n

Surgent CPA Review<\/a>, which was founded in 2014 by Liz Kolar (CPA, CGMA) and Jack Surgent (CPA and founder of Surgent CPE), says it can determine, within four points, what a CPA candidate will score on the Uniform Certified Public Accountant (CPA) Exam, before he or she ever walks into a testing center. \u201cAnd it\u2019s important to note that those four points are in the student\u2019s favor, on average\u201d Kolar said. \u201cMeaning that, if a student is scoring a 75 in our course, their score on the actual exam tends to be a 79.\u201d<\/span><\/p>\n

Accurately predicting a candidate\u2019s exam score, of course, is not as important as getting that candidate to pass. On this front, the company has achieved significant results. On a national average, roughly half of the CPA candidates who take the exam fail. Among those who train with Surgent, however, 88% pass. <\/span><\/p>\n

Inspired by a Shift in Student Learning Styles<\/b><\/h2>\n

Kolar has been preparing CPA candidates for the exam for over 30 years. She\u2019s taught at the undergraduate and graduate level at Pace University, Seton Hall University, and currently doubles as a professor at Delaware Valley University. It was during these many years of direct student interaction that she began to realize that CPA Exam preparation needed to evolve.<\/span><\/p>\n

\u201cBack around 2007 or 2008, I noticed a change in my students,\u201d Kolar said. \u201cI was teaching four nights a week, four hours a night. I was using primarily a lecture approach to prepare my students, and I saw a shift in their ability to attend class. Their employers were placing greater demands on their work schedules. They wanted the course to work for them on their time. Also, their attention spans had shortened and they were looking for more of a customized approach.<\/span><\/p>\n

Kolar took her course online and broke most of her lectures into short five-to-ten minute videos.<\/span><\/p>\n

\u201cIt was quite amazing\u2014about 70% of my students left the classroom and chose to take my course online,\u201d she said. \u201cSo I knew I had struck gold.\u201d<\/span><\/p>\n

Noticing the success of Kolar\u2019s online course, Surgent approached her with an interest to partner. They agreed that it would be ideal to incorporate an AI component in their courses. So in 2013, the two pooled their resources and set about designing the courseware. <\/span><\/p>\n

Designing Adaptive Courseware for an Elusive Exam <\/b><\/h2>\n

Developing the adaptive algorithm was not a simple project, considering the structure of the CPA Exam. It\u2019s broken into four sections, and candidate scores are compiled on a statistical scale ranging from 0 to 99. Candidates must clear a score of 75 to pass. Each of the four sections is comprised of multiple choice questions and simulations, which ask candidates to respond to hypothetical situations and tests them on a range of issues. (The Business Environment and Concepts section also contains a written component.)<\/span><\/p>\n

The multiple choice section doesn\u2019t present too many issues when it comes to designing adaptive courseware in the sense that each question is objectively correct or incorrect. The simulations, however, are more tricky. They involve a higher degree of complexity, and the American Institute of Certified Public Accountants (AICPA) is tightlipped about their process. <\/span><\/p>\n

\u201cThe AICPA will not tell us exactly how they score the simulations,\u201d Kolar said. \u201cThey have given some data to us. Think of it this way: there is a table that has two columns and ten answers, so 20 answers in total. Each cell in the answer has value. A student can receive partial credit. So there\u2019s one row and two answers a student has to give. They could get one wrong and one right, and they\u2019ll receive partial credit.\u201d<\/span><\/p>\n

The Surgent developers, in other words, were put in a difficult situation. They had to design AI to replicate a process that they themselves did not fully understand. Luckily, they had some solid HI (human intelligence) to lean on. <\/span><\/p>\n

A Change in the CPA Exam Prep Landscape<\/b><\/h2>\n

The team combined the \u201cnano\u201d content that Kolar had begun testing a few years earlier with a new adaptive algorithm and introduced the resulting product as the first adaptive CPA Exam review course to market. The company has continued to focus on refining its technology since, implementing features like continuous individual assessment and a candidate score predictor, which they call ReadySCORE.<\/span><\/p>\n

\u201cSurgent\u2019s A.S.A.P. Technology is built on the mantra that no two candidates are the same. To us, achieving maximum efficiency and success requires treating candidates as unique, three-dimensional people, rather than static stencils,\u201d said Phil Salgado, Surgent VP of Product Management and User Experience (UX). \u201cTo accomplish this objective, we created a proprietary ReadySCORE, which is a real-time approximation of the candidate\u2019s score on the exam, as if they were testing today.\u201d<\/span><\/p>\n

\u201cOur software is designed to utilize the ReadySCORE, not just as helpful feedback to the candidate, but also to build personalized study sessions comprised of multiple choice questions, simulations, videos, and readings from our database. All of these target the candidate\u2019s weaknesses and reinforce their strengths.\u201d<\/span><\/p>\n

\"A
A student\u2019s baseline ReadySCORE is accompanied by a detailed diagnostic report which outlines unique strengths and knowledge gaps, making it an extremely beneficial tool for a student\u2019s study plan.<\/figcaption><\/figure>\n

By comparison, other CPA Exam preparation courses on the market at the time focused on a \u201clinear\u201d learning model that is very similar to the type of in-person classroom-based courses that had been popular for decades. These types of courses guide students through the course\u2019s lectures, books, and practice questions, chapter-by-chapter, in order, until the student reaches the end of the course. Many of these courses now also market an adaptive learning element.<\/span><\/p>\n

\u201cOur competitors tout their adaptiveness, but we\u2019ve looked at them closely. Their software is really just designed to guide candidates down fairly defined paths based on user personas, rather than being uniquely tailored to each user,\u201d said Salgado . \u201cFurther, because these user personas are created via a backward or historical look at previous user paths, there are limits to their predictive capabilities as the CPA Exam evolves over time (i.e. whenever the exam changes, these courses will be far less reliable until they have sufficient data to adjust the personas). Additionally, once users have been assigned personas, they do not change, so these courses fail to account for candidate growth factors like area of employment, time since graduation, and learning efficiencies.\u201d<\/span><\/p>\n

Focusing Development on Saving Study Time<\/b><\/h2>\n

The company has continued to invest in the development of their AI, using user experience and performance data pulled from its LMS to guide development decisions. One such recent development has been a difficulty progression feature. It allows candidates to choose various levels of difficulty as they continue through their training, in essence, choosing how aggressively they want to push themselves to achieve exam-readiness.<\/span><\/p>\n

\u201cOur data shows us that students selecting the most aggressive path are completing the course 50% faster compared to those choosing a less difficult path,\u201d Salgado said. \u201cIn fact, they\u2019re completing the entire CPA program, while achieving the same results, in the time it would take to complete a single section in less adaptive courses.\u201d<\/span><\/p>\n

Regarding their impressive 88% pass rate, Salgado is quick to point out that Surgent doesn\u2019t cook their data. \u201cSurgent surveys all users as soon as they pass their exam date to gather their actual score, not simply whether they pass or not. We compare their actual scores to their ReadySCORE at the time they sat. That\u2019s how we know that we\u2019re accurately predicting their real exam scores,\u201d he said. \u201cFurther, all users are included in our pass rate calculation. To date, we have received score reports from approximately 20% of our user population, which is a statistically significant sample size. The pass rates have been remarkably consistent across testing windows over the years, despite our continued growth.\u201d<\/span><\/p>\n