CompTIA Provides Info about Performance-Based Testing

In August, 2012, CompTIA announced that it would be adding performance-based questions to the mix of question types it presents to exam candidates, starting with the A+ exam (for more information on this announcement, see my August 20 blog on this very topic). On September 6, CompTIA posted a 16-minute video entitled CompTIA Exams: The Candidate Experience on YouTube. In its introduction, the narrator indicates that this material was posted to address questions from prospective CompTIA exam candidates, with a strong emphasis on its use of performance-based questions.

New Question Types Coming to A+, Network+, and Security+

The CompTIA Advanced Security Practitioner (CASP) exam, released in 2011, was CompTIA’s first-ever exam to incorporate performance-based questions. Even so, CompTIA has kept pretty mum about the types of such questions it uses on exams, how simulations, images, and other exhibits figure into those exams, and what kinds of underlying technology might be at work. Although the YouTube video does not address exam content except through a series of deliberately trivial examples, it is still worth watching to give CompTIA exam candidates a good idea of what kinds of questions will be involved, how the interface works, and what to expect in a VUE testing center when taking any such CompTIA exam. The video also provides some useful tips and tricks for managing the exam experience, especially as regards time and question management, so I’ll summarize and comment on that information here as well.

Performance-based Questions to the Fore!

In addition to the multiple choice/single answer, and multiple choice/multiple answer questions that CompTIA has used from time immemorial on its exams, candidates can expect to see fill-in-the-blank questions as well as exhibit and simulator-driven questions on upcoming versions of the A+, Security+, and Network+ exams. In fact, CompTIA plans to release the A+ 801 and 802 exams, which include performance-based questions, for general public release in September 2012.

The fill-in-the-blank items present readers with a question that requires one word or a short phrase to answer, followed by a text box into which the candidate must enter his or her response. Figure 1 hearkens back to the old nursery rhyme about Jack and Jill to pose an incredibly obvious example question.

And now, a Word About CompTIA’s Partners...

One of the most interesting aspects of the video appears in what you’ll have to call the “credits” that appear on-screen about a minute-and-a-half into the running content. This is depicted in Figure 2, and specifically mentions ATA, Inc. (a Chinese services provider for testing, assessment and related services, and the company that provided the Windows XP simulator shown in the video’s single simulation-based example) and Pearson VUE (an American-based operator of testing centers worldwide, and the exclusive outlet for CompTIA certification exams).

As the narrator explains on the video’s voiceover, VUE provided the test engine and a detailed look at the GUI candidates will encounter when they head to a testing center to sit for any CompTIA exam. ATA provided the simulator technology used to drive a Windows XP environment for answering a performance-based question in the video. I’m also guessing that they provide the animation and display capability for the animated network traffic flow used as an exhibit in a subsequent example question later on in the video as well.

Simulators Can Be Enticing, But...

Any tech-head serious enough to understand what it takes to put on an OS simulation in general, and a Windows OS simulation in particular, may be tempted to play around with the environment just to see what it can and can’t do. That’s undoubtedly why the narrator warns exam-takers to stick at the tasks at hand in the video, and shows them that unrelated actions or commands will probably not work in any simulator intended to plumb an exam candidate’s knowledge in some specific area of OS form or function.

Caveats aside, the Windows XP GUI that the simulator presents within the test engine is pretty neat, as Figure 3 will show, with the black-and-white cmd.exe window at the left, and the question candidates have been asked to answer on the right.

The question reads “Please navigate to ‘My documents’ folder, and then configure MyProfile.txt as read-only using the appropriate commands.” The video then takes viewers through a change directory (cd)maneuver to enter My Documents, and then uses the command-string attrib +R MyProfile.txt to make the necessary permissions change to the indicated file there. The video also shows how to use the VUE test engine GUI when running a simulation, and makes the important observation that the exam timer becomes invisible whenever candidates enter a simulation (it stays visible when accessing images or videos for exhibits, however). This means candidates must watch and manage time for themselves while running any simulations—a point well worth remembering for any exam that is bound to include at least a handful of such items. They can also click a variety of buttons to manage a simulation window while it is active as well, as shown in Figure 4.

Users must click Done to close the simulation window and move onto the next question. If they click the checkbox next to Incomplete, they can revisit a simulation question later (much like using the Flag for Review option available for all questions in the VUE test engine). To delete any and all inputs or answers to the simulation, users should click the Reset button. Clicking Help will produce a help window to explain how the interface and its buttons and controls work.

Other types of interactive questions include a “drag and drop” type where candidates are asked to move on-screen elements to categorize or identify them, as shown in Figure 5. There, candidates must distinguish between audio and mouse cable connectors.

There’s also an exhibit-driven question type that permits exam-takers to view a diagram, image, or video to answer related questions. The example provided in the video animates a packet transmission from the Internet, through a switch port, to an end node to pose a question about the path the packet takes, as shown in Figure 6.

And again, while the video shares no information about the kind of exam content that candidates will see when they sit for a CompTIA exam that includes performance-based questions, it does provide useful and illustrative information about what the interfaces look like, how they behave, and how best to use them. That’s why I recommend that anyone who’s not already familiar with these new CompTIA question types should sit through the video (you can skip ahead to the 5:00 mark if you want to jump past the information about the NDA and other official mumbo-jumbo).

Other Hints on Exam Strategy from CompTIA

I got some other very useful reminders about CompTIA exams from the video as well, which I’d like to share. First, exams run 90 minutes, but candidates can take up to 30 minutes at their seats in the exam center before actually starting the exam itself. CompTIA gives you this time to read their NDA, but you can use it for other things as well. I strongly urge candidates practice memorizing and writing down important pieces of information (the seven layers of the OSI Reference Model and their meaning and significance is an excellent example) as a preamble to taking practice tests. Then, when you sit down at a VUE testing center to take the real thing, you can start by writing all that stuff down on the single sheet of paper candidates are allowed to take along with them to their exam station. Later, you can refer to your cheat sheet instead of having to remember technical details while also puzzling your way through exam questions.

Candidates should also make careful and judicious use of the Incomplete and Flag for Review elements on exam questions. Don’t mark a simulation as incomplete unless you want to return to it later, if time permits. Likewise, don’t flag too many questions for review, either. Mark those you think you might be able to work your way through if time permits, but take the hit on those about which you’re clueless. 100 questions in 90 minutes doesn’t leave you much time for circling back, unless you can rip through the easy questions with incredible dispatch. And of course, this is yet another good reason to practice, practice, practice with practice exams ahead of time—simply so you can get used to dealing with the questions and handling the volume of text and other material you must work through in the hour-and-a-half allotted for your exam completion.

When the exam is over, you’ll get a score report on-screen. You can—and always should—request and obtain a printed copy of this report before you leave the testing center. If you need to retake the exam, the report will offer pointers to help you prepare for your next try. And if you pass, CompTIA recommends that you keep the score report around as a printed record of your exam activity, and follow its instructions to obtain a printed certificate and to make sure your exam results are properly recorded.