A principal component of a sound risk management framework, particularly in highly regulated institutions, is an effective control testing program. What is control testing? One illustration is the story of a United States House of Representatives member from Missouri named Willard Vandiver who, at an 1899 Naval banquet in Philadelphia, stated “I come from a state that raises corn and cotton and cockleburs and Democrats, and frothy eloquence neither convinces nor satisfies me. I am from Missouri. You have got to show me.” While this is believed to have given rise to Missouri’s nickname as the “Show Me State,” it also capably represents the purpose of control testing: demonstrating that controls are designed to mitigate risk function as intended.
The control testing lifecycle typically consists of 5 steps: gathering evidence, validating evidence, analyzing evidence, assessing and concluding effectiveness and documentation or certification. This lifecycle occurs after a control is appropriately designed as part of the business process. Generally, many institutions implement manual controls, usually because the business process around generating the required evidence is manual. Thus, both the control execution and the testing process tend to be manual.
Why automate? There are three key reasons:
- Efficiency—If the control is automated, then less labor time is required to operate the control, produce the evidence, analyze the evidence and conclude effectiveness. The benefit of automation of the testing lifecycle goes beyond labor efficiency; it can mean additional labor capacity to execute a growing inventory of controls. This reduction in labor within the testing lifecycle can be repurposed into developing a continuous testing lifecycle for both control testers and control owners, resulting in earlier identification of control ineffectiveness and an earlier opportunity for remediation and response. This can lead to lower residual risk for the enterprise as well as self-governance.
- Risk reduction—Automated controls are considered more reliable and less error prone. A properly automated control can also increase coverage of technology assets, thereby providing a higher level of assurance that the control will operate effectively across the environment. For example, a manual control verifying proper user access for 200 applications would require sampling (i.e., 25 applications would be tested because 200 is too time consuming). Automation can increase this to full coverage, giving the tester the conclusive analysis for all applications. In addition, full coverage testing allows for a more comprehensive and aggregate risk assessment of the control failure. Rather than extrapolating the results of a sample with one failure, full population testing can show the exact number of times of noncompliance, providing the organization with a better understanding of the residual risk associated with the control failure.
- Effectiveness—Automation can introduce continuous monitoring, where a control is automatically tested for compliance periodically before formal periodic testing. If a control is noncompliant, an automatic message can be generated to the control owner who can fix this issue. This ensures 100% effectiveness when tested if properly executed, reducing overall compliance and enterprise risk while avoiding noncompliance costs and adverse regulatory actions.
The key to executing a successful automation agenda rests with the maturity of the risk management process, a clear understanding of the business process and related controls, and the availability of a system or systems that manage the controls and testing (i.e., workflow ticketing, centralized access management, risk management system). Assuming a reasonable level of maturity exists, along with technology systems supporting the given business processes, an organization should examine their control inventory to identify appropriate candidates for automation at the control level and test lifecycle level. The inventory can be used to identify potential automation candidates based on return on investment since there is an inherent investment in time or money required to achieve the desired efficiency.
Generally, several ideal technology domains are ripe for optimal gain (lowest investment, highest return). These include:
- Access management—User access tends to be one of the more complex and manual intensive areas, even for organizations with centralized systems supporting granting access. Moreover, the sheer volume of user requests, transfers and privileged access types means test sampling is required. Automation can eliminate sampling by providing increased coverage of the population.
- Change management—Similar to access management, change management tends to have multiple workflows with a centralized support system, lending itself to be a prime candidate for automation.
- Cybersecurity operations—Alert generation, data loss prevention monitoring and similar cybersecurity processes also have high volumes, centralized systems and large evidence populations that make automation a priority in this domain.
In addition to automation of the business process and supporting controls, data analytics tools can provide a testing team with powerful capabilities to reduce the test cycle time. A testing team can develop a workflow analytics capability in a short period, then apply it to large spreadsheets of evidence for automated analysis and validation with full coverage of the population. This is an ideal use case where the control itself is manual and cannot be automated (usually due to time or priority) but the testing team can achieve risk reduction and labor efficiency.
Although automation cannot address maturity or deficiencies in a risk management framework, it can increase the efficiency of control execution and testing, reduce risk and strengthen the overall effectiveness of the control environment. In short, it makes the “Show Me” aspect of technology control testing much more compelling and verifiable.
Editor’s note: For further insights on this topic, read Michael Powers’ recent Journal article, “A Technology Control Testing Automation Case Study,” ISACA Journal, volume 5, 2021.
ISACA Journal Turns 50 This Year! Celebrate with us—and don’t forget you can still receive the print copy by visiting your preference center and opting in!