In addition to reviewing general arrangements for IT, the IRM manager/auditor may be asked to look at the controls within a specific application system (e.g. payroll, sales, ERP). Consider a nice, simple system. We own a shop and want to know how much stock it contains so we request a stock take. How do we know that the stock level shown on the stock take is correct? It will be correct if:
• actual stock has been independently verified/counted.
• we include every line and item of stock.
• the prices shown for each line are accurate and realistic.
• the calculation of total values is correct.
• we only include those items we own (for example, not including items we had sold already).
• we are happy that all of the stock is legal.
Now consider if we owned 2,000 hypermarkets across the country and still wanted to know the stock levels – we are likely to use a computer system – but the same principles apply. We would expect the controls to be embedded within the computer system rather than relying on manual controls. But we still need the same assurance over:
These are very important for accurate financial reporting and are used in SOX and external auditing (often referred to as financial statement assertions). The same principles can, however, be applied to any computer system.
In this chapter we will explore application risks and the application controls we would expect to be built into systems to mitigate them, and how they can be tested.
Just because a system has not shown signs of error does not mean that it is fully reliable, neither does the idea that the system may have been thoroughly tested in the past, or a vague idea that manual controls may compensate. Problems may be hidden for years and not observed – for example I have helped to implement new systems and when we have migrated data from the previous system we have encountered inaccuracies – at first people blame the new system but it often becomes apparent that the data errors have existed in the data for some time and were not identified by the previous system.
The risks can be summarised as follows:
• Errors are introduced which cause mis-financial or other reporting.
• Failure to complete business transactions, causing additional cost and loss of income.
• Additional work and cost required to correct errors.
These errors or incidents will be down to:
• Incomplete processing – either missing records or missing vital information from within the record (e.g. customer address).
• Inaccurate data – this could be due to input errors (e.g. date formats incorrect), or due to calculation or other processing errors (e.g. calculation of a customer’s balance, expiring date of a contract).
• Transactions being performed by unauthorised, or untrained users.
To be able to rely on the performance of application controls we need to ensure that the general IT controls are adequate. This link between the two is often ignored. But if we are relying on a control built into the system it could become ineffective if unauthorised or untested changes were made to the program code. Application controls will also be less effective if there are super-users with access rights that enable them to avoid the normal control processes (e.g. for the bulk upload of data).
Some (in)famous bugs
‘First actual case of a bug being found’ – was actually a moth found in computer relays at Harvard University in 1947.
In 2005 a famous store in the UK offered widescreen TVs for 49 pence, instead of over £350. They sold 10,000 before anyone noticed the mistake.
In the early days of computing, space to store data was at a premium. To save storage space, dates were often abbreviated so ‘1960’ would be stored as ‘60’. This was OK until the year 2000 when the calendar clicked over – so how would we know whether ‘01’ was meant to be 1901 or 2001? This caused a massive scare, with many organisations having to re-write, or in many cases, replace old software. As it turned out, the impact of the bug was very minimal. Watch out for the 2038 bug!
There are two main types of application controls:
1. Preventative – stop errors or other irregularities being introduced.
2. Detective – discover and report errors or other irregularities after they have been introduced.
Most systems will include a mix of the two – especially for high risk areas, such as controlling access (see below). Preventative controls are generally more efficient and effective as they reduce the amount of work to correct data. Detective controls are preferred by some auditors (who like something they can ‘tick’) and can also be useful as an indication of the integrity of the system. For example, following a system change a detective exception report may indicate a large number of error conditions if the change has impacted a preventative control – say, for example, a control to ensure the format of a date is in a certain range. Detective controls are also useful to ensure that the preventative controls have not been circumvented. For example, data may have been entered by a different route.
Preventative controls are often seen as being ‘better’ than detective controls because they stop things from going wrong. However, other controls that rely on manual review, authorisation, or approval, may also stop a process and can make that process efficient. The advent of ‘big data’ – making powerful analytics possible, may tip the balance more in favour of detective controls, reducing the level of approval but using real-time analytics to flag up potential mistakes for manual review. The auditor will always need to apply judgment to consider:
• What is this control trying to do?
• Is it designed to achieve it?
• What impact will this have on the efficiency and effectiveness of the business process?
• What is the alternative?
Most modern packages, including the large ERP systems, contain preventative control facilities – these do, however, need to be configured and activated. These include:
• Data input controls – for each field input can check and either warn the user, or prevent further processing until the error has been corrected:
Ensuring data input is authorised
Ensuring data is entered for key information (mandatory fields)
Format of data (e.g. dates, numbers, alphanumeric)
Range the data lies in/reasonableness of data/sequence checking
Prevention of duplicate data entry
Relationship to other records
Attempt to delete a record or field.
• Processing controls:
Controls similar to data input controls (see above)
Cross checking of data between different tables
Routing transactions via a pre-defined workflow for authorisation.
• Access controls to identify and assess users before they are given access to systems and resources.
These controls can be applied to all data – both ‘master data’ (such as customer address details, record types, etc.) and transactional data (e.g. orders being placed).
Detective controls usually involve some form of reporting – either by creating a paper or soft report (e.g. in MS Excel format) or by sending an email or SMS alert. Examples include:
• Exception reports or dashboards – to show when an error has been created so that it can be corrected. Reports or dashboards to confirm that processes have completed satisfactorily (e.g. by providing totals and balances).
• Reports for reconciliation and other manual controls (e.g. stock checks).
• Audit trails–to show details of changes made to records, batch processing runs, etc.
• Embedded audit modules/automated testing (e.g. synthetic tests).
Controls over user access
Access to data and system functionality should be restricted, based on the level of least access, depending on the user’s legitimate needs to access the system. Access can be restricted:
• to functions, menus, screens, or even individual fields.
• geographically or by other structural distinctions (e.g. by operating branch).
• to ensure users do not exceed their delegated authorities to perform transactions (e.g. that they can only make refunds up to a certain value).
Delegation of authorities ensures that people can only perform transactions up to their designated limits. For example, you go into a car showroom and want to purchase a car. When you ask for a bigger discount, the salesperson may need to get it authorised by the sales manager first.
Segregation of duties and delegation of authorities are areas of great interest to auditors, especially in organisations impacted by Sarbanes-Oxley or other compliance frameworks. One way to do this is for people to do more, rather than having to refer the issue to someone else for completion. Segregation of duties reduces the risk of error or fraud by ensuring that, for example, the person creating a transaction cannot also authorise it. Another example may be that you make a purchase in a shop and the cashier mistakenly enters £500 rather than £5. To correct the transaction, a supervisor may be required to use a special transaction. In this case the transactions of cash sales and refunds have been segregated. In some cases, management may wish to accept the risk, or there could be other compensating controls (in the example given the supervisor could review a report of refunds made at the end of each day).
An analogy for segregation of duties:
Farmer Georgina has a problem. She has come to a river and only has a small boat – she can only carry one item at a time. With her she has corn, a fox and chicken. The fox and chicken cannot be left alone, nor can the chicken and the corn. Georgina needs to cross the river with all three. How can she organise this?
One way to represent the problem would be a table that shows what can and what cannot be left together on one bank of the river and this could be used as a basis to solve the problem:
The same technique can be used to compare the actions of roles, transactions, or business activities and can be used for large numbers of variants.
Now consider Georgina’s problem again. The main issue is the chicken, as it cannot be left with either the fox or the corn. If we were to put the chicken in a fox proof cage, away from the corn, we would no longer have a problem – we have mitigated the risk. The problem can be once again solved – and it will take fewer trips for Georgina to get all three across the river.
We have created a mitigating control – we could do the same for the purchasing example by building a workflow to prevent the user authorising transactions they have created – they would then be able to both create transactions and authorise those created by other users.
A review of application controls will consider:
• What are the specific risks for this application and its main data, in terms of financial assertions, regulatory and compliance risk?
• Documentation of preventative and detective controls in system documentation and process flows.
• An assessment of whether these controls would mitigate the risks identified if they operate as designed (‘design effectiveness’).
• Evidence of testing performed to confirm the controls operate as designed.
• Re-performance of testing – to confirm that controls operate in the production environment. This is often referred to as test of operating or implementation effectiveness. This could be re-testing in production under very controlled circumstances, or could be other evidence, such as version control, to ensure production and test environments are consistent.
Application controls ensure that systems perform as required and give valid outputs and results. They can be more effective than manual controls – once set up the computer will always perform the control in a consistent way until the system is changed. The main risks to consider are the financial statement assertion (completeness and accuracy) and also compliance and regulatory control.