The SaaS accounting questions you need to ask

In my Fired! It had to happen post, Alastair commented:

So what’s a decent SAAS offering, and how do you evaluate the stuff you can’t see, like security, availability, speed, respect for your data? There are lots of differences, which your client won’t have the first clue about. And that’s not to say non-SAAS offerings are not similarly riddled with problems you can’t see. Accountants need to savvy up if they want to work in this space.

It’s a great observation. These are issues for which every software vendor should have demonstrable answers:

Security

This is an old chestnut that gets tossed out on pretty much every call. SaaS vendors often go to great lengths to describe how your ’stuff’ is encrypted in flight and held in some sort of bomb proof bunker only accessible by someone with a magic key and password. All sounds great but security is much more than that to accountants. I’ve raked over (and been raked over) on the issue of access control but this is something every accountant I’ve ever spoken with on the topic wants an answer to. Here’s why. Many years ago I heard about a 20+ year time served senior who was fired for falsifying the date on a tax return. He’d become so trusted that no-one bothered to question what he did and he was usually good at his job. It’s an easy mistake to make. But he screwed up and got caught out on something that could have cost the client a LOT of money and then tried to cover it up. If you’re providing open access to an accounts program then what mayhem might someone wreak?  So I’d go further still. I’d ask questions like:

1. Can I get a snapshot of the data at a point in time? How?
2. Can I delete any entries?
3. Is there an audit trail that logs all actions? If so then what does it look like?
4. Can I alter any entries without it appearing in an audit trail?
5. Is the app PCI compliant?  For more on this check Phil Wainewright

Availability

As my good pal Vinnie Mirchandani often says, we somehow hold SaaS apps to a higher standard than we would do an on-premise application. Why is that? It seems we expect there to be statements around 24×7 availability when really we need 9 to 5 most of the time with occasional access at other times. SaaS applications often have a higher level of availability than on-premise because the data centres are hosting for many customers with unpredictable operating requirements. Even so, I’d still want to know what the reported server uptime is and see some evidence of that. Whatever you do, don’t be fooled by the mention of a brand you know. Even Google has unexpected downtime. Ask also about planned downtime – how frequent and what notice will you get.

Speed

Performance is often a subjective issue but it is true to say that some apps run faster than others. The other week I remarked that NetSuite seemed to be running relatively slowly until it was made clear that the company was demonstrating it on an Edge network which is inherently slower than dedicated broadband. It makes sense then to run tests on the network you intend operating the application over, noting any tweak requirements for the browsers you will be using.

Respect for your data

There are a range of issues here, the first being about security but the second which is about what the vendor (and possibly third parties) can do with your data. Few ever read terms of service but that’s essential if you are to understand what the vendor is committing you and them to.

1. I mentioned in my Sage Billing Boss piece that Sage’s ToS are not reflective of what you’d expect to see but relate to more general website usage. If you see something like this then insist on a contract that reflects ToS with which you can agree. Update: Sage has fixed this issue. It’s worth clicking the BillingBoss ToS link (PDF) to see how they’ve updated.
2. The vendor might talk about regular backups but what does that mean? Once a day, every hour, every 15 minutes? Does the vendor have a plan to cover backup as they grow? You might find that you have a responsibility to make backups. That’s the vendor’s get out of jail free card if the data center or server goes down. Is that acceptable to you and if so then how are you going to implement a backup policy that works?
3. One long term potential revenue stream comes from aggregating data that is then anonymized and used for benchmarking purposes. It’s a good idea but how can you be certain that the people doing the aggregation can be trusted, especially if it is a 3rd party solution?
4. When a vendor says it has an open Web API, do you understand what this means? Has the vendor ring fenced aspects of the service that help keep data secure? How does the vendor manage those who have access to the API? Is there evidence of how they police the API against improper use? What does the API cover – some functionality? All? Something in between?

More generally – has the application service been subject to a SAS70 audit review. I”m betting that very few vendors have looked at this issue but it is vitally important in providing customers with peace of mind that the service has been independently assessed for control issues. If the vendor says yes then:

1. See and read a copy of the report
2. Check actions against any adverse comments
3. Independently verify that the report is the latest to hand
4. Ask how frequently the system is assessed
5. Ensure the vendor commits to providing you with SAS70 report updates

The SaaS market is not mature. While applications are being developed at a brisk pace and making market penetration strides, as Alastair implies, there is no reason why those services should not be subject to scrutiny beyond an assessment of features and functions. Vendors who are up front about these issues are more likely to be following practices with which professionals can feel comfortable.

This check list is only scratching the surface but it should give you a starting point from which to build up a standard questionnaire.

UPDATE: please note Brandan’s comment about SAS70 Type I/II – I should have made the distinction in the main post.

Tagged as: Access control, Backup, Cloud computing, Data center, Security, Statement on Auditing Standards No. 70: Service Organizations, Terms of Service