A Debate on Warranty for Software

Last updated on 28th March 2013

The post, The Without Warranty Wild West Software Industry, whose contents I mailed out to some correspondents, resulted in a few of them responding with the problems that warranty for software has, and some related views of theirs. I have given some of the responses below, and then my response to them.

Correspondent1 (Corr1) is 'someone with decades of experience in academia and industry'. He wrote:

This is an area I have worked in for decades so it is of great interest.

I think the basic problem you are ignoring is that most software is developed to meet imprecise and incomplete requirements and specifications. I can give you numerous cases where the specifications failed to define all the possible cases that some software needs to cover and left it open to the implementer to decide what to do. There are other cases where assumptions are made but not stated and yet other cases where the specifier just did not know the existence of the case and/or did not know what to do if it occurred.

We are talking about immensely complex systems, far more complex than any machine that is built. Just look at the billions or trillions of states that a large complex system will go through in its lifecycle.

I think that given the limitations of the field today, it is not correct to talk about warranties because a warranty specifies a performance against a specification. However, we are far from having anything like a precise and complete specification for any software system.

Instead, the focus should be on the process used to produce the software: the kind of design, the steps used for its construction and the tests done during the development. All this needs to be recorded and saved for ever.

In the 1980s in the UK, the defence authorities were persuaded to issue standards called DEF 0055 and 0056 for software. The standard had a lot of conditions that no developer could hope to meet (including one that required all tools used during development to themselves have formal proof of their capabilities). The standards have since been made more realistic.

You may be interested to read what Nancy Leveson has written on software and systems reliability (she has written a whole book on the subject), likewise Peter Neumann and others. Their main objection is to the lack of rigorous method during software development, the sort of due diligence that is essential and often watered down or ignored.

---

Another correspondent, Rajendra Chittar (Raj), is an industry veteran and CS researcher of nearly three decades total experience who has a patent to his name and who states that he has delivered mission-critical bug-free applications. He wrote (slightly edited):

That is interesting.  But the problem begins at the specifications level itself - software is so poorly specified that there will always be interpretations on what is "correct and bug-free" - and to the joke "it is not a bug but a feature" - and the hype is perpetrated even by the biggies - ---,--- etc.

There is only one way in which the specifications can be made precise - so that there are no two ways about something being a bug or not - but that is via Formal Methods - and something that the industry scoffs at without even attempting to understand it fully.

We are only using too long a rope to hang ourselves!!!

I would like to put 2 definitions that go hand-in-hand - and are very relevant to the issue.  These are definitions of V&V (Verification and Validation):

Verification: The process of ensuring that we are building the product correctly.
Validation: The process of ensuring that we are building the correct product.

Either only by itself is a road to disaster - and so is the one that does neither.  The software industry (at large) is about doing neither.  That is why we have delayed, over-budget, over-time, buggy software - and which ultimately is not what the customer wanted.

---

My (Ravi's) response to the above:

Yes, I agree with both Corr1 and Raj that it is a very difficult, if not impossible task to define precise specifications for most of the complex software that is developed and used nowadays.

But I still feel a limited warranty of some sort could be considered. Now, I am not an expert on this matter and have not studied the work of researchers on this topic. However, IMHO, I am entitled to express my opinion based on my nearly three decades of experience of the software industry and software academia.

The test cases, test reports and test data that are used by a software vendor to check the quality of the software prior to releasing it to the customer, should be handed over to the customer. This will enable the customer to independently, if the customer so desires, check the quality of the software for themselves. Further, such test information & data can constitute the operating conditions for which the software vendor can provide a warranty. These operating conditions could also specify the version and other details of software components supplied by other vendors that the software solution is based on. E.g. Database, Report Generation tools and the Operating System.

If the customer encounters a bug for a specific use case in the delivered software system, they can check whether the test cases and other operating conditions cover such a use case. If it does cover it and the use case does not work as required in the test environment using the specified/agreed-upon test data, even though the test report claims that it worked, then the customer has a clear case of the software vendor not having tested the software properly as per the agreed software development contract (or software product contract with the variation that the software product company unilaterally fixes the test cases, test data, environment etc.) The limited warranty will come into play with the software vendor having to do whatever is specified in such a warranty, e.g. fixing the bug(s) at no charge to customer, and, in the case of a very-confident/aggressive warranty, provide compensation for loss faced by the customer due to the bug(s).

But will such a limited warranty which applies only to test cases & test data etc. considered at software development time be of any use at all? I think it will. As of now, I don't think that, in typical software deliveries, such test details are given to customers in such a way that they can run the tests independently. Once such a culture gets introduced, customers will get more involved with the testing process. Specifically they may ensure that the test data covers all the cases that they want in the software, as they get a warranty for such test data. If for their test data the software is found to not work during period of warranty they can hold the software vendor accountable! That can give tremendous confidence to customers about some sort of limited accountability for the software, as compared to the situation now where essentially the customer is told that the entire software is to be used "At your own risk"!

Just imagine the business risks that come into play when a business becomes heavily reliant on such software which is used "at your own risk". Further try to imagine a chairperson or managing director of such a business trying to explain to angry shareholders in an AGM, a catastrophic software failure that caused significant damage to the company, and steps that will be taken to avoid similar catastrophic software failures in future.

Now software vendors offering such limited type of warranties may charge extra for the warranty. That is fair, IMHO. The customer gets a choice between less expensive software but without any form of warranty and somewhat more expensive software with some sort of limited warranty. This would parallel many other fields in India. E.g. mobile phones. The customer can buy cheap mobile phones produced by almost unknown company(ies) with no warranty OR mobile phones produced by reputed company(ies) with warranty which are significantly more expensive than the former. There is room in the mobile phone market for both.

Similarly, I think, if some type of limited warranty is offered for some software, over time, we will have both types of software solutions - no warranty and limited warranty, each with their own space in the market.

I should also mention that many devices/machines which use embedded software (e.g. medical devices, some or most cars, airplanes), I presume, are being sold with a warranty for the devices/machines which will include the embedded software part. But that is a specialized use of software, and anyway they seem to be providing a warranty already. It is the only software solution vendors (no device) that I am referring to in this post (and previous post mentioned at the top of this post). Now, in the only software solution space there is a huge variety of software from Operating Systems to Compilers to Database-oriented business applications. While I was writing the above my thoughts have focused more on Database-oriented business applications, as life in Indian towns & cities today seems to be heavily interacting with or dependent on such applications (e.g. banking, mobile top up, railway reservation, hospital admission etc.) but I have tried to be generic. So a lot of it may apply to other kind of software like a mobile spreadsheet application too.

Now about importance to be given to the software process (requirement analysis, requirements capture, design, code, test, deployment etc.) - I entirely agree. Ideally there should be a minimum standard or a set of few minimum standards for the software process which should be published by government approved industry standard bodies. When a software vendor takes up a software development order it should inform the customer about which industry standard process it is following. Further, key artifacts produced as the software is developed using the industry standard process (design specifications, program specifications etc.)  should be provided to the customer, so that the customer can, if needed, itself, or by using a third party, inspect the artifacts to check whether the specific industry standard process is indeed being followed.

Like in the limited warranty suggestion, the customer can be given a choice: less expensive software development but which may not follow any industry standard process OR more expensive software development which follows a specific industry standard process and with various artifacts produced by the process being provided to the customer.

An important and significantly large exception to the above is that some software solutions are proprietary to the vendor and so most artifacts produced by any software development process followed for a software solution will have proprietary information that the vendor will not want to share with the customer.

Regarding Formal Methods and (other) techniques to produce bug-free software, my knowledge about these topics is very, very limited. I believe such methods and techniques have not yet been widely adopted by the large majority of software solution providers. Perhaps there are some significant challenges/drawbacks which are impeding their widespread adoption or perhaps there are other political/mind-set type reasons for it. Anyway, their adoption is something that I am not in a position to contribute to in any way. I go by what the mainstream software development industry follows. If and when they switch to Formal Methods in a big way, I may have no option but to consider it seriously then.

---

Ravi adds:
A correspondent brought up the "fit for its intended use/purpose" Engineering concept. I did some reading up about it. http://en.wikipedia.org/wiki/Implied_warranty states (for USA):

In common law jurisdictions, an implied warranty is a contract law term for certain assurances that are presumed to be made in the sale of products or real property, due to the circumstances of the sale. These assurances are characterized as warranties irrespective of whether the seller has expressly promised them orally or in writing. They include an implied warranty of fitness for a particular purpose, an implied warranty of merchantability for products, implied warranty of workmanlike quality for services, and an implied warranty of habitability for a home.
...
An implied warranty of fitness for a particular purpose is a warranty implied by law that if a seller knows or has reason to know of a particular purpose for which some item is being purchased by the buyer, the seller is guaranteeing that the item is fit for that particular purpose.
...
In some jurisdictions, an implied warranty in a sales contract can be expressly disclaimed by the use of specific language, such as the words, "as is" or "with all faults".

--- end wiki page extracts ---

So warranties need not always be limited to performance against a specification. A flawed car tyre that leads to many car accidents may not have failed its specifications, but would have failed the "fit for its intended use/purpose" implied warranty and so the manufacturer would be held accountable by the legal system/government.

The software industry seems to have disclaimed the implied warranty of "fit for its intended purpose" by usually adding the words "as is" in the contract.

Okay, so the software industry is legally not accountable even for any implied warranty. But how long can this "unaccountable" thing go on? I think it is just a matter of time before some government or some judiciary somewhere in the world responds to one or more catastrophic software failures badly affecting large number of people, by imposing accountability on software solution providers. And such imposed accountability, in all probability, ain't gonna be pretty!

It would be far, far better if the software industry puts its own house in order by experimenting with some sort of limited accountability. My suggestions outlined earlier may or may not be good enough. There may be better suggestions somewhere else. Or, if nothing good enough is around yet, if the software industry puts its mind to the problem, I am sure they can come up with some concrete suggestions for limited accountability (with its possibly higher costs which the customer must be willing to bear) and which they can experiment with.

Another correspondent questioned my assumptions about a viable market for the more expensive limited-warranty software. My views on it are as follows:

The impression that I have is that people think that software can never have any sort of even limited warranty. As far as I know, concerted efforts by software industry to show that some sort of limited warranty can even be considered, are absent.

I think for software that companies and government become heavily dependent on, e.g. hospital management software which controls many aspects of hospital functioning, income tax return online filing etc., both companies and government may be very interested in more expensive limited-warranty software solutions. In fact, they may consider only such solutions, if they were available. I am not saying that vendors should be forced to offer limited warranty software - customers should be able to specify that they are only interested in limited warranty software solutions. As of now, it seems to me, that no software (only) solutions or insignificantly small number of software (only) solutions fall in this category and so customers will not be able to specify in their Request for Proposals/(Request for) Tenders for software solutions that they want limited warranty software solutions only.

Another point was about whether the service contract offered by software solution providers does not meet the need. My views:

My initial thoughts are that typically a service contract comes into play after the warranty period runs out. Further the service contract is more limited than the warranty - e.g. may not involve replacement of product at no charge.

To a TV manufacturer kind of guy, the software industry support contract may seem like milking the customer for the flaws in your product right from day one of the sale! I think a mature product/solution offering should have a period of free repairs/replacement, and I think warranty seems to be the appropriate name for it.

A larger issue is what if the product/solution is discovered to be not "fit for its intended use/purpose" some period after the sale is made/software is used. A warranty seems to imply far more effort and cost to handle such situations than a service contract.

---

A correspondent wrote that disclaiming the implied warranty of "fit for its intended purpose" (or to be more precise, "fitness for a particular purpose", see another definition also using the same phrase as the wiki page: http://www.businessdictionary.com/definition/implied-warranty-of-fitness-for-a-particular-purpose.html) with an "as is" clause may not really work/be valid (in all countries/jurisdictions). He mentioned that he had to learn about such matters when he took the exam for a Professional Engineering license (in Canada, I believe). The correspondent wrote that if somebody presses the issue then the implied warranty liability will hold even if there is an "as is" clause (at least in some countries/jurisdictions like Canada).

---

Correspondent Corr1 wrote:

"warranty of fitness for a particular purpose"

The whole point about large and complex software systems is precisely that the "particular purpose" is (a) not defined, (b) incompletely defined or (c) wrongly defined.

Moreover, what works with one version of infrastructural software (operating system, database system, communication system etc.) may not work with later versions.

Of course we need more rigour in the way software is defined and produced. This is something software engineers aspire to and it will take time. Nevertheless, good software engineering practice today already follows best practices and uses all available tools to make software reliable.

-- end Corr1 message --

Ravi's response:

The particular purpose of a car tyre too may not be defined completely. But that does not allow the tyre manufacturers to escape accountability for repeated accidents involving their tyres. We have a famous case of millions of tyres manufactured by a particular company being recalled, http://en.wikipedia.org/wiki/Firestone_and_Ford_tire_controversy, http://www.thecarconnection.com/news/1061100_a-decade-after-the-firestone-recall-tire-safety-still-newsworthy. The reports do not mention that the tyres did not meet the specifications. They apparently did but, perhaps for cost saving purposes, lacked a crucial extra liner on the tire which would have made it safe and car/road worthy.

I feel that the software industry needs to seriously explore some sort of limited accountability instead of sheltering in an "as is" or "use at your own risk" completely unaccountable strategy because the specifications for software typically are not well defined. The limited accountability can take into account the fact that specifications for software are typically not well defined and the version changes of infrastructure software (OS, Database, Comm. s/w etc.) E.g. The accountability could be limited to only the specific versions of the infrastructure software that was used for development and test of the software solution, and could be limited to the functionality specified in the requirement specifications and test specifications/reports.