Sample Worksheet - Safety Critical Systems

1. Indicate the status of the software testing as of D-Day (see scenario).
2. If appropriate, make recommendations, not suggestions (!), for action.
3. If appropriate, attach calculations or supplemental data, that is – "backup."

Be prepared to defend the statements made in your memo.

Suppose that the input variables are 16-bit words which are read into the program. Suppose that in any one word the probability of a 1-bit error occurring is 10-7, and that such errors are statistically-independent and that they are equally-distributed over a word.

1. What is the probability that a word is error-free?
2. What is the probability of precisely 2 bit-errors occurring in one word?
3. What is the probability of precisely 1 bit-error occurring in each of two successive words?

NOTE: Do not use the information of problem 2 as data in problem 1.

You are employed by VaporWare, Inc. (VWI) in its Software Development Group. Among your responsibilities is the testing and quality sign-off of custom software before full release to the customer or client. Normally, this is the last in-house work prior to delivery of product.

VWI is a small, privately-held, "start-up" which does custom hardware and software design. You own no part of VWI, but your employment contract specifies that in two years you will be considered for inclusion in the limited partnership. A majority vote of the owners is sufficient for you to be so rewarded.

You are the only person involved in the testing of the software named APUDA. The product is to be delivered to One Big Monopoly, Inc. (OBM) who intend to use APUDA in an unstated process control system. The APUDA product was to be designed to a specification furnished by OBM, which has from the outset refused to discuss the application. (Rumors have circulated that the product is being developed for the CIA. Still other rumors have it that market and trade considerations is the reason for secrecy.)

The profit to VWI, based on an on-time delivery of APUDA, is estimated to be $170 thousand. There is no bonus for early delivery. (Other estimates indicate that for the current fiscal year the total VWI profits will be $1.2 million.) A penalty clause stipulates that payment to VWI will be reduced by an amount such that VWI profit for the APUDA project will be reduced by about $20 thousand for each day of delay after the due day (D-Day). Thus, for example, if the product is 10 days late, VWI will suffer a loss of $30 thousand.

Your close friend Pat was the team leader in writing the code. Because of illness to one of the team members, and another leaving VWI for another job, you received APUDA 3 days after the scheduled time to begin full-functionality testing. If you work 18-hour days, and if only minor bugs need fixing, you are confident that you can complete the testing to the original schedule and thus ship "on time." You begin testing.

Testing proceeds well and is nearly complete at 3:00 a.m. on the last day. For on-time delivery APUDA source-code is to ship by overnight carrier at 4:00 p.m. this (last) day. You enter the next-to-last input sequence and to your horror observe that the output states are not as specified. You examine the input sequence just entered and are relieved to note that you made an error in two bits in entering the input-variable sequence. Entering the correct input sequence yields the correct output. You complete the test and all is well. Of the 1,024 allowed input sequences of 16-bit words, only the next-to-last word yielded erroneous outputs.

You check again the OBM specification and observe that it does not specify performance for input bit-errors, i.e. except that if an input sequence is not in the set of allowed inputs the outputs shall remain in its previous states.

You talk to Pat at 8:00 a.m. of D-Day and are assured by Pat that, "The code meets specifications."