© Collier Research Systems | Charlottesville, VA

Search
  • Zachary Collier

Can You Validate a Decision?

Sometimes when I am out speaking about decision making, I get asked whether or not you can validate a decision.

The short answer is, “not really”.

The longer answer is, of course, more complicated. First, it helps to define what we mean by validation. Roughly, validation is the process of determining the “correspondence of the model and its outputs to perceived reality” (1).

Most models in the hard sciences can be validated by running experiments. The models about gravity that you learn in high school physics class can be validated by dropping some objects and measuring the time they take to hit the ground. Here we have a case where we can compare our experiments in physical reality and compare them to the mathematical abstractions of our models. And they generally line up quite nicely.

But it isn’t so straightforward when the thing you are modeling is a decision. For example, you can’t really run an experiment with a big decision - you can’t compare the world where you made the decision against a different world where you made a different decision. It just doesn’t work like that.

If physical models are about the physical world, what are decision models about? Decision models capture the judgements and understandings held by individuals and groups (2). Stated differently, “decisions are not found in nature, they are creations of the human mind” (3).

That is why the concept of decision quality, discussed in a previous post, is so important. In the absence of formal validation methods, we must rely upon making sure that we make the best decisions we can with the information we have on hand. This includes asking questions like:


-Is the problem framed correctly, and am I even asking the right question?

-What are my objectives?

-How will I know that my objectives have been met, and how might they be measured?

-Do I have the right data, and where can I find more data?

-What alternatives are available to me, and what are the constraints on my actions?

-What are my preferences, and what tradeoffs am I willing to make?

-What are the risks? What can go wrong, what are the likelihoods, and what are the consequences?

-Who else is involved in this decision, and do I need their input?

There is no magical recipe for making good decisions, and no easy way to validate a decision that has been made. But that doesn’t mean that there isn’t enormous value added by using decision models to thoughtfully step through the decision making process.

In fact, the best way to ensure that your decisions turn out favorably is to develop and implement good decision making practices. A trained expert in decision analysis can help greatly with this.

So can you validate a decision? Probably not. But can you build models that help you make better decisions and cultivate good decision making practices? Definitely.


At Collier Research Systems, we help clients with exactly that - making good decisions. To learn more about the decision support and modeling services we offer, visit: www.collierresearchsystems.com.

References

(1) Gass, S.I. (1983). “Decision-aiding models: validation, assessment, and related issues for policy analysis.” Operations Research, 31(4): 603-631.

(2) Phillips, L.D. (1984) “A theory of requisite decision models.” Acta Psychologica, 56: 29-48.

(3) Howard, R.A. (2007). “The foundations of decision analysis revisited.” In: Advances in Decision Analysis: From Foundations to Applications. Edwards, W., Miles, R.F., and Von Winterfeldt, D., (eds.). Cambridge University Press: Boston, pp. 32-56.