Risk, Failure and Uncertainty - An essay on the habits and attitudes beneficial to solving complex problems
The Iterative Process of Solving Complex Problems
Refer to the graphic from yesterday's post. The process begins by generating initial-step options symbolized by letters A through Z. The initial-step options are created by using various techniques, or possibly by repeated application of just one technique. Consciously or unconsciously, expert solvers do the same when they attack such problems. For the expert solver, however, just one initial-step option is usually all it takes, being created intuitively and accurately from years of experience and stored information. But even the expert must follow through the iterative process to validate his/her initial-step choice. For the novice solver (my students), several initial-step options probably need to be considered. If the novice uses his or her intuition, the initial-step options are almost certainly biased by a limited experience-base, although s/he still might intuitively produce an acceptable choice. Alternatively, the novice may draw upon a 'rule of thumb', a guideline s/he learned to associate with some special feature in the problem's starting point (e.g., "whenever you see this, do that"). If you think 'rule of thumb' sounds like an algorithm, you're correct. However, remember that the 'rule of thumb' only provides an initial-step option, rather than guaranteeing a pathway to a solution. Other techniques commonly used to generate initial-step options are reasoning-by-analogy whereby a complex problem is seen in relationship to another problem whose solution is known. If all else fails, a slow, detailed and deliberate analysis is pursued which, for organic chemistry, might involve a bonds-made, bonds-broken association chart. This list of techniques isn't exhaustive (e.g., try working backwards from the solution), but the list represents the common approaches to create initial-step options.
Next the solver must choose an option from those created. It is very possible that one of the initial-step options stands out as better than the others, but it is also possible that none of them appear more reasonable than the others. It is also possible that the solver has yet to consider the correct initial-step that leads to a solution; only time will tell. The solver must now take further action since inaction gets him/her no closer to a solution. So, a
risk must be taken - a choice must be made - an option must be pursued. With the initial choice in hand, the solver performs further action on the initial choice to advance it toward a solution. This "something" might be to create another set of options - the secondary-step options - using the techniques outlined above, possibly with the aid of field-specific tools (e.g., organic chemists might use curved arrow notation). Once the secondary-step options are created, an evaluation is made by comparing the developing solution to the targeted solution, to see if there is an obvious pathway to completion. If no clear pathway to the targeted solution is yet in sight, the solver must contemplate defeat or face the uncertainty of continuing onward. If the developing solution shows obvious shortcomings, the solver may accept
failure and return to the initial-step choices, beginning the entire process again, possibly by creating an even larger set of initial-step choices. Alternatively, the solver may decide to proceed onward through
uncertainty by generating another round of next-step options and further evaluating the result. This iterative process continues until a solution is reached.
The challenges are obvious: successfully generating a sufficient number of options, overcoming narrowness that results from biases, continually making choices from a growing and tangled web of options, knowing when to cut one's losses, and knowing when to continue to plow through uncertainty are all very daunting, especially for the novice. So how does the novice become more like the expert? S/he acquires a set of experiences that produce accurate intuition. This takes concentration, persistence, adaptability and regular practice over an extended time. The gratification realized as the solver develops expert-like intuition is incredibly rewarding, but it comes slowly. Those novices who become successful solvers are probably comfortable delaying their gratification.
The tomorrow's post I'll present my views on the role of memory in solving complex problems.