- In many ways, managing a large computer programming project is like managing any other large undertaking--in more ways than most programmers believe. But in many other ways it is different--in more ways than most professional managers expect.
- Briefly, I believe that large programming projects suffer management problems different in kind from small ones, due to division of labor. I believe that critical need to be the preservation of the conceptual integrity of the product itself.
- No one thing seems to cause the difficulty--any particular paw can be pulled away. But the accumulation of simultaneous and interacting factors brings slower and slower motion.
- It [a program] is complete in itself, ready to be run by the author on the system on which it was developed. That [the program] is the thing commonly produced in garages, and that is the object the individual programmer uses in estimating productivity.
- This [a programming product] is a program that can be run, tested, and extended by anybody. It is usable in many operating environments, for many sets of data. To become a generally usable programming product, a program must be written in a generalized fashion.
- As a rule of thumb, I estimate that a programming product costs at least three times as much as a debugged program with the same function.
- This [a programming system] is a collection of interaction programs, coordinated in function and disciplined in format, so that the assemblage constitutes an entire facility for large tasks. To become a programing system component, a program must be written so that ever input and output conforms in syntax and semantics with precisely defined interfaces.
- A programming system component costs at least three times as much as a stand-alone program of the same function. The cost may be greater if the system has many components.
- This [a programming systems product] differs from the simple program in all of the above ways. It costs nine times as much. But it is the truly useful object, the intended product of most system programming efforts.
- Why is programming fun? First is the sheer joy of making things. Second is the pleasure of making things that are useful to other people. Deep within, we want others to use our work and to find it helpful. This is the fascination of fashioning complex puzzle-like objects of interlocking moving parts and watching them work in subtle cycles, playing out the consequences of principles built in from the beginning. The programmed computer has all the fascination of the pinball machine or the jukebox mechanism, carried to the ultimate. Fourth is the joy of always learning, which springs from the non-repeating nature of the task. Finally, there is the delight of working in such a tractable medium. The programmer, like the poet, works only slightly removed from pure thought-stuff.
- Programming then is fun because it gratifies creative longings built deep within us and delights sensibilities we have in common with all men.
- Human beings are not accustomed to being perfect, and few areas of human activity demand it. Adjusting to the requirement for perfection is, I think, the most difficult part of learning to program.
- In management terms, one's authority is not sufficient for his responsibility. It seems that in all fields, however, the jobs where things get done never have formal authority commensurate with responsibility. In practice, actual (as opposed to formal) authority is acquired from the very momentum of accomplishment.
- The dependence upon others has a particular case that is especially painful for the system programmer. He depends upon other people's programs. These are often mal-designed, poorly implemented, incompletely delivered (no source doe or test cases), and poorly documented.
- With any creative activity come dreary hours of tedious, painstaking labor, and programming is no exception.
- The new and better product is generally not available when one completes his own; it is only talked about. It, too, will require months of development. The real tiger is never a match for the paper one, unless actual use is wanted. Then the virtues of reality have a satisfaction all their own.
- As soon as one freezes a design, it becomes obsolete in terms of its concepts. But implementation of real products demands phasing and quantizing.
- The challenge and the mission are to find real solutions to real problems on actual schedules with available resources.
- More software projects have gone awry for lack of calendar time than for all other causes combined.
- First, our techniques of estimating are poorly developed. More seriously, they reflect an unvoiced assumption which is quite untrue, i.e., that all will go well.
- Second, our estimating techniques fallaciously confuse effort with progress, hiding the assumption that men and months are interchangeable.
- Techniques proven and routine in other engineering disciplines are considered radical innovations in software engineering.
- All programmers are optimists.
- So the first false assumption that underlies the scheduling of systems programming is that all will go well, i.e., that each task will take only as long as it "ought" to take.
- For the human makers of things, the incompleteness and inconsistencies of our ideas become clear only during implementation.
- We tend to blame the physical media for most of our implementation difficulties; for the media are not "ours" in the way the ideas are, and our pride colors our judgment.
- Because our ideas are faulty, we have bugs; hence our optimism is unjustified.
- Cost does indeed vary as the product of the number of men and the number of months. Progress does not. Hence the man-month as a unit for measuring the size of a job is a dangerous and deceptive myth. It implies that men and months are interchangeable.
- Men and months are interchangeable commodities only when a task can be partitioned among many workers with no communications among them.
- When a task cannot be partitioned because of sequential constraints, the application of more effort has no effect on the schedule.
- In tasks that can be partitioned but which require communication among the sub tasks, the effort of communication must be added to the amount of work to be done.
- The added burden of communication is made up of two parts, training and intercommunication.
- This training cannot be partitioned, so this part of the added effort varies linearly with the number of workers.
- Intercommunication is worse. If each part of the task must be separately coordinated with each other part, the effort increases as n(n-1)/2.
- The added effort of communicating may fully counteract the division of the original task.
- Since software construction is inherently a systems effort--an exercise in complex interrelationships--communication effort is great, and it quickly dominates the decrease in individual task time brought about by partitioning. Adding more men then lengthens, not shortens, the schedule.
- No parts of the schedule are so thoroughly affected by sequential constraints as component debugging and system test.
- Because of optimism, we usually expect the number of bugs to be smaller than it turns out to be. Therefore testing is usually the most miss-scheduled part of programming.
- In examining conventionally scheduled projects, I have found that few allowed one-half of the projected schedule for testing, but that most did indeed spend half of the actual schedule for that purpose.
- Failure to allow enough time for system test, in particular, is peculiarly disastrous. Since the delay comes at the end of the schedule, no one is aware of schedule trouble until almost the delivery date.
- Observe that for the programmer, as for the chef, the urgency of the patron may govern the scheduled completion of the task, but it cannot govern the actual completion.
- But false scheduling to match the patron's desired date is much more common in our discipline than elsewhere in engineering.
- Until estimating is on a sounder basis, individual managers will need to stiffen their backbones and defend their estimates with the assurance that their poor hunches are better than wish-derived estimates.
- I like the advice given by P. Fagg, an experienced hardware engineer, "Take no small slips." That is, allow enough time in the new schedule to ensure that the work can be carefully and thoroughly done, and that rescheduling will not have to be done again.
- Adding manpower to a late software project makes it later.
- The number of months of pa project depends upon its sequential constraints. The maximum number of men depends upon the number of independent sub tasks.
- More software projects have gone awry for lack of calendar time than for all other causes combined.
- Programming managers have long recognized wide productivity variations between good programmers and poor ones. But the actual measured magnitudes have astounded all of us.
- Within just this group the ratios between best and worst performances averaged about 10:1 on productivity measurements and an amazing 5:1 on program speed and space measurements!
- The data showed no correlation whatsoever between experience and performance.
- This, too, suggests that one wants the system to be built by as few minds as possible. Indeed, most experience with large programming systems shows that the brute-force approach is costly, slow, inefficient, and produces systems that are not conceptually integrated.
- The conclusion is simple: if a 200-man project has 25 managers who are the most competent and experienced programmer, fire the 175 troops and put the mangers back to pr9ogramming.
- On the other hand, it fails to approach the ideal of the small sharp team, which by common consensus shouldn't exceed 10 people.
- This then is the problem with the small, sharp team concept: it is too slow for really big systems.
- For efficiency and conceptual integrity, one prefers a few good minds doing design and construction. Yet for large systems one wants a way to bring considerable manpower to bear, so that the product can make a timely appearance.
- Mills proposes that each segment of a large job be tackled by a team, but that the team be organized like a surgical team rather than a hog-butchering team. That is, instead of each member cutting away on the problem, one does the cutting and the others give him every support that will enhance his effectiveness and productivity.
- I will contend that conceptual integrity is the most important consideration in system design. It is better to have a system omit certain anomalous features and improvements, but to reflect one set of design ideas, than to have one that contains many good but independent and uncoordinated ideas.
- The purpose of a programming language system is to make a computer easy to use.
- Ease of use is enhanced only if the time gained in functional specification exceeds the time lost in learning, remembering, and searching manuals.
- Because ease of use is the purpose, this ration of function to conceptual complexity is the ultimate test of system design. Neither function alone nor simplicity alone defines a good design.
- Function, and not simplicity, has always been the measure of excellence for its designers.
- Simplicity is not enough.
- Simplicity and straightforwardness proceed from conceptual integrity. Every part must reflect the same philosophies and the same balancing of desiderata. Every part must even use the same techniques in syntax and analogous notions in semantics. Ease of use, then, dictates unity of design, conceptual integrity.
- Conceptual integrity in turn dictates that the design must proceed from one mind, or from a very small number of agreeing resonant minds.
- Schedule pressures, however, dictate that system building needs many hands.
- The separation of architectural effort from implementation is a very powerful way of getting conceptual integrity on very large projects.
- Architecture must be carefully distinguished from implementation.
- Good features and ideas that do not integrate with a system's basic concepts are best left out.
- If a system is to have conceptual integrity, someone must control the concepts.
- In an unconstrained implementing group, most thought and debate goes into architectural decisions, and implementation proper gets short shrift.
- It is a very humbling experience to make a multi-million-dollar mistake, but it is also very memorable.
- The opportunity to be creative and inventive in implementation is not significantly diminished by working within a given external specification, and the order of creativity may even be enhanced by that discipline.
- The architect has two possible answers when confronted with an estimate that is too high: cut the design or challenge the estimate by suggesting cheaper implementations.
- An architect's first work is apt to be spare and clean. He knows he doesn't know what he's doing, so he does it carefully and with great restraint.
- Then general tendency is to over-design the second system, using all the ideas and frills that were cautiously sidetracked on the first one.
- The second-system effect has another manifestation some what different from pure functional embellishment. That is a tendency to refine techniques whose very existence has been made obsolete by changes in basic system assumptions.
- How does the architect avoid the second-system effect? Well, obviously he can't skip his second system. But he can be conscious of the peculiar hazards of that system, and exert extra self-discipline to avoid functional ornamentation and to avoid extrapolation of functions that are obviated by changes in assumptions and purposes.
- A discipline that will open an architect's eyes is to assign each little function a value: capability x is worth not more than m bytes of memory and n microseconds per invocation. These values will guide initial decisions and serve during implementation as a guide and warning to all.
- How does the project manager avoid the second-system effect? By insisting on a senior architect who has at least two systems under his belt.
- The manual is the external specification of the product. It describes ad prescribes every detail of what the user sees.
- The manual must not only describe everything the user does see, including all interfaces; it must also refrain from describing what the user does not see.
- In most computer projects there comes a day when it is discovered that the machine and the manual don't agree. When the confrontation follows, the manual usually loses, for it can be changed far more quickly and cheaply than the machine. Not so, however, when there are multiple implementations.
- Schedule disaster, functional misfits, and system bugs all arise because the left hand doesn't know what the right hand is doing.
- His thesis is that the programmer is most effective if shielded from, rather than exposed to the details of construction of system parts other than his own. This presupposes that all interfaces are completely and precisely defined.
- A good information system both exposes interface errors and stimulates their correction.
- If there are n workers on a project, there are (n^2-n)/2 interfaces across which there may be communication, and there are potentially almost 2^n teams which coordination must occur. The purpose of organization is to reduce the amount of communication and coordination necessary; hence organization is a radical attack on the communication problems treated above.
- Organizations must be designed around the people available; not people fitted into pure-theory organizations.
- First, the man with strong management talent and strong technical talent is rarely found. Thinkers are rare; doers are rarer; and thinker-doers are the rarest.
- It is hard for the producer to delegate enough of his duties to give him any technical time. It is impossible for the director to delegate his without compromising the conceptual integrity of the design.
- The job done least well by project managers is to utilize the technical genius who is not strong on management talent.
- Communication and its consequent, organization, are critical for success.
- The linear extrapolation of such sprint figures is meaningless. Extrapolation of times for the hundred-yard dash shows that a man can run a mile in under three minutes.
- Programing productivity may be increased as much as five times when a suitable high-level language is used.
- How big is it? Aside from running time, the space occupied by a program is a principal cost.
- In most previous operating systems, systems residence had been on tape, and the long search times of tape meant that one was not tempted to use it casually to bring in program segments.
- No amount of space budgeting and control can make a program small. That requires invention and craftsmanship.
- Obviously, more function means more space, speed being held constant. So the first area of craftsmanship is in trading function for size.
- The second area of craftsmanship is space-time trade-offs. For a given function, the more space, the faster. This is true over an amazingly large range. It is this fact that makes it feasible to set space budgets.
- The manager can do two things to help his team make good space-time trade-offs. One is to ensure that they are trained in programming technique, not just left to rely on native wit and previous experience. For a new language or machine this is especially important.
- The second is to recognize that programming has a technology, and components need to be fabricated. Every project needs a notebook full of good subroutines or macros for queing, searching, hashing, and sorting. For each such function the notebook should have at least two programs, the quick and the squeezed.
- Beyond craftsmanship lies invention, and it is here that lean, spare, fast programs are born. Almost always these are the result of strategic breakthrough rather than tactical cleverness.
- Much more often, strategic breakthrough will come from redoing the representation of the data or tables. This is where the hear of a program lies. Show me your flowcharts and conceal your tables, and I shall continue to be mystified. Show me your tables, and I won't usually need your flowcharts; they'll be obvious.
- Representation is the essence of programming.
- Not merely a constraint, the budget is one of the manager's most useful documents. Existence of the budget forces technical decisions that otherwise would be avoided; and, more important, it forces and clarifies policy decisions.
- Conway's Law predicts: "Organizations which design systems are constrained to produce systems which are copies of the communication structures of these organizations." Conway goes on to point out that the organization chart will initially reflect the first system design, which is almost surely not the right one. If the system design is to be free to change, the organization must be prepared to change.
- Writing the decisions down is essential. Only when one writes do the gaps appear and the inconsistencies protrude. The act of writing turns out to require hundreds of mini-decision, and it is the existence of these that distinguished clear, exact polices from fuzzy ones.
- The documents will communicate the decisions to others.
- Since his [the manager's] fundamental job is to keep everybody going in the same direction, his chief daily task will be communication, not decision-making, and his documents will immensely lighten this load.
- The task of the manger is to develop a plan and then to realize it. But only the written plan is precise and communicable. Such a plan consists of documents on what, when , how much, where, and who. This small set of critical documents encapsulates much of the manager's work. If their comprehensive and critical nature is recognized in the begging, the manager can approach them as friendly tools rather than annoying busywork. He will set his direction much more crisply and quickly by doing so.
- In most project, the first system built is barely usable. It may be too slow, too big, awkward to use, or all three. There is no alternative but to start again, smarting but smarter, and build a redesigned version in which these problems are solved.
- Where a new system concept or new technology is used, one has to build a system to throw away, for even the best planning is not so omniscient as to get it right the first time.
- Plan to throw one away; you will, anyhow.
- Cosgrove has perceptively pointed out that the programmer delivers satisfaction of a user need rather than any tangible product. And both the actual need and the user's perception of that need will change as programs are built, tested, and used.
- Both the tractability and the invisibility of the software product expose it's builders to perpetual changes in requirements.
- Nevertheless, some changes in objectives are inevitable, and it is better to be prepared for them than to assume that they won't come.
- The throw-one-away concept is itself just an acceptance of the fact that as one learns, he changes the design.
- Quantization of change is an essential technique. Every produce should have numbered versions, and each version must have its own schedule and a freeze date, after which changes go into the next version.
- Structuring an organization for change is much harder than designing a system for change. Each man must be assigned to jobs that broaden him, so that the whole force is technically flexible.
- The barriers are sociological, and they must be fought with constant vigilance. First, managers themselves often think of senior people as "too valuable" for actual programming. Next, management jobs carry higher prestige.
- Managers need to be sent to technical refresher courses, senior technical people to management training.
- Whenever talents permit, senior people must be kept technically and emotionally ready to manage groups or to delight in building programs with their own hands. Doing this surely is a lot of work; but it surely is worth it!
- The total cost of maintaining a widely used program is typically 40 percent or more of the cost of developing it. Surprisingly, this cost is strongly affected by the number of users. More users find more bugs.
- The fundamental problem with program maintenance is that fixing a defect has a substantial (20-50 percent) chance of introducing another. So the whole process is two steps forward and one step back.
- As a consequence of the introduction of new bugs, program maintenance requires far more system testing per statement written than any other programming. Theoretically, after each fix one must run the entire bank of test cases previously run against the system, to ensure that it has not been damaged in an obscure way. In practice such regression testing must indeed approximate this theoretical ideal, and it is very costly.
- System program building is an entropy-decreasing process, hence inherently meta-stable. Program maintenance is an entropy-increasing process, and even its most skillful execution only delays the subsidence of the system into unfixable obsolescing.
- The target machine is the one for which software is being written, and on which it must ultimately be tested. The vehicle machines are those that provide the services used in building the system.
- If the target computer is new, one needs a logical simulator for it. This gives a debugging vehicle long before the real target exists. Equally important, it gives access to a dependable debugging vehicle even after one has a target machine available.
- Dependable is not the same as accurate.
- Lab-built, pre-production, or early hardware does not work as defined, does not work reliably, and does not stay the same from day to day.
- The most important two tools for system programming today are two that were not used in OS/360 development almost a decade ago. They are still not widely used, but all evidence points to their power and applicability. They are (1) high-level language and (2) interactive programming. I am convinced that only inertia and sloth prevent the universal adoption of these tools; the technical difficulties are no longer valid excuses.
- The chief reasons for using a high-level language are productivity and debugging speed.
- The debugging improvement comes from the fact that there are fewer bugs, and they are easier to find. There are fewer because one avoids an entire level of exposure to error, a level on which one makes not only syntactic errors but semantic ones, such as misusing registers. The bugs are easier to find because the compiler diagnostics help find them and, more important, because it is very easy to insert debugging snapshots.
- I cannot easily conceive of a programming language system I would build in assembly language.
- There is a widespread recognition that debugging is the hard and slow part of system programming, and slow turnaround is the bane of debugging. So the logic of interactive programming seems inexorable.
- Harr's data suggest that an interactive facility at least doubles productivity in system programming.
- The most pernicious and subtle bugs are system bugs arising from mismatched assumptions made by the authors of various components.
- Careful function definition, careful specification, and the disciplined exorcism of frills of function and flights of technique all reduce the number of system bugs that have to be found.
- In a very clear 1971 paper, Niklaus Wirth formalized a design procedure which had been used for years by the best programmers. Briefly, Wirth's procedure is to identify design as a sequence of refinement steps.
- The important point, and the one vital to constructing bug-free programs, is that one wants to think about the control structures of a system as control structures, not as individual branch statements. This way of thinking is a major step forward.
- The unexpectedly hard part of building a programming system is system test.
- From all of that, one should be convinced of two things: system debugging will take longer than one expects, and its difficulty justifies a thoroughly systematic and planned approach.
- Common sense, if not common practice, dictates that one should begin system debugging only after the pieces seem to work.
- One does not know all the expected effects of known bugs. If things were straightforward, system testing wouldn't be hard.
- One form of scaffolding is the dummy component, which consists only of interfaces and perhaps some faked data or some small test cases.
- Another form is the miniature file. A very common form of system bug is misunderstanding of formats for tape and disk files. So it is worthwhile to build some little files that have only a few typical records, but all the descriptions, pointers, etc.
- The limiting case of miniature file is the dummy file, which really isn't there at all.
- Yet another form of scaffolding are auxiliary programs. Generators for test data, special analysis printouts, cross-reference table analyzers, are all examples of the special-purpose jigs and fixtures one may want to build.
- There must be controlled copies of the system: one locked-up copy of the latest versions, used for component testing; one copy under test, with fixes being installed; playpen copies where each man can work away on his component, doing both fixes and extensions.
- Add one component at a time.
- One must assume that there will be lots of bugs, and plan an orderly procedure for snaking them out.
- Note that one must have thorough test cases, testing the partial systems after each new piece is added. And the old ones, run successfully on the last partial sum, must be rerun on the new one to test for system regression.
- The replacement of a working component by a new version requires the same systematic testing producer that adding a new component does, although it should require less time, for more complete and efficient test cases will usually be available.
- Indeed, major calamities are easier to handle; one responds with major force, radical reorganization, the invention of new approaches. The whole team rises to the occasion. But the day-by-day slippage is harder to recognize, harder to prevent, harder to make up.
- How does one control a big project on a tight schedule? The first step is to have a schedule. Each of a list of events, called milestones, has a date. Picking the dates is an estimating problem, discussed already and crucially dependent on experience.
- For picking the milestones there is only one relevant rule. Milestones must be concrete, specific, measurable events, defined with knife-edge sharpness.
- It is more important that milestones be sharp-edged and unambiguous, than that they be easily verifiable by the boss.
- Sharp milestones are in fact a service to the team, and on they can properly expect from a manager. The fuzzy milestone is the harder burden to live with. It is in fact a millstone that grind down morale, for it deceives one about lost time until it is irremediable. And chronic schedule slippage is a morale-killer.
- But every boss needs two kinds of information, exceptions to plan that require action and a status picture for education.
- A computer program is a message from a man to a machine. The rigidly marshaled syntax and the scrupulous definitions all exist to make intention clear to the dumb engine.
- Different levels of documentation are required for the casual user of a program, for the user who must depend upon a program, and for the user who must adapt a program for changes in circumstance or purpose.
- Every copy of a program shipped should include some small test cases that can be routinely used to reassure the user that he has a faithful copy, accurately loaded into the machine.
- The flow chart is a most thoroughly oversold piece of program documentation. Many program don't need flow charts at all; few programs need more than a one-page flow chart.
- The one-page flow chart for a substantial program becomes essentially a diagram of program structure, and of phases or steps. As such it is very handy.
- In fact, flow charting is more preached than practiced. I have never seen an experienced programmer who routinely made detailed flow charts before beginning to write programs. Where organization standards require flow charts, these are almost invariably done after the fact.
- A basic principle of data processing teaches the folly of trying to maintain independent files in synchronization. It is far better to combine them into one file with each record containing all the information both files held concerning a given key. Yet our practice in programming documentation violates our own teaching. We typically attempt to maintain a machine readable form of a program and an independent set of human-readable documentation, consisting of prose and flow charts. The results in fact confirm our teaching about the folly of separate files. Program documentation is notoriously poor, and its maintenance is worse. Changes made in the program do not promptly, accurately, and invariably appear in the paper. The solution, I think, is to merge the files, to incorporate the documentation in the source program. This is at once a powerful incentive toward proper maintenance, and an insurance that the documentation will always be handy to the program user. Such programs are called self-documenting.
- The time has come to devise radically new approaches and methods for program documentation.
- As a principle objective, we must attempt to minimize the burden of documentation, the burden neither we nor our predecessors have been able to bear successfully.
- Since the documentation is built into the structure, naming, and formats of the program, much of it must be done when the program is first written. But that is when it should be written. Since the self-documentation approach minimizes extra work, there are fewer obstacles to doing it then.
- Refer to standard literature to document basic algorithms wherever possible. This saves space, usually points to a much fuller treatment than one would provide, and allows the knowledgeable reader to skip it with confidence that he understands you.
- The tar pit of software engineering will continue to be sticky for a long time to come. One can expect the human race to continue attempting systems just within or just beyond our reach; and software systems are perhaps the most intricate and complex of man's handiwork. The management of this complex craft will demand our best use of new languages and systems, our best adaptation of proven engineering management methods, liberal doses of common sens, and a God-given humility to recognize our fallibility and limitations.
20170623
The Mythical Man-Month by Frederick P. Brooks, Jr.
Labels:
books
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment