Support Your Local Product Owner

I've been searching for a software architect position (or something similar). Job titles aren't always the best indicator of what a company wants out of a role. Many of these roles involve some aspect of product ownership. I'm usually a bit cautious because in order to be successful as a product owner there are five things needed:

• Bandwidth
• Having enough time to work with the Scrum team, manage the product, and work with the users, customers, and stakeholders.
• Power
• Having the authority to manage the stakeholder needs, order and refine the product backlog and product needs, and the ability to accept the product increments.  
• Knowledge
• Having a deep understanding of the product, who the users and stakeholders are, what they find valuable, and what the organization’s goals are for the product.       
• Interest
• Having a strong desire to be the product owner and being able to maximize the value of the product.
• Vision
• Having a great understanding of what the product is, where it should be going, and the goals for the product.

These attributes are a pretty big deal and an employer needs to make sure that each one of them is supported to ensure success of the product owner and to improve the odds of the product's success.

Principles of Distributed System Design

Every day software engineers face the task of designing new systems or maintaining existing systems. Whether the need to make those systems distributed is due to performance or reliability requirements it hardly matters. Distributed system design needs to be considered and broken into a limited number of principles to adequately assess the tradeoffs and costs.

Below are 10 principles of distributed system design that I think do a good job summarizing and separating the problem. These are the principles that Amazon used when designing their S3 service (see reference at bottom).

▸ Decentralization: Use fully decentralized techniques to remove scaling bottlenecks and single points of failure.

▸ Asynchrony: The system makes progress under all circumstances.

▸ Autonomy: The system is designed such that individual components can make decisions based on local information.

▸ Local responsibility: Each individual component is responsible for achieving its consistency; this is never the burden of its peers.

▸ Controlled concurrency: Operations are designed such that no or limited concurrency control is required.

▸ Failure tolerant: The system considers the failure of components to be a normal mode of operation and continues operation with no or minimal interruption.

▸ Controlled parallelism: Abstractions used in the system are of such granularity that parallelism can be used to improve performance and robustness of recovery or the introduction of new nodes.

▸ Decompose into small, well-understood building blocks: Do not try to provide a single service that does everything for everyone, but instead build small components that can be used as building blocks for other services.

▸ Symmetry: Nodes in the system are identical in terms of functionality, and require no or minimal node-specific configuration to function.

▸ Simplicity: The system should be made as simple as possible, but no simpler.

Reference:

Amazon Web Services Launches

Control Baselines

It is the responsibility of software configuration management (SCM) to hold the agreed-upon specifications and regulate changes to them. You might not have a board in charge of SCM. I once worked as a technical product manager (TPM) and controlled the backlog for several teams. Regardless of the name, there is someone or a group that sets priorities for tasks yet to be completed.

While repairing or enhancing a software component, a software engineer will occasionally discover something else that can be changed, perhaps to fix a yet unreported bug or to add some quick new feature. This kind of uncontrolled (and untracked) change is intolerable. SCM should avoid incorporating such changes into the new baseline. The correct procedure is for the software engineer to make a change request (CR). This CR is then processed along with the others from development, marketing, testing, and the customers by a configuration control board, which prioritizes and schedules the change requests. Only then can the engineer be allowed to make the change and only then can the SCM accept the change.

Configuration control board is a bit of an old, formal name. I think most companies these days just use the representatives of the stakeholders - product managers, engineering directors, and architects.

So, this practice is immensely frustrating to the go-getter developers who see problems and want to fix them. But it is immensely practical in the sense that one man's bug is another man's expected functionality and changing things breaks expectations which leads to upset customers.

Reference:

Bersoff, E., Henderson, V., and Siegel, S., Software Configuration Management, Englewood Cliffs, NJ: Prentice Hall, 1980.

The Thought That Disaster Is Impossible Often Leads To Disaster

Gerald Weinberg's "Titanic Effect" principle is that believing that disaster is impossible often leads to disaster. You must never become so smug that you think everything is under control and will remain that way. Overconfidence is the primary cause of many disasters. It is the mountain climber who says, "It's just a small climb; I don't need a belay," or the hiker who says, "It's just a short hike; I don't need water," who gets into trouble. Reduce risk to your projects by analyzing all your potential disasters up front, develop contingency plans for them in advance, and continually reassess new risks. This principle emphasizes the need to expect these risks to become real. Your biggest management disasters will occur when you think they won't.

Perhaps if Weinberg had been writing Quality Software Management in 2020, he would have called it the "Covid-19 Effect." No matter what it is called you'll still face a very big problem. It takes costly time and effort analyze the risks and make contingency plans. Most projects will be lucky and won't face disaster and some will but they contingency plans will work and make the problems seem smaller than they actually were. It won't take long and someone, probably your company's CFO, will say, "Do we really have to do all this expensive analysis and planning?" This is the disaster that will sink many future projects if you don't plan for it in advance by documenting the problems that such planning avoids and estimating the costs had contingency plans had not been in place. Furthermore, you must make disaster prevention and recovery a company culture value that everyone believes in and ensures it's success.

Reference:

Weinberg, G., Quality Software Management, Vol. 1: Systems Thinking, New York: Dorest House, 1992.

Use An Appropriate Process Model

Dozens of process models are available for software projects to utilize: agile, waterfall, SAFe, throwaway prototyping, incremental development, spiral model, etc. There is no such thing as a process model that works for all projects in an organization. Every project must select a process that makes the most sense for it. The selection should be based on corporate culture, risk willingness, application area, volatility of requirements, and the extent to which requirements are well understood.

Study your project's characteristics and select a process model that makes sense. For example, when building a prototype, you should follow a process that minimizes protocol, facilitates rapid development, and does not worry about checks and balances. When building a life-critical product, the opposite is true.

References

https://www.geeksforgeeks.org/software-processes-in-software-engineering/

Alexander, L., and Davis, A., "Criteria for the Selection of a Software Process Model," IEEE COMPSAC '91, Washington, DC: IEEE Computer Society Press.

Community Adoption of New Practices

Community adoption of new practices can mean a lot of things.  From getting a small team to try a new methodology or tool to convincing the world that a product or service is needed.

In 1996, Robert Metcalfe was awarded the IEEE Medal of Honor for "exemplary and sustained leadership in the development, standardization, and commercialization of Ethernet." The story of how Ethernet came to be follows the 8 steps of essential practices below. I've added headings for what departments typically handles each practice in a large company. You don't have to have different people or departments. I find it helpful to get into the mindset of each role when implementing a specific task.

• Product Management
• Sensing -- giving voice to a concern over a breakdown in the community
• Envisioning -- design a compelling story about a future without the breakdown
• Development
• Offering -- committing to do the work to produce that feature
• Sales
• Adopting -- gaining commitments from early adopters to join the innovation for a trial period
• Sustaining -- gaining commitments from majority adopters to join the innovation for an indefinite period
• Implementation
• Embodying -- working with the community until the new practice is fully embodied, ordinary, and transparent
• Navigating -- moving ever closer to the goal despite surprises, contingencies, and obstacles
• Marketing
• Mobilizing -- building a network of supporters of the innovation from within dispersed communities

Many times over my career I've been assigned the job of getting multiple software development teams to adopt a practice; coding standards, secure coding practices, SOX requirements, architectural guidelines, etc. Eventually, the above 8 practices were performed but not always in an order that was useful. I got better with experience, but my life would have been easier if I would have had those practices written down to guide me. To make notes against. And to revisit when progress was slow or stopped. I hope you'll save the above list and use it for your future innovations.

References

Denning, P., "Avalanches Make Us All innovators," Communications of the ACM, Vol. 63, No. 9 (Sept 2020), 32 - 34.

Sorry, I don't have a reference for Metcalfe's Ethernet story. I heard him talk about it at a conference in 2015 or 2016.

Top 10 Project Management Risks

As a software architect, I've assisted in project management and filled the role of product owner. For all of these roles it is important to be familiar with the situations that most often cause software disasters. These are your most likely risks, but not all of them:

• Personnel shortfalls
• Unrealistic schedules
• Not understanding the requirements
• Building a poor user interface
• Trying to gold-plate when the customer doesn't want it
• Not controlling requirements changes
• Shortfalls of reusable, interfaced components, or microservices
• Shortfalls in externally performed tasks
• Poor response time
• Attempting to exceed the capability of current computer technology

If you don't already have one, this list is a good starting point for a project planning checklist. Additionally, you should add risks unique to your environment, industry, and project then develop plans on how to mitigate them.

Reference:

Boehm, B., "Software Risks Management: Principles and Practices," IEEE Software, January 1991.

Quantify Requirements

All to often terms such as "fast," "responsive," and "extensible" are listed as software product requirements. Software architects often spend a lot of time helping product owners turn these desires into objective requirements. We do this by asking a lot of questions: How often? How many in what period? Increasing or decreasing at what rate?

It is a process that most product owners find maddening. Which is why it is important to explain that an exact answer is not expected. In fact, an exact answer would be a little worrying and I'd want to know how they got it. It is that process of how to get quantitative criteria that often needs to be explained and planned for. Finding ranges for quantitative requirements and checking against them is a time-consuming and expensive process. If no one wants to spend the time, effort, and money to do so then it reveals that the requirement isn't actually needed.

Focus your efforts on the parts of the system that stakeholders actually consider worth paying for.

Communication Skills Are Essential

When recruiting personnel for your project, don't underestimate the importance of teamwork and communication. The best software architect becomes a poor asset if he or she is unable to communicate, convince, listen, and compromise.

Communication breakdowns can occur at any process level. The effects of these problems are not independent. For instance, fluctuating requirements increase a development team's need for communication both with customers and with the project's other teams.

Exceptional architects are skilled at communicating their technical vision to other project members. They usually possess exceptional communication skills and often spend much of their time educating others about the application domain and its mapping into computational structures. In fact, much of their design work is accomplished while interacting with others. The integrative role of an exceptional designer compounds itself. This happens because those perceived as most knowledgeable will become communication focal points, providing them more knowledge about the system to integrate into a more comprehensive model.


Reference:
Curtis, B., Krasner, H., and Iscoe, N., "A Field Study of the Software Design Process for Large Systems," Communications of the ACM, November 1988.

Problem Solving

Beware of the person who knows all the answers:

Suppose two politicians are running for president, and one goes through the farm section and is asked, "What are you going to do about the farm question?" And he knows right away - bang, bang, bang. Now he goes to the next campaigner who comes through. "What are you going to do on the farm problem?" "Well, I don't know. I used to be a general, and I don't know anything about farming. But it seems to me it must be a very difficult problem, because for twelve, fifteen, twenty years people have been struggling with it, and people say that they know how to solve the farm problem. And it must be a hard problem. So the way I intend to solve the farm problem is to gather around me a lot of people who know something about it, to look at all the experience that we have had with this problem before, to take a certain amount of time at it, and then to come to some conclusion in a reasonable way about it. Now, I can't tell you ahead of time what solution, but I can give you some of the principles I'll try to use - not to make things difficult for individual farmers, if there are any special problems we will have to have some way to take care of them," etc., etc., etc.

Now such a man would never get anywhere in this country, I think. It's never been tried, anyway. This is in the attitude of mind of the populace, that they have to have an answer and that a man who gives an answer is better than a man who gives no answer, when the real fact of the matter is, in most cases, it is the other way around. And the result of this of course is that the politician must give an answer. And the result of this is that political promises can never be kept. It is a mechanical fact; it is impossible. The result of that is that nobody believes campaign promises. And the result of that is a general disparaging of politics, a general lack of respect for the people who are trying to solve problems, and so forth. It's all generated from the very beginning (maybe - this is a simple analysis). It's all generated, maybe, by the fact that the attitude of the populace is to try to find the answer instead of trying to find a man who has a way of getting at the answer.

 —Richard Feynman

And make sure that person is not you.

Historic [Software] Preservation

At its most basic, historic preservation is about keeping old places alive, in active use, and relevant to the needs of communities today. In software engineering, historic preservation is called software maintenance. The only difference between them is that the word "places" is better represented by the word "systems".

At its most basic, software maintenance is about keeping old systems alive, in active use, and relevant to the needs of communities today.

Use Optimal Data Structures

The structure of data and the structure of programs manipulating that data are intimately interrelated. If you select the right data structures, your algorithms (and thus your code) become easy to write, and easy to read, and therefore easy to maintain. Read any book on algorithms or on data structures (they're one and the same!).
    When preparing to write a program, you should develop the algorithms and data structures together. Try two or three or more different pairs before you select the best one. And be sure to encapsulate the data structure in one component so that, when you later find a better structure, you can change it easily.


Reference:
Kernighan, B., and Plauger, P., The Elements of Programming Style, New York: McGraw-Hill, 1988.

Design Is Multidimensional

When designing a home, architects represent it in many ways to fully understand and convey its essence to builders, buyers of materials, and home buyers: elevations, floor plans, framing, trusses, electrical routing, plumbing routing, concrete shape, door and window framing details, and other points of view. The same is true of software design.
    A complete software design includes at least:

1. Packaging. Often drawn as a hierarchy chart, this captures "what is part of what?" It often implies data visibility. It also shows encapsulation, such as data and functions within objects.
2. Needs hierarchy. This captures "who needs whom?" Drawn as a network of components, arrows indicate which components need something. The needs might be data, logic, or any other information.
3. Invocation. This captures "who invokes whom?" Drawn as a network of components, arrows indicate which components "call," "interrupt," or "send messages to" others.
4. Processes. Sets of components are packaged together as asynchronous processes. These are copies of components that are running simultaneously with other processes. Zero, one, or more copies may exist at one time. This should also specify conditions that cause a process to be created, executed, stopped, and destroyed.

Reference:

Witt, B., Baker, F., and Merritt, E., Software Architecture and Design, New York: Van Nostrand Reinhold, 1994.

Provide All The Information The User Needs And Nothing More

A key part of the design process is the precise definition of each and every software component in the system. This specification will become the "public" part of the component. It must include everything a user needs, such as its purpose, its name, its method of invocation, and details of how it communicates with its environment. Anything that the user does not need should be specifically excluded. In most cases, the algorithms and internal data structures used should be excluded. For if these were "visible," users might utilize this information. Subsequent enhancement or modification then becomes profoundly more difficult because any change to the component has a cascading effect on all components that use it (related to encapsulation).


Reference:
Parnas, D., "A Technique for Software Module Specification with Examples," CACM (May 1972).

Use Efficient Algorithms

Knowledge of the theory of algorithm complexity is an absolute prerequisite for being a good designer. Given any specific problem, you could specify an infinite number of alternative algorithms to solve it. The theory of "analysis of algorithms" provides us with the knowledge of how to differentiate between algorithms that will be inherently slow (regardless of how well they are coded) and those that will be orders of magnitude faster. Dozens of excellent books exist on this subject. Every good undergraduate computer science program will offer a course on it.


Reference:
Horowitz, E. and Sahni, S., Fundamentals of Computer Algorithms, Potomac, MD: Computer Science Press, 1978.

Software Reliability Can Be Achieved Through Redundancy

In hardware systems, high reliability or availability (Specify Reliability Specifically) is often achieved through redundancy. Thus, if a system component is expected to exhibit a mean-time-between-failures of x, we can manufacture two or three such components and run them in either:

1. Parallel. For example, they all do all the work and, when their responses differ, one is turned off with no impact on overall system functionality.
2. Or cold standby. A backup computer might be powered on only when a hardware failure is detected in the operational computer.

Manufacturing cost is slightly more than doubled. Design cost increases slightly. Reliability increases exponentially.

    In software systems, we cannot use the same approach. If we make two copies of the same software, no increase in reliability will be achieved. In one fails, the other will as well. What can be done, however, is to design (using two different design teams) two versions of the software from the same requirements specification, and deploy them in parallel. Development cost doubles. Reliability increases exponentially. Notice that, in the case of hardware, design increases in cost only slightly, whereas software design cost (the primary cost of software) doubles. Ultrahigh reliability in software is very expensive (High Quality Software Is Possible).

Reference:

Musa, J., et al., Software Reliability, New York: McGraw Hill, 1987.

Keep It Simple

A simple architecture or a simple algorithm goes a long way toward achieving high maintainability. Remember KISS. Also, as you decompose software into subcomponents, remember that people have difficulty comprehending more than seven (plus or minus two) things at once. C.A.R. Hoare has said:

There are two ways of constructing a software design. One way is to make it so simple that there are obviously no deficiencies and the other is to make it so complicated that there are no obvious deficiencies.

Build Flexibility Into Software

A software component exhibits flexibility if it can be easily modified to perform its function (or a similar function) in a different situation. Flexible software components are more difficult to design than less flexible components. However, such components are more run-time efficient than general components and are more easily reused than less flexible components in diverse applications.

Build Generality Into Software

A software component exhibits generality if it can perform its intended functions without any change in a variety of situations. General software components are more difficult to design than less general components. They also usually run slower when executing. However, such components:

1. Are ideal in complex systems where a similar function must be performed in a variety of places.
2. Are more potentially reusable in other systems with no modification.
3. Reduce maintenance costs for an organization due to reduced numbers of unique or similar components. Think about the hassle of maintaining multiple different repositories and build plans.

When decomposing a system into its subcomponents, stay cognizant of the potential for generality. Obviously, when a similar function is needed in multiple places, construct just one general function rather than multiple similar functions. Also, when constructing a function needed in just one place, build in generality where it makes sense - for future enhancements.

Evaluate Alternatives

A critical aspect of all engineering disciplines is the elaboration of multiple approaches, trade-off analyses among them, and the eventual adoption of one. After requirements are agreed upon, you must examine a variety of architectures and algorithms. You certainly do not want to use an architecture simply because it was used in the requirements specification. After all, that architecture was selected to optimize understandability of the system's external behavior. The architecture you want is the one that optimizes conformance with requirements.

For example, architectures are generally selected to optimize throughput, response time, modifiability, portability, interoperability, safety, or availability, while also satisfying the functional requirements. The best way to do this is to enumerate a variety of software architectures, analyze (or simulate) each with respect to the goals, and select the best alternative. Some design methods result in specific architectures; so one way to generate a variety of architectures is to use a variety of methods.