Problem Solving

Beware of the person who knows all the answers:

Suppose two politicians are running for president, and one goes through the farm section and is asked, "What are you going to do about the farm question?" And he knows right away - bang, bang, bang. Now he goes to the next campaigner who comes through. "What are you going to do on the farm problem?" "Well, I don't know. I used to be a general, and I don't know anything about farming. But it seems to me it must be a very difficult problem, because for twelve, fifteen, twenty years people have been struggling with it, and people say that they know how to solve the farm problem. And it must be a hard problem. So the way I intend to solve the farm problem is to gather around me a lot of people who know something about it, to look at all the experience that we have had with this problem before, to take a certain amount of time at it, and then to come to some conclusion in a reasonable way about it. Now, I can't tell you ahead of time what solution, but I can give you some of the principles I'll try to use - not to make things difficult for individual farmers, if there are any special problems we will have to have some way to take care of them," etc., etc., etc.

Now such a man would never get anywhere in this country, I think. It's never been tried, anyway. This is in the attitude of mind of the populace, that they have to have an answer and that a man who gives an answer is better than a man who gives no answer, when the real fact of the matter is, in most cases, it is the other way around. And the result of this of course is that the politician must give an answer. And the result of this is that political promises can never be kept. It is a mechanical fact; it is impossible. The result of that is that nobody believes campaign promises. And the result of that is a general disparaging of politics, a general lack of respect for the people who are trying to solve problems, and so forth. It's all generated from the very beginning (maybe - this is a simple analysis). It's all generated, maybe, by the fact that the attitude of the populace is to try to find the answer instead of trying to find a man who has a way of getting at the answer.

 —Richard Feynman

And make sure that person is not you.

Provide All The Information The User Needs And Nothing More

A key part of the design process is the precise definition of each and every software component in the system. This specification will become the "public" part of the component. It must include everything a user needs, such as its purpose, its name, its method of invocation, and details of how it communicates with its environment. Anything that the user does not need should be specifically excluded. In most cases, the algorithms and internal data structures used should be excluded. For if these were "visible," users might utilize this information. Subsequent enhancement or modification then becomes profoundly more difficult because any change to the component has a cascading effect on all components that use it (related to encapsulation).


Reference:
Parnas, D., "A Technique for Software Module Specification with Examples," CACM (May 1972).

Use Efficient Algorithms

Knowledge of the theory of algorithm complexity is an absolute prerequisite for being a good designer. Given any specific problem, you could specify an infinite number of alternative algorithms to solve it. The theory of "analysis of algorithms" provides us with the knowledge of how to differentiate between algorithms that will be inherently slow (regardless of how well they are coded) and those that will be orders of magnitude faster. Dozens of excellent books exist on this subject. Every good undergraduate computer science program will offer a course on it.


Reference:
Horowitz, E. and Sahni, S., Fundamentals of Computer Algorithms, Potomac, MD: Computer Science Press, 1978.

Write Programs For People First

In the early days of computing, computers were relatively slow. Almost anything that could be done to shave off a few instructions was worth the effort. The most efficient use of any of the resources on the very expensive computer system was the major goal. Things have changed. The most valuable resources is now people: people to develop the software, people to maintain the software, and people to enhance capability. With few application exceptions, programmers should think first of the people who will later attempt to understand and adapt the software. Anything that can be done to assist them should be done.¹ Efficiency is also important,² but they are not mutually exclusive. If you need efficiency, that's fine but upgrade the readability of your program so that you don't lose the humans in the process.

1. Avoid Tricks
   Avoid Global Variables
   Write To Read Top-Down
   Avoid Side-Effects
   Use Meaningful Names
2. Evaluate Alternatives
   Use Efficient Algorithms
   Get It Right Before You Make It Faster

Reference:
McConnell, S., Code Complete, Redmond, WA: Microsoft Press, 1993.

Use Meaningful Names

Some programmers insist on naming variables with names like nflt or nFlight. The usual argument is that it makes programmers more productive because of reduced key presses. Good programmers should spend a very small percentage of their time typing (maybe 10 to 15 percent); most time should be spent thinking. So how much time is really being saved?

A better argument is that overly shortened names actually decrease productivity. There are two reasons:

1. Testing and maintenance costs rise because people spend time trying to decode names, and
2. more time is spent typing comments to explain the shortened names!

Reference:

Ledgard, H., Programming Proverbs, Rochelle Park, NJ: Hayden Book Company, 1975.

Avoid Side-Effects

A side-effect of a procedure is something the procedure does that is not its main purpose and that is visible (or whose results are perceivable) from outside the procedure. Side-effects are the sources of many subtle errors in software, that is, the ones that are the most latent and the ones that are most difficult to discover once their symptoms manifest themselves.


Reference:
Ledgard, H., Programming Proverbs, Rochelle Park, NJ: Hayden Book Company, 1975.

Write To Read Top-Down

People generally read a program from top to bottom. Write your programs to help others understand them. Among the implications of this principle are:

1. Include a detailed external specification up front to clearly define the program purpose and use.
2. Specify externally accessed routines, variables, and algorithms up front.
3. Use the "structured" programming constructs, which are inherently easier to follow.

Reference:

Kernigham, B., and Plauger, P., The Elements of Programming Style, New York: McGraw-Hill, 1978.

Avoid Global Variables

Global variables make it convenient to write programs; after all, if you need to access or change x, you just do it. Unfortunately, if x is ever accessed and found to have an inappropriate value, it is difficult to determine which software component is at fault. "Global" implies that anybody could have altered its value incorrectly.

As an alternative, encapsulate important data in its own module, so that anybody who wants to change it or access it must do so by means of that routine. Alternatively, explicitly pass parameters to routines that need specific data. If you find an excessive number of parameters, perhaps your design needs to be reworked.


Reference:
Ledgard, H., Programming Practice, Vol II, Reading, MA: Addison-Wesley, 1987.

Avoid Tricks

Many programmers love to create programs with tricks. These are constructs that perform a function correctly, but in a particularly obscure manner. Typically, they use a side-effect of a function to implement a primary function. Programmers see these as "clever," but, as Allen Macro points out, they "are often merely the stupid use of high intelligence."
    There are many ways to explain why tricks are used so often:

1. Programmers are extremely intelligent and want to demonstrate that intelligence.
2. Maintainers, when they finally figure out how the trick works, will not only recognize how smart the original programmer was, but also will realize how smart they themselves are.
3. Job security.

Bottom line: Show the world how smart you are by avoiding tricky code!

Reference:

Macro, A., Software Engineering: Concepts and Management, Englewood Cliffs, NJ: Prentice-Hall International, 1990.

Store Requirements in a Database

Keep in mind that the follow quote is from 1995. It is still completely relevant.

Requirements are complex and highly volatile. For these reasons, storing them in electronic media, preferably a database, is a good idea. This will facilitate making changes, finding implications of changes, recording attributes of specific requirements, and so on.
      Some of the things you want to store in the database are unique identifier (Separately Number Every Requirement), the text of the requirement, its relationship to other requirements (such as more abstract or more detailed descriptions of the requirement), importance (Prioritize Requirements), expected volatility, pointers to its sources (Record Why Requirements Were Included), applicable product versions (Identify Subsets and Give Every Intermediate Product a Name and Version*), and so on. Ideally, the requirements specification itself is nothing but an organized "dump" of the entire database.

I have used JIRA. There are many tools like it. They can do all of the above, but it is up to the person creating a ticket in JIRA to ensure those things are done. I think they are really important so I'm going to list them again.

• Create the ticket in a backlog or project that is unique. Do not lump unrelated tickets together. This will result in a "fatlog" - a fat backlog that is difficult to groom and manage.
• The text of the requirement goes in the "description". The "summary" is not where the requirement should be detailed. Personally, I think systems like JIRA should not allow tickets that have no description to be saved. Engineers should refuse to estimate such tickets and Product Owners who create them should be ashamed.
• JIRA has a feature to link tickets. Use it. If there is a dependency then utilize the sub-task feature. Make a story that encapsulates both tickets and make them linked sub-tasks. Use the system to track if everything for that requirement is actually done.
• Order the tickets by importance. There can only be one highest priority and it at the top.
• Use comments to document the sources/stakeholders for a ticket. Do not use comments to have a discussion about a ticket. Email, call, instance message, etc with the people involved. If you absolutely must use the comments section of a ticket to have a discussion then clean it up later. Delete anything that will mislead people reading the ticket. If you want to save the history then move that stuff into a document and attach it to the ticket with a title that makes it clear that it is for history and it is not the requirement. The actual requirement is in the description.
• Ensure that you put what product the ticket applies to either in the summary or the description. You might be inclined to think that it is clear by where the ticket sits. For example, the ticket is in the product A backlog. Invariability, someone is going to get confused and try to work that ticket for product C which is really similar to product A. I've also seen people accidentally move a ticket to a different backlog then not be able to figure out where it belonged.
• If you can't take the issues in a JIRA backlog and print them out in order and call that your software requirements specification (SRS) then your backlog is a mess and you have some work todo.

6 Tips to Make Your Backlog Lean

Self-Destruct TBDs

A software requirements specification (SRS) with a TODO or TBD (To Be Determined) note is obviously not complete. There may be good reasons for approving and starting work on a project with a TBDs in the specification. This is particularly true for requirements whose precision are not critical to fundamental design decisions.

When you create a TBD, be sure to footnote it with a "self-destruction note," that is, specify who will resolve the TBD and by when.


Reference:
IEEE, ANSI/IEEE Guide to Software Requirements Specifications, IEEE Computer Society Press, Washington, DC, 1994.

Strive To Fix What You Can

I will admit to sometimes getting frustrated at work when someone doesn't want to work in a specific codebase because they think it has too many problems. George Bernard Shaw said, "If there were nothing wrong in the world, there wouldn't be anything for us to do." The systems that need the most work don't just validate the need for developers but are an opportunity to make major improvements.

I had a friend in grad school who went to work fixing and upgrading Cobol applications at a large financial company. Much of the software had no major maintenance in years. He loved it. He said that everywhere he looked he found things to improve that made him look like a rock star.

When you find yourself in that legacy application that is out-of-date and has little to no documentation. Strive to make it better and people will notice. You might even change people's perceptions of the application.

Quality is #1

A product with poor quality should not be tolerated. Delivering a product at the cost of quality should only be accepted in the short-term. If you are using Agile Methodologies, in the span of a sprint, it is acceptable to deliver what you have at the end of the sprint. This will allow for continued progress and enable the gathering of valuable feedback. However, it is political suicide to continue to deliver a product with poor quality. If there is a quality issue then change your roadmap and milestones to address the quality issue.

Not Even In The Same Ballpark

Not so long ago, a new Product Manager told me that she also is a programmer... a software engineer. She implied that if she wanted she could be part of the development team. She had just started and I had no reason not to believe her and accepted it as an interesting side note and went on. As time has gone by it has become increasingly apparent that this individual was never a software engineer in the way that my peers are. The comparison is almost offensive in the stark difference and extreme lack of understanding of what a software engineer does.

To anyone reading this:
Please do not say that you are part of someone else's group unless you truly know about that person's group. I've brewed beer and cider a handful of times. I've even had the lucky experience of brewing some stuff that is better than some store bought stuff. But I would never call myself a brewer in the presence of someone like Sam Calagione or Jamil Zainasheff. I've never brewed commercially and have no concept of the constraints involved, and I don't have near the same range of experiences. Notice the difference in tone the following two statements have:

• I'm a brewer like you.
• I've brewed at home.

If they both seem innocuous then here are some example responses, respectively:

• Oh yeah, what bars are serving your beer?
• Since I have never brewed commercially, I do not grok the need for high efficiency sparging, dealing with distribution laws, etc.
• That's cool. What is your favorite style.
• I love Bock beer. And Belgium style. And Kolsch. And Scotch ale. And, oh man, Southern Tier's Creme Brulee stout is just epic. But I mostly make cider because I'm decent at it.

Of course, those responses could go any number of ways but the point is that you can share that you have some shared experience without implying your of the same level. Especially, when you have no idea. Heck, you might be better but you just met so you have no idea.

In short, just be genuine. If you were hired to do a job, focus on that. Don't worry about trying to "fit in" with people in other roles.

Managing Technical Debt

Technical debt is all the shortcuts that save money or speed up progress today at the risk of costing money or slowing down progress in the future. It is inevitable, and can even be a good thing¹ as long as it is managed properly, but this can be difficult. The difficultly comes from many causes, usually has hard-to-predict effects, and almost always involves a gamble about what will happen in the future. Managing technical debt is similar to risk management, and similar techniques can be applied. If technical debt is not managed then it will tend to build up over time until a crisis results. It can be the catalyst to system failure, or a multiplier of failures.

Technical debt can be viewed in multiple ways and can be caused by all levels of a company. It can be managed properly only with the understanding and assistance from all levels. It is extremely important to help nontechnical parties understand the costs that can arise from mismanaging that debt.

1. Releasing code early to get feedback sooner rather than later can save a company from investing heavily in something that is not desired. Fast feedback helps development stay on track with the customer's desires.

Web Development Best Practices

The following are techniques for maximizing website performance on mobile devices. Personally, since more and more mobile devices are being used to browse content, I think all Web development should be done against the constraints of mobile. Please don't make a "full website" and a "mobile website". Those setups never look good or function well. Furthermore, redirecting a mobile request to a mobile site is expensive and slow. If you are dead set on the "full website" and "mobile website" configuration then make mobile the default and redirect to the full site if the request is not from a mobile device.

A mobile site needs to compensate when bandwidth or service becomes spotty. In other words, the site needs to be responsive even when the network is not. The same can be said for regular websites. Especially if you are trying to break into areas with questionable Internet service, e.g. Bolivar, Missouri, or questionable firewalls, e.g. China.

Eliminate HTTP Requests and Round Trips

• Use CSS sprites to represent images embedded as inline data:URLs.
• Use HTML5 application cache (app cache) to force the browser to cache all the unchanging content.
• Insert dynamic content in the document via XHR (XMLHttpRequest).
• Most effective if you can design the prefix of your page to fit into a single packet, 1492 bytes, render a basic framework for the page before any script is executed. To give the user immediate feedback that the page is loading, load the framework of the page as a fade effect by providing an opacity transitions from 0.0001 to 1.0 just after the framework is loaded. If you cannot get the basic framework of your page to fit into the first packet, then you can instead load a spinner or a logo, again as an opacity transition from 0.0001 to 1.0. The spinner or logo should be conditionally loaded after checking whether the app cache is already populated.
• <script>
    if (window.applicationCache.status == 0) { // Reveal the spinner
  } else {
    // Page was loaded from app cache. Bring out the kitchen sink.
  }
  </script>

• Make changes to minimize DNS lookups and redirects.

Use Compression

• nuff said.

Manage JavaScript Parse Time

• Move the vast majority of script to the bottom of the page.
• Allow the browser to load your JavaScript without recognizing it as script and defer parsing and initial evaluation until needed.
• To do this, set the type of the script to an unrecognized type and change it to text/JavaScript later. For example,
  <script type="deferred" id="module1">
    // deferred code here
  </script>
  When you are ready to use the script, reference the script tag by the id and change the type. The browser will parse and evaluate the script at that time.

Avoid Layout and Style Calculation

• Avoid reading properties that rely on the position of elements on the page to be returned. Any such property could cause the browser to recalculate styles on demand and return the value to your script.

Monitor Request Size

• Reduce the number of cookies you are using.
• Ideally, all requests should be smaller than a TCP packet.

Preload Components

• When the odds are good that you know what a user will request next then it is time to preload.
• This can be seen in Google's Calendar website. The next day's events are loaded as the user clicks through each day, giving the effect that all of the data is already loaded.

Optimize Images

• It is worth the time and effort to experiment with finding a good balance of image compression vs image quality.

Common Recommendations with Questionable Consequences

Make JavaScript and CSS External?

• Inlining all your JavaScript and CSS into each webpage will deliver the best results in terms of speed. For maintenance reasons, you could use XHRs to fetch the JavaScript or CSS and the HTML5 database to store the resource for later reuse.

Make AJAX Cacheable?

• Skip trusting the browser to cache AJAX responses. You're better off to build a full-blown write-through cache layer or a XHR caching layer on top of the HTML5 database.

Pair Programming

I'm a big fan of pair programming. I have used it many times and almost everytime the result was better code that was developed faster. Pair programming is like having continuous code review.

Sadly, there is a lot of controversy with this particular agile/XP practice. Seeing two valuable resources sitting down at one computer makes most managers nervous. They think their team's productivity has just been halved. The worst thing about this mentality is that it makes it clear that the manager thinks programming is just typing.

Mere typing, programming is not. A good idea or a bug caught early on in the process can save a team a lot of work and rework later. By pairing you share valuable knowledge and practices throughout the team, catching more bugs early, and increasing code quality.

Merciless Refactoring

Many people are afraid to make code changes out of fear that they will break the code. To ensure you can safely make code changes you should change it often. I'm not talking about pointless changes but solid improvements. Run FindBugs against your code and fix the issues. If you disagree with FindBugs then use PMD or any static analysis tool. However, if static analysis suggests that your code looks like a bug then consider changing it to look less dangerous.

Anyway, the point of merciless refactoring is to always be improving your code. To make changes and ensure that it was easy to make those changes. If it isn't fast and easy then you need to make changes to make it fast and easy. Write unit tests, delete unused code, optimize loops, automate tests; everything you can do to make it possible to complete your next project in record time.

Merciless refactoring can also be described as following the boy scouts rule of always leaving a place cleaner than when you found it. If you add a method to a class then add some JavaDocs to the other methods while you're there. If you change a single line of code in a method, take a look at the entire class and fix it up if you can.

Donald Knuth

Dr. Dobb's Journal published an Interview with Donald Knuth in April of 1996. It was a very good read.

Knuth discusses what distinguishes a "computer scientist" from a "computer programmer" but neatly side steps actually separating the two terms. Choosing to instead explain how computer science is about thinking about problems in a different way. (You'll have to ready the article to see what I mean.)

There is some discussion about Knuth's The Art of Computer Programming. Which Charles Moss tells me is a wonderful read. I'm not sure I believe that but the MSU library has volumes 1-3 in case I want to check them out.

I must quote this because one of my coworkers is just fanitical about C++. To him, everything that is not C++ is crap with the exception of javascript and ruby.

DDJ: You've mentioned Edsgar Dijkstra. What do you think of his work? DK: His great strength is that he is uncompromising. It would make him physically ill to think of programming in C++.

Kunth followed that quote up with some very reassuring comments about trying to find the middle ground between very specific control of code and allowing a library to be used without knowing how it was implemented.

[W]hen you write a program, think of it primarily as a work of literature. You're trying to write something that human beings are going to read. Don't think of it primarily as something a computer is going to follow. The more effective you are at making your program readable, the more effective it's going to be: You'll understand it today, you'll understand it next week, and your successors who are going to maintain and modify it will understand it.