Fast Software Performance

Here are some aspects of software development that should be considered when trying to improve an application's performance.

Write Better Algorithms

Ultimately, the performance of an application is based on how well is is written. If the program loops around all elements in an array, the JVM will optimize the array bounds-checking so that the loop runs faster, and it may unroll the loop operations to provide an additional speedup. But if the purpose of the loop is to find a specific item, no optimization in the world is going to make the array-based code as fast as a different version that uses a HashMap.

A good algorithm is the most important thing when it comes to fast performance.

Improving your Algorithms & Data Structure Skills | medium

Write Less Code

A small well-written program will run faster than a large well-written program. This is true in general of all programs, and it applies specifically to Java programs. The more code that has to be compiled, the longer it will take until that code runs quickly. The more objects that have to be allocated and deallocated, the more work the garbage collector will have to do. If the objects are allocated and retained, the longer a garbage collection cycle will take. The more classes that have to be loaded from disk in the JVM, the longer it will take for a program to start. The more code that is executed, the less likely it will fit in the hardware caches on the computer. And the more code that has to be executed, the longer it will take.

More code / more features is a "Death by a thousand cuts". A developer will add a very small check for a feature then other developers will do the same and suddenly the performance has regressed. The cycle is repeated in the next release, and the performance starts to be noticeable. A couple of times during the process, performance testing might hit some resource threshold - a critical point in memory use, or a code cache overflow, etc. In those cases, the regular performance tests might make it possible to fix what appears to be a major regression. But over time, as the small regressions creep in, it will be harder and harder to fix.

I'm not advocating that you never add new features or code. The benefits of enhancing your applications is clear. But be aware of the small trade-offs you are making.

Writing Less Damn Code | heydonworks
Write Less Code | codeahoy
The Best Code Is No Code At All | codinghorror

Optimization Best Practices

Software engineers are expected to be familiar with the coding constructs that are known to be bad for performance. When writing code it is not "premature optimization" to apply your knowledge of best practices. So long as you are able to write clean, straightforward code that is simple to read and understand then you don't need to worry that you are prematurely optimizing your code.

30 Java Programming Tips and Best Practices for Beginners | javacodegeeks

Look Elsewhere

If you are developing standalone applications that use no external resources, then the performance of that application is mostly all that matters. Once an external resource such as a database is added, then the performance of both programs is important. And in a distributed environment, say an application server, a load balancer, a database, and more, then the performance of the single application server might be the least of the performance issues. In such an environment, a structured approach must be taken towards all aspects of the system. CPU usage, I/O latencies, and throughput of all parts of the system must be measured and analyzed; only then can it be determined which component is causing the performance bottleneck.

Optimize for the Common Case

It is tempting to treat all performance aspects as equally important. But focus should be given to the common use case scenarios.

• Optimize code by profiling it with realistic usage and focusing on the operations taking the most time.
• Apply Occam's razor to diagnosing performance problems. The simplest explanation for a performance issue is the most conceivable cause; a performance bug in new code is more likely than a bug in the JVM or the operating system.
• Write simple algorithms for the most common operations in the application. In general, optimize for the majority of your users. That will give you good overall performance stats but might not be what you need. For example, if 10% of your users, make 90% of your profit then you should optimize for that 10%.

Finally, JVM Tuning

Now that you have gone through the previous aspects that affect program performance, you are in a good place to tune the JVM. 

Align Reputation With Organization

It is generally recognized that Japanese software engineers view software bugs differently than American software engineers. Although many factors influence this, one relates to the perception in Japan that an error in a product is a disgrace to the company, and a disgrace to the engineer. This works more effectively in Japan than in the United States because Japanese workers tend to remain in one company for their entire careers. The mind-set, however, is important regardless of employment longevity.

In general, when anybody finds an error in a software engineer's product, that engineer should be thankful, not defensive. To err is human. To accept, divine! When an engineering error is found, the person causing it should broadcast it, not hide it. The broadcasting has two effects:

1. It helps other engineers avoid the same error.
2. It sets the stage for future non-defensive error repair.

Consider some guidelines for reporting errors.

1. Ask a coworker to check your work. Someone who will give you time to contact the author and won't judge you if you are wrong.
2. Give the author time to verify your report of an error.
3. Discuss with the author how to best report the error so that the error gets fixed with as little drama as possible and so the right people are informed.

Reference:
Mizuno, Y., "Software Quality Improvement," IEEE Computer, March 1983.

Know Formal Methods

Formal methods are not easy without strong discrete mathematical skills. On the other hand, their use (even on the back of an envelope) can aid significantly in uncovering problems in many aspects of software development. At least one person on every project should be comfortable with formal methods to ensure that opportunities to building quality into the product are not lost. And that requirements are properly understood.

Many people think that the only way to use formal methods is to specify a system completely using them. This is not true. In fact, one of the most effective methods is to write a natural language specification first. Then attempt to write parts using formal methods. Just trying to write things more formally will help you find problems in the natural language. Fix the natural language and you now have a better document. Discard the formalism if desired after it has helped you.

Reference:
Hall, A., "Seven Myths of Formal Methods," IEEE Software, September 1990.

Stop When You Achieve Your Goal

Software engineers use many development techniques that correspond to a sub-goal of software development. For example, structured or object-oriented analysis has the goal of understanding the problem being solved, DARTS has the goal of a process architecture, and structured design has the goal of calling hierarchy. In each case a series of steps are followed. Do not be so taken in by the method that you forget your goal. This is called goals displacement. If you have achieved your goal then stop. On the other hand, if completing the process will fulfill additional goals then continue.

"Know-When" Is As Important As Know-How

Knowing how to use a technique well does not make it a good technique, nor does it make you a good engineer. The good engineer knows dozens of diverse techniques well and knows when each is appropriate for a project or a segment of a project.

When doing requirements engineering, understand which techniques are most useful for which aspects of your problem. When doing design, understand which techniques are most useful for which aspects of your system. When coding, pick the most appropriate language and framework.

Developing Great Product Management and Engineering Relationships

A healthy relationship between Product Management and Engineering is critical to build successful products. It’s also essential in creating a team where people want to work.

When it goes well, we’re two partners working shoulder to shoulder towards a shared mission. Each is grateful for the contribution of the other. When it goes badly, there’s finger pointing, ignored input, wasted work, frustrating processes, and resentment.

Here are the 4 keys to a healthy engineering and product management relationship.

Share Leadership and Credit

One of the top ways Product Managers damage relationships with engineers is by hoarding all the credit. They claim that they thought of all the ideas and make sure they’re always the ones presenting to executives, leading meetings, or announcing results. They act like they’re the most important person on the team.

Explicitly share leadership between product managers, engineers, and designers. Each role has a clear responsibility, but we all work together to share our points of view and come to better solutions together. We have different points of view, but we all know we’re on the same team working towards shared goals.

One specific way to share leadership responsibilities is through the concept of Program Lead (PL), who can be either a Product Manager or an Engineer on the team. PLs own communicating progress, establishing the rhythm for the team, and team unity. The Program Lead can be the same person as the Tech Lead, but on some teams, they are different engineers — one person is responsible for running the team, while the other person is responsible for the technical direction.

Another way to share leadership and credit is with Key Results. These are team goals that get shared across the company. We assign them to the person who contributes the most towards their success, whether it’s an engineer, PM, designer, user researcher, or data scientist.

With this system, more people get to take on leadership roles and everyone gets recognition for their work.

Include Engineers in Product Decisions

Some PMs think it’s their job to have all the ideas and do all the product thinking. They’ll go off with their designers and come up with a grand vision to toss over the wall to the engineers for implementation. If an engineer comes up with a proposal, they’ll quickly dismiss it — either out of a lack of respect or being territorial.

Engineers need to be involved and can lead product direction from the beginning of the product lifecycle. Gather input from across the company when planning a roadmap and include all the engineers on a team in research and design brainstorming at the beginning of each project.

Engineers appreciate understanding the reasoning behind product decisions. PM’s love when engineers come up with valuable ideas enabled by creative technical solutions. We know that aligning everyone on the team around a shared purpose and shared understanding will help achieve bigger and better things together.

Clarify Roles and Reinforce them with Mutual Respect

An easy way to muddle any work relationship is with unclear roles and lack of respect. PMs insult engineers by trying to tell them how the code should work, or endanger the schedule insisting on frivolous features with outsized costs.

Focus on clarity of purpose, plan, and responsibility. PMs own the problems and Engineers own the solutions. For team leadership responsibilities that can vary from team to team, go a step further and fill out a checklist so it’s clear for each responsibility whether the PM or Program Lead will take it.

Mutual respect is really in the cultural norms. Never have a PM sign an engineer up for a deadline without their buy-in. Engineers take the time to explain technical challenges to PMs and really care about the customer impact of our work. Consider providing all new employees with some training that reinforces the idea to approach situations with curiosity rather than a need to be right.

Clear Away Inefficiency and Help Your Teammates Succeed

Some engineers think that PMs just add bureaucracy and inefficiency. They’ve worked with PMs who try to solve every problem with more meetings and who burden engineers with work about work without convincing them that it’s worth it. Even worse, those PMs thrash projects in the middle because they’ve changed their minds, wasting weeks or months of work.

PMs need to be a productivity multiplier. Encourage them to look for every kind of creative way to help our teammates be more efficient and effective. Develop a product process to ensure everyone really understands the customer problem you’re going after and are all on the same page about goals at the beginning to avoid unnecessary changes later.

Healthy relationships between PMs and Engineers are critical to the success of any project. With shared leadership, inclusive decision making, mutual respect, and a commitment to helping each other succeed, you can build a team that delivers results and attracts great people.

Software Tools Are Expensive

Software tools can be expensive. You have to factor in the cost of the tool, the hardware to run the tool on, and the training of your employees to use the tool. But consider the higher costs of not buying the tools. Lower productivity, higher probability of customer dissatisfaction, delayed product release, increased rework, poorer produce quality, etc.


Reference:
Huff, C., "Elements of a Realistic CASE Tool Adoption Budget," CACM, April 1992.

Give Software Tools To Good Engineers

Users of software tools become more productive. However, a tool cannot convert a poor software engineer (one that produces code that is unreliable, incomplete, and so on) into a good one. Thus, you want to give tools only to good engineers. The last thing you want to do is to provide tools to the poor engineers: You want them to produce less, not more, poor-quality software.

Use Tools, But Be Realistic

Software tools can make their users more efficient and should be used. Remember that the hard work (the thinking) is not done by the tool. Use tools but be realistic concerning the effect on productivity.


Reference:
Kemerer, C., "How the Learning Curve Affects Tool Adoption," IEEE Software, May 1992.

Technique Before Tools

"An undisciplined software engineer with a tool becomes a dangerous undisciplined software engineer."

A software engineer must first understand and be able to follow an appropriate software technique. Next, must know how to use the tool before using a new tool.

Software engineers should first attempt any new software techniques by hand to convince their self and their management that the technique before investing time and money in tools to automate the technique. If the technique doesn't work without automation, it won't work with automation.


Reference:
Kemerer, C., "How the Learning Curve Affects Tool Adoption," IEEE Software, May 1992.