Big O Notation

Yaacov Apelbaum-big-o-and-efficiency

Recently, I was chatting with a friend of mine about pre-acquisition due diligence.  Charlie O’Rourke is one of the most seasoned technical executives I know. He’s been doing hardcore technology for over 30 years and is one of the pivotal brains behind FDC’s multi-billion dollar payment processing platforms.  The conversation revolved around a method he uses for identifying processing bottlenecks.
 
His thesis statement was that in a world where you need to spend as little as you can on an acquisition and still turn profit quickly, problems of poor algorithmic implementations are “a good thing to have”, because they are relatively easy to identify and fix.  This is true, assuming that you have his grasp of large volume transactional systems and you are handy with complex algorithms.

In today’s environment of rapid system assembly via the mashing of frameworks and off-the shelf functionality like CRM or ERP, the mastery of data structures by younger developers is almost unheard of.

It’s true, most developers will probably never write an algorithm from scratch. But sooner or later, every coder will have to either implement or maintain a routine that has some algorithmic functionality. Unfortunately, when it comes to efficiency, you can’t afford to make uninformed decisions, as even the smallest error in choosing an algorithm can send your application screaming in agony to Valhalla.

So if you have been suffering from recursive algorithmic nightmares, or have never fully understood the concept of algorithmic efficiency, (or plan to interview for a position on my team), here is a short and concise primer on the subject.

First let’s start with definitions.

Best or Bust:
An important principal to remember when selecting algorithms is that there is no such thing as the “best algorithm” for all problems. Efficiency will vary with data set size and availability of computational resources (memory and processor).  What is trivial in terms of processing power for the NSA, could be prohibitive for the average Joe.

Efficiency:
Algorithmic efficiency is the measure of how well a routine can perform a computational task. One analogy for algorithmic efficiency and its dependence on hardware (memory capacity and processor speed) is the task of moving a ton of bricks from one location to another a mile a way.  If you use a Lamborghini for this job (small storage but fast acceleration), you will be able to move a small amount of bricks very quickly, but the down side is that you will have to repeat the trip multiple times.  On the other hand, if you use a flatbed truck (large storage but slow acceleration) you will be able to complete the entire project in a single run, albeit at slower pace.

Notation:
The expression for algorithmic efficiency is commonly referred to as “Big O” notation.  This is a mathematical representation of how the algorithm grows over time. When plotted as a function, algorithms will remain flat, grow steadily over time, or follow varying curves.

The Pessimistic Nature of Algorithms:
In the world of algorithm analysis, we always assume the worst case scenario.  For example, if you have an unsorted list of unique numbers and it’s going to take your routine an hour to go through it, then it is possible in the best case scenario that you could find your value on the first try (taking only a minute). But following the worst case scenario theory, your number could end up being the last one in the list (taking you the full 60 minutes to find it). When we look at efficiency, it’s necessary to assume the worst case scenario.

 Yaacov Apelbaum-big-o Plot
Image 1: Sample Performance Plots of Various Algorithms

O(1)

Performance is constant for time (processor utilization) or space (memory utilization) regardless of the size of the data set size. When viewed on a graph, these functions show no-growth curve and remain flat.

O(1) algorithm’s performance is also independent of the size of the data set on which it operates.

An example of this algorithm is testing a value of a variable based on some pre defined hash table.  The single lookup involved in this operation eliminates any growth curves.

O(n)
Performance will grow linearly and in direct proportion to the size of the input data set.  The algorithm’s performance is directly related to the size of the data set processed. 

O(2N) or O(10 + 5N) denote that some specific business logic has been blended with the implementation (which should be avoided if possible).

O(N+M) is another way of saying that two data sets are involved, and that their combined size determines performance.

An example of this algorithm is finding an item in an unsorted list or a Linear Search that goes down a list, one item at a time, without jumping.  The time taken to search the list gets bigger at the same rate as the list does.

O(nn)
Performance will be directly proportional to the square of the size of the input data set.  This happens when the algorithm processes each element of a set and that processing requires another pass through the set (this is the square value). Processing a lot of inner loops will also result in the form O(N3), O(N4), O(Nn.).

Examples of this type of algorithm are Bubble Sort, Shell Sort, Quicksort, Selection Sort or Insertion Sort.

O(2N)
Processing growth (data set size and time) will double with each additional element of the input data set. The execution time of O(2N) can grow exponentially.

The 2 indicates that time or memory doubles for each new element in data set.  In reality, these types of algorithms do not scale well unless you have a lot of fancy hardware.

O(log n) and O(n log n) 
Processing is iterative and growth curves peak at the beginning of the execution and then slowly tapper off as the size of the data sets increases.  For example, if a data set contains 10 items, it will take one second to complete; if the data set contains 100 items, it will takes two seconds; if the data set containing 1000 items, it will take three seconds, and so on. Doubling the size of the input data set has little effect on its growth because after each iteration the data set will be halved. This makes O(log n) algorithms very efficient when dealing with large data sets.

Generally, log N implies log2N, which refers to the number of times you can partition a data set in half, then partition the halves, and so on.  For example, for a data set with 1024 elements, you would perform 10 lookups (log21024 = 10) before either finding your value or running out of data.

Lookup # Initial Dataset New Dataset
1 1024 512
2 512 256
3 256 128
4 128 64
5 64 32
6 32 16
7 16 8
8 8 4
9 4 2
10 2 1

A good illustration of this principal can be found in the Binary Search, it works by selecting the middle element of the data set and comparing it against the desired value to see if it matches. If the target value is higher than the value of the selected element, it will select the upper half of the data set and perform the comparison again. If the target value is lower than the value of the selected element, it will perform the operation against the lower half of the data set. The algorithm will continue to halve the data set with each search iteration until it finds the desired value or until it exhausts the data set.

The important thing to note about log2N type algorithms is that they grow slowly. Doubling N has a minor effect on its performance and the logarithmic curves flatten out smoothly.

An example of these type of algorithms are Binary Search, Heap sort, Quicksort, or Merge Sort

Scalability and Efficiency
An O(1) algorithm scales better than an O(log N),
which scales better than an O(N),
which scales better than an O(N log N),
which scales better than an O(N2),
which scales better than an O(2N).

Scalability does not equal efficiency. A well-coded, O(N2) algorithm can outperform a poorly-coded O(N log N) algorithm, but this is only true for certain data set sizes and processing time. At one point, the performance curves of the two algorithms will cross and their efficiency will reverse.

What to Watch for when Choosing an Algorithm
The most common mistake when choosing an algorithm is the belief that an algorithm that was used successfully on a small data set will scale effectively to large data sets (factor 10x, 100x, etc.).

For most given situations, an O(N2) algorithm like Bubble Sort will work well. If you switch to a more complex O(N log N) algorithm like Quicksort you are likely to spend a long time refactoring your code and will only realize marginal performance gains.

More Resources
For a great illustration of various sorting algorithms in live action form, check out David R. Martin’s animated demo.  For more informal coverage of algorithms, check out Donald Knuth’s epic publication on the subject The Art of Computer Programming, Volumes 1-4.

If you are looking for some entertainment while learning the subject, check out AlgoRythimic’s series on sorting through dancing.

 

 

© Copyright 2011 Yaacov Apelbaum All Rights Reserved.

Advertisements

Just Say No to Features

Yaacov Apelbaum-Just Say No

A quick feature inventory of most “mature” commercial software products like Microsoft Word, Lotus Notes, etc. reveals that more than half of their features are either never accessed or are outright useless (the historical evolution of the Microsoft Word tool ribbon is an example of feature bloat).  If you are developing software in a startup, useless features will be the death of you and your product.  The secret to developing the right set of features is learning how to say no to the rest.

Yaacov Apelbaum - Word 1.0 ribbon

Yaacov Apelbaum - Word 6.0 ribbon

Yaacov Apelbaum- Word 2003 ribbon

Yaacov Apelbaum - Word 2003 ribbon

Yaacov Apelbaum - Word 2013 ribbon

Figure 1. Bloat progression in Microsoft Word from version 1.0 to 2013

Whenever you say “yes” to a new feature, you agree to adopt a baby. And with the baby come such responsibilities as changing it’s diaper, waking up at 3 AM to feed it, and paying for its education (e.g. architecting, developing, integrating, and testing).

To hit the market early and within budget, your feature development strategy must focus on creating the bare essential functionality. So in this vein, stay away from developing a Swiss arm knife with dozens of capabilities solutions.  You should make each feature prove itself to be a “worthy survivor”. Your features need to be tough, lean, and mean.  Embrace the SEAL’s “hell week” screening approach before letting any one of them into your development cycle.

Yaacov Apelbaum - Swiss Army Knife Bloat

Whenever product, sales, marketing, or even the CEO requests a feature – it should instinctively meet a resounding no. If you don’t feel comfortable saying no, try a variation along the lines of “not now honey, I have a splitting headache” or “I’d love to chat about this feature, but I’ve got to run into a day long meeting right now”. If you are cornered and can’t escape, listen and take some notes, but stop there. There is no need to be heroic or do anything just yet.

If a request for a particular feature keeps surfacing over and over again from different sources, that’s when you know it’s time to take a closer look at it.  Only at this point should you start considering it seriously.

What do you say to the product team that complains vocally that you won’t adopt the top one hundred features on their wish list? Remind them that developing even the most basic feature is prohibitively expensive.  Alternatively, you can use the following list of tasks to illustrate the amount of work needed to insert a simple image on a MVC CMS driven web page:

To develop this feature, you need to (times are in minutes)…

  1. Hold a management meeting to discuss it (30 min.)
  2. Hold a feasibility meeting with the technical teams (30 min.)
  3. Develop a schedule and if you use SCRUM work the feature into a sprint (30 min.)
  4. Develop screen wireframes (60 min.)
  5. Develop HTML mockups (60 min.)
  6. Develop a POC (120 min)
  7. Allocate resources that are working on other features (30 min.)
  8. Develop an analysis and design document for system, CMS, DB impact, etc. (30 min.)
  9. Create a code branch (30 min.)
  10.   Write unit tests (120 min.)
  11.   Write the code (120 min.)
  12.   Peer review the code (30 min)
  13.   Merge the code back into trunk (30 min.)
  14.   Test, revise, test, revise…(60 min.)
  15.   Modify user documentation (30 min.)
  16.   Update the product guide (30 min.)
  17.   Update the marketing and engineering collateral (60 min.)
  18.   Refactor product cost and check to see if pricing structure is affected (60 min.)
  19.   Deploy to production (30 min.)
  20.   Cross you fingers and hope that everything worked out as planed

Total development cost = two days of work

As you can see from these steps, even the most basic feature request can cost two days of planning, design, development, testing, and documentation effort.  On average, feature tend to be 5 to 10 times more complex.

Sometime User Feature Request Can be Evil 
What about using your customers as the main driver for feature development?  Well, this strategy has several small flaws as well.  Your customers want everything under the sun, yesterday, and for free.  You should fault users for making feature requests. I encourage our customers to speak up and I diligently collect their input.  In the end, most of our functionality comes from user requests.  But as I have indicated, even for user feature requests, my first response is to always say no.  So what do I do with all these new customer feature requests? I read them, try to forget them and finally, for safe measure, I shred them.

It sounds terrible, I know, but don’t worry, if the feature request is important enough, it will keep on coming back and you won’t have any difficulty remembering it then. These persistent features will be the ones essential to your products that you will need to prioritize and implement.

It may not be apparent, but most software feature surveys revolve around questions like “What features are missing in our product?” or “What what would be the one feature you couldn’t live without?” or “What would make this product a killer app?”. Rather than continue to ask for more in this way, the questions that you should be asking are “If you could remove one feature, what would it be?” or “How would you simplify the product?”. Sometimes the best thing you can do for you product is to develop less.

Now go ahead and practice your newly acquired skill of just saying no.

© Copyright 2011 Yaacov Apelbaum All Rights Reserved.

Crafting Great Software Features Part-2

Yaacov Apelbaum-Sleep Master

The Sleep Master 7000SX: It captures and Tweets all your sleep stats while you snooze!

In his book, “The Diamond Age,” Neal Stephenson classifies technologists as belonging to one of two categories: (1) those who hone existing ones and (2) those who forge and create new ones.

There is a fundamental difference between how ad hoc assemblers and software crafters approach building a product. Ad hoc assemblers tend to start with the technology and the solutions it offers.  They speak in terms of using a framework, language, or protocol to solve a problem.  They frequently make statements like “the next version of X will solve the problem of Y”. Ad hoc assemblers who frequently suffer from myopic vision will solve customer needs first by boxing some existing technologies together and then by shoehorning a GUI on top of it.  The sum of the feature and functionality will be driven entirely by the framework of the underlying technology.

This almost always results in marginal user experience and product performance. The solution isn’t designed for ease of use. It’s not even intended to solve any concrete problem. Its primary purpose is to act as a vehicle for marketing hype and sales. You can recognize these products by the emphasis they places on mile long lists of features, most of which are poorly implemented and are of little commercial value.

It’s a simple value proposition: it is more important for some neat technologies to be shipped (the latest buzz is cloud computing and social networks)  than for products to be useful. Very few great products are designed this way.

Software crafters, on the other hand, understands that real people will consume their product, and every decision about its design is made not only with a specific user in mind but also the specific problems the user needs to solve in mind. This and only this drives the choice of platform, language, communication protocol or database.

Your customers are no different than the people who are looking to buy a specific tool for a job. To deliver the right product functionality without getting lost in the technology jungle, you need to develop an understanding of how successful products are developed in other fields.

Manufacturers of tools and appliances all go through the same steps in balancing technology, requirements and usability. You can learn a lot from the successes of products like the IPod by recognizing that when we buy a product, we almost never care about unnecessary “fluff” features (like a social network enabled timer that can capture 4 different types of sleep statistics).  Rather, what we want is what provides valuable features (services) and perform them well.

 

© Copyright 2010 Yaacov Apelbaum All Rights Reserved.

Designed for Humans

Yaacov Apelbaum-Designed for Humans

In my previous life, I was a civil engineer. I worked for a large power marine construction company doing structural design and field engineering. The work assignments were pretty interesting. I got to blow up a bridge, salvage a sunken vessels, and build a lot of interesting marine structures.  On one of my projects, I was given the responsibility to design a set of beds for pre-stressed concrete piles.  The design challenges in this project were significant. We had limited real-estate and the loads involved were higher than any previously attempted.

Yaacov Apelbaum-Prestressed Concrete Piles Beds for pre-stressed concrete have massive anchors on each end. Between them steel forms are placed and steel cables are strung.  The cables are first tensioned, then the concrete is poured into the form.  When the concrete hardens, you cap the cables and cut them.  The result is a pile or a girder that is significantly resistant to various loads.

Following best engineering practices, I completed a structural load analysis document, a set of production blueprints with full dimensional drawings, welding, coating and assembly instructions, a bill of materials, and even a balsa scale model to help the manufacturing facility to visualize my design. Yaacov Apelbaum-Prestress Bed Scale Model

I was proud of my hard work and I felt that it was a great achievement.  The day before the presentation, I went over all the calculations again and rehearsed my slides.  After one last sleepless night, I arrived to the conference room to find several structural engineers, the yard superintendent, a number of field engineers from several divisions, and the chief engineer from corporate, an elderly white haired gentleman in his mid-sixties.  I remember feeling confident in my ability to sell my design to them.

The entire presentation went off without a glitch.  There were some stylistic comments but the overall feedback was good.  After the presentation, the chief engineer stopped by, shook my hand, and said that he liked my design very much.  Then with a straight face, he told me that he expected to see two additional alternative designs before we finalize our decision.

I was speechless.  “I’m not sure I understand, sir” I said. “Didn’t you just say that you liked the design?” I pointed out that none of the participants had found any flaws in my proposal.  “Why", I asked, “did you think we need to develop two additional designs?”

He paused for a moment, and then said, "You never know what the best idea is unless you compare several good ones side by side." I nodded politely, but I was disappointed. I felt like this was probably some form of engineering hazing. Was it truly the case that it’s impossible to achieve reasonable quality on a first try? I didn’t really understand how valuable his advice was until years later.

Yaacov Apelbaum-Pre-Stressed Concrete Bed

Completed pre-stressed concrete beds

Fast forward several years. I switched from civil engineering to software development.  At the time I was working as a lead front-end designer.  One of our key customers hired us to migrate a large VC++ client to a browser application.  In the mid-nineties, rich browser based clients were relatively unheard of.  We were stumped. Problems like session security, persistence, and lack of basic GUI controls seemed Insurmountable.

During meetings, I would regularly sketch various GUI solutions.  But I often found that as soon as I came up with a solution, a new set of problems would be exposed and a redesign would be necessary. In retrospect, most of the ideas I came up with at the time were sub-par. But with each design, no matter how bad, another potential solution was discovered.  Each new design I sketched out was closer to the solution than its predecessor. Even the poor designs peeled away some layers that obstructed the problem that I didn’t initially see.

After dozens of attempts, I had an epiphany and came up with one design that it was possible to implement in several ways. Sketching and contemplating the various designs helped me tremendously, but when the time came to present my solution, I made a tactical mistake. I deliberately neglected to show all of the other working ideas for fear that they would think that I was a mediocre designer; why else did I need to work so hard on so many designs just to yield one single decent one.

I realized in retrospect that there would have been any number of acceptable designs and by not presenting some other ideas I considered before arriving at the one I chose, I short changed myself. If anybody had suggested one of the other options I had discarded but not mentioned, I would have had to explain that I had already discarded that idea. But at that point , it would jeopardize my credibility because it would look as if I was only trying to brush them off.

 

 Yaacov Apelbaum-Poor Design   Yaacov Apelbaum-Quality Design M1917

Multiple product designs


After participating in and leading many painful design meetings, I have come to the realization that the best way to sell the top design idea is to first share some of the alternative and inferior designs.

If you are responsible for usability or user interface design, you have to develop at least several alternative options for credibility purposes. By that I don’t mean that you should become a cynic and create duds just for the sake of generating volume. The alternate ideas have to represent meaningful and functional choices.

Once you have your alternates worked out, walk through the various options during your design meeting and call out what the pros and cons are for each and what the overall solution trade-offs would be. When discussing designs, place emphasis on both the positive and negative qualities of each alternative.  This will help your peers view you as an unbiased and professional presenter.  Also, you may find that when you present your top candidates, your team will come up with some hybrid solutions that otherwise would have been impossible to generate if you had only presented a single one.

Nowadays, I am often tasked with working on problems that are exceptionally difficult to overcome (with respect to both technology and schedule) and the typical, off the shelf solution is just not sufficient. But there is hope. Usually after a few days of intense interterm deliberations complete with often heated exchanges of alternate designs, magic happens.

My secret sauce for breaking down the most difficult design problems consists of the following steps:

1. Get your entire team into a conference room, order plenty of pizza and write down all possible solutions on the whiteboard. Make sure that everyone offers an opinion. Don’t make any go-no-go decisions during your first meeting; rather leave the information on the board for several days (don’t forget to mark it as ‘do not delete’)  and schedule a follow-up meeting. Tell everyone to document the pros an cons list for each option and provide specific use cases.

2. Get your team into a conference room a second time, order plenty of pizza and write down the pros and cons list for each choice.  Boil down your choices to the top three candidates.

3. Work out the feasibility of each of the top three candidates and cast a vote for the best one.  This is the time to establish consensus and a team buy-in.

    Way back when the chief engineer asked me to come up with two additional alternate designs, he was in fact telling me that no matter how talented a person is, there is tremendous value in variety.  He was also saying, that in order to come up with a ‘good’ design there must first be several inferior ones. If you are responsible for the design of any product futures, you will want to encourage your team to flesh out the bad designs on the whiteboard or as POC, not in your final product.  Unfortunately, the only way to achieve this is by expending resources and time exploring several possible solutions, up to and including some unattractive ones.

    A common development folly (see It’s Good Enough for me) is the notion that there is in fact a ‘best’ solution or one right answer to a given problem.  Actually, the opposite is true. Considering time and resources, in most cases, the ‘best’ possible solution isn’t worth the effort and a ‘good’ solution would more than suffice.

    If you are curious abut which design I ended up using for the prestress pile beds, it was the third one.  It turns out that unexpectedly, after I reconsidered the problem again, I realized that due to the yard’s location at sea level, the water table was too high to accommodate my initial proposal. As a result, my updated design required various modifications in order to solve this problem.

Live, design and prosper.

 

© Copyright 2010 Yaacov Apelbaum All Rights Reserved.

Developers Just Wanna Have Fun

 Yaacov Apelbaum-The Cinawaffle

The Cinawaffle DX250 Waffle Maker\DVD\MP3 Player is Remote Controlled and Bluetooth Enabled.  It Comes Standard with an SDK and a Built-in Web Server.

One of the greatest fallacies in software development circles is that great products must be made with cutting edge technologies.

This belief is not coincidental, as most of the people who work in high tech maintain a passionate love affair with technology. If you ask most of us about our willingness to work long hours on risky and challenging projects, the answer is likely to sound something like: “I love technology, it’s fun,” or “I enjoy playing with technology; I can’t believe that someone is actually willing to pay me to do this job.”

Hardcore techies aren’t the only ones who are enamored, It seems that since the ushering in of the industrial revolution, we all have come to embrace the utopian belief that technology will one day solve all of humanity’s problems.

Believing this, many seasoned business and dev managers fall into a trap and tend to package unnecessary technology into their applications. Unfortunately, this is most often done to the detriment of the product and its users who fail to leverage or appreciate the complexity it brings.

I have witnessed numerous product launches where the entire company stood in awe at the sight of their own technical achievements.  Managers and developers alike were congratulating each other on overcoming what early on seemed to be impossible development roadblocks. All without any regards for the sketchy implementation and clear sight of the purpose that such functionality might serve.

Yaacov Apelbaum-Neo LudditeYaacov Apelbaum-Luddite Reward At the risk of sounding like a Neo luddite, I would argue that our infatuation with technology doesn’t always lead us to develop the products our customers need.

When you were interviewing for your last gig, you likely stated that the ability to work with new technology and learn new tricks was a significant factor in your decision to apply for the position. You may have also been told by your interviewer that his company prides itself on maintaining a fun work environment and that you would get the opportunity to work on some very cool and cutting edge projects.

I believe that personal and company success hinges on establishing a creative, unpretentious, and challenging work environment.  But that’s not the whole recipe for success.  Any organization that hires expensive technical staff cannot afford to do it for fun’s sake only.  A company can only sustain a playful atmosphere if an increasing number of paying customers embrace and consider its product useful.

The reason for this is simple. Contrary to the common misconception, it’s not angel funding, seed money, or bridge loans that pay for your operation. Rather, your end users end up picking up the tab for all of your company’s salaries as well as for the espresso machine, the new Wii\Xbox station, and the billiards table.   In light of this, every activity you engage in (e.g. design, coding, or testing) must directly benefit your costumers and end user.

How granular are the effects of product usability on your daily work?  Each line of code you write, every bug you find, every new feature you consider must help the user improve his productivity in some quantifiable way. No matter how obscure or indirect your current project is, streamlining the user interface, improving application stability, or optimizing performance, all must directly benefit your customer. If you can’t clearly see how your work improves the user experience, consider investing your time on some other activity. The more frequently you think about your work in terms of end-user productivity, the more impact you will have on creating a profitable product.

Beauty of Simplicity

The greatest engineering feats are the ones we don’t notice. The hallmark of a great designer is his ability to translate complexity into simplicity. The automatic transmission in a car represents significantly more engineering effort than a manual transmission, but it dramatically transforms the average user experience. The best engineering, architectural, and consumer products always focus on improving user experience and hiding complexity, not showcasing it.

The most effective approach to adding value to products is to add power and ease of use while reducing the learning curve. When you are developing a new feature, ask yourself, is there some way to add it without changing the user interface? (users hate learning new interfaces)  Can we solve a problem by re-designing workflow or consolidating instead of adding more screens and menus?  Or is there some other feature we can modify to include new and improved functionality? Think of car manufactures and how they add major new features with minimal user impact. The dashboard almost never changes, rear anti-collision video monitoring is integrated into the GPS display, anti-lock brakes are added to the standard brake pedal UI, and power steering is added to the steering wheel UI. Minimal training is required on the part of the user to reap the benefits of these new features. This kind of design approach—where complex functionality appears simple to the user—helps create great products.

There are countless opportunities throughout your application to provide a great user experience. Watch your customers usage of your top features and ask yourself how they compares to the level of service you’d get in a 5 star establishment.  Is the product fast, intuitive and easy to use?  Is efficient help always available promptly? Is it easy to get information out of it? And most importantly, does your product enhance the user’s productivity and business process flow?

Regardless of the technology you adopt, the closer your application’s usability matches the levels of service people get in their daily physical experiences, the closer you’ll be to having a great and useful product.

 

© Copyright 2010 Yaacov Apelbaum All Rights Reserved.

To Make Errors is Human, to Handle Them is Divine

Yaacov Apelbaum-We Have Bugs

What a deal! They’ll fix their own bugs “free of charge!

Reading this advertisement made me realize how clever the software industry has become.  Why bother fixing your product prior to shipment when you can sell it on the premise that you will fix the bugs “free of charge” when the users find them for you.  Interestingly, anyone who bothered to read their licensing guide will find the following sobering caveat:

“…From an engineering point of view, it is impossible to fix bugs in multiple source code branches. If we would have to do this, we would never be able to implement a major redesign. Major redesigns are required now and then to be able to fix bugs and add features fast.” 

Nothing communicates your attitude towards your users better than the way you handle exceptions and error messages. As soon as something goes wrong with your application the user is at a heightened emotional state and is the most impressionable. Some software products, including the leading market applications, have developed a bad reputations for having cryptic error messages that are impossible to resolve, leaving the user feeling helpless and outraged.

The worst offenders include fortune teller style messages that inform you (not without irony) that you are about to lose all of your work because the application has encountered an unknown problem and needs to shot down.

Yaacov Apelbaum-Useless Error Message
A Useless Error Message

This is even more pronounced in the session-less environment of the internet. It seems that when it comes to web application reliability and robustness, we’ve been steadily taking a step backward in the way we treat our users.

Yaacov Apelbaum-Lost my Browser Yaacov Apelbaum-Stopped Working

Yaacov Apelbaum-Blogger Error Yaacov Apelbaum-Microsft Live Writer Error
Useless Error Messages

Engineering and Failure Handling
A civil engineer designing a bridge will invest a significant amount of time and resources in predicting potential structural failure scenarios. Failure analysis and safety factoring (i.e. redundancy) are two important cornerstones of the engineering discipline. In the physical world of machines and structures, the ability to indentify a potential design flaw and remedy it is a given. Similarly, we should strive to achieve the same in the virtual software world by accounting for critical error conditions and developing robust application code capable of handling those cases.

Software engineering does have certain nuances that differ from classical engineering, which makes prioritization of work more arbitrary and less straightforward.  For example, a small memory leak in a server component may be considered by the development team to be a critical bug, but a relatively small data validation problem that forces the user to retype a lengthy application could have a bigger user impact and rank higher on the bug fix priority.

A 12 Step Program for Error Rehabilitation
Making your application more agile in handling failure and enabling it degrade gracefully are not a single step processes and there is no silver bullet technology out there that will fix this problem.  If you want to break the cycle of application instability and user frustration, you will have to dedicate time and your best technical talent to solving it.   I have found that a phased approach works best.  In this approach you first handle the low hanging fruits, (addressing the mechanics of the error handling), and than gradually move to higher ground (addressing automated problem resolution and preemptive countermeasures).

The following is my 4-phased program for working out a resolution to application errors. Classification is inclusive, so the 4th phase (the highest level of reliability) also includes the properties of the preceding levels:.

 

Phase-1: Create Unique and Traceable Errors and a way to Record them

If you are under the gun and don’t have time for any other remedy, at least make sure that your error cases are unique. Telling your users that an error has occurred in the application without providing details is a sign of an immature product. When your technical support team receives an error report, they should be able to determine precisely what is causing the problem.

Generic error handling (same message for all errors), or different error causes that return identical messages, are easy to implement, but when it comes to debugging they are useless. Unique error IDs allow us to more efficiently track bugs and translate them to a more stable product.

Error codes should be visible in the error messages but not be the focal point of the  the message.  You should develop a library of descriptive text that provides a human readable explanation of what the error means.  Provide a simple mechanism to either log the message directly into your app or send it to you via email.  Nothing is more annoying to the user than being asked to type in the error message manually.

Establish an Issue Tracking System that allows quick data entry and reporting.  At the minimum record the error code, error description, and the steps to reproduce it, effected environments, and its frequency.

Phase-2: Keep the User Calm and his Data Safe

Error messages should always carry a mature and responsible tone. Always use supportive, polite language, like a good teacher would when instructing a pupil.

If the user opts to leave a mandatory field empty, or mistypes the data type (CC#, zip , etc.), don’t go ballistic. Non-critical errors deserve non critical messages. Instead, indicate on the entry form where the problem was, place the cursor in the relevant field and leave the rest of the data intact. This is especially important for long entry forms that require a lot of effort to complete.

Don’t force the user to duplicate entry of some previously supplied data for verification purposes (such as billing and shipping information) as it may introduce human error and trigger him to abandon the application altogether.

Phase-3: Good Errors Messages are Clear and Provide Remedies

The way the user perceives the error is much different from the way you do. He thinks in business terms and knows nothing about the inner workings of your application, nor does he care. That’s why you should always design the error UI from the user’s perspective.

Here are the seven golden attributes of error messages:

  1. Describe the error in user terms and language
  2. Instruct the user as to how to complete the task and resolve the error
  3. Explain how to prevent the problem in the future
  4. Avoid technical mumbo jumbo and acronyms
  5. Avoid modal pop up error messages and instead write error directly to the page
  6. Provide help links that better explain the nature of the error
  7. Keep the text formatting simple and avoid bright colors and animations

    When providing a solution, give clear step by step instructions as to how to fix the problem. Be specific and do not assume any pervious user knowledge. If there is a relevant tutorial or the specific solution in your on-line help, provide links directly there.
    If it’s a critical problem—for example, the Website is not accessible—provide a mechanism for the user to report the problem to you and immediately acknowledge the receipt of his complaint, provide an explanation and an estimate of time before this problem will be resolved.
Phase-4: Handle Errors Internally

Write code to robustly handle all errors. This will eliminate the most severe and common errors (like missing data or validation). You can achieve this by automating data entry components from the user interaction (i.e. deriving city name from zip code).

To the extent possible, take corrective action before an error occurs.  For example, if the user is in the middle of a lengthy entry form, save the contents as he moves between fields, this will allow you to restore the information if he inadvertently navigates off the page or even closes his browser session.

It’s often expensive to identify and address all possible failure cases, but if you have been tracking your top bugs, you can start with the biggest offenders first.

The way you handle and communicate application errors directly reflects on your team’s and your company’s reputation. When building a new or reworking existing functionality don’t assume that the old error messages apply to your new logic and boundaries.  Building test cases around various error scenarios (missing data, wrong data, bad data, etc.) and dedicating a test cycle to generate all known error messages is also an excellent strategy.

Error handling and messages should be thought of as required phase of any feature development, and adequate engineering time for it should be budgeted into all SDLC estimates.

Real quality of service goes beyond just acknowledging your application’s faults. My rule of thumb is that there is no such thing as an “informative error message”. A good error is one that has been eliminated through error-handling code and through superior product design.

 

© Copyright 2010 Yaacov Apelbaum All Rights Reserved.

It’s Good Enough for Me

Yaacov Apelbaum-Giacamo the Good

Fighting the Best Defending the Good

I commute frequently, so I tend to have some down time at the airport while sitting at the gate and waiting for my ship to come in. I usually use this window to catch up on my technical reading, but recently I decided to take a break and venture in to one of the book stores in the concourse. After skimming the offerings, I discovered a bookshelf filled with titles of the “How I Became the Best In ___, and How You Too Can By Simply Following My Easy Three-Step Program” genre. These books, mind you, are not cheep paper backs. I was looking at thick hardbacks, generously illustrated and accordingly priced. Apparently, the “How to Become the Best” series industry is booming.

This got me thinking: statistically speaking, the best of any kind takes up only a tiny outlier of the bell curve. So why the hype? Clearly, if this industry is thriving there are enough literate people out there who were willing to buy into the idea that being the “best” is worth their time and money.

Then a few weeks ago, I found myself confronted with this concept again. I was having lunch with a colleague and he raised the argument that the only way to win in today’s lean software economy is to develop the “best” features and functionality. He expressed his strong conviction by recounting his recent experience at a trendy “how to become the best” seminar. “I am a new man,” he said, “This event has changed my entire outlook on product development”. “How’s that?” I asked, curious. He leaned forward, squinted, and in a lower and somewhat more mysterious voice, he summarized his newly acquired philosophy. He said that according to the presenters, Trump, Robbins, and Kiyosaki, success hinges on one’s ability to tap into one’s inner best. Either you’re Napoleon or you’re out of the game.

At this point, I was done with my burrito and so I seized the opportunity to respond in kind with a rival French maxim. I quoted Voltaire: “Le mieux est l’ennemi du bien” (The Best is the Enemy of the Good). Wellington, I pointed out, was by no means the best, but he certainly outlasted Napoleon in the game.

My companion was startled and said he didn’t understand what I meant. I offered an explanation: “It’s not that I am a proponent of mediocrity; to the contrary,” I said, “I pride myself on my attention to quality. I have absolutely no problem with the concept of pursuing excellence. What prevents me from realizing perfection are mundane details such as looming deadlines, shrinking budgets, and a chronic shortage of resources.”

Of course it’s easy to invoke demagoguery and claim that it’s either “best” or “bust”. Many development managers adapt this mistaken philosophy, assuming that it has a positive motivational value. The average corporate culture doesn’t help dispel this myth either, by creating unattainable criteria for personal performance and compensation plans. Regardless of how fond of the cliché’ you may be, unfortunately preaching the best when it comes to delivering software under time, quality, and budgetary constraints is one thing, actually being able to deliver on such promises is quite another. If we learn anything from human endeavors, it is that “good enough” is more than acceptable. As far as I know, most of us don’t drive the best car on the market, live in the best built house, or exclusively buy the best clothes or appliances. Compromise is the order of the day.

My favorite story that illustrates this concept is the World War II race to develop the radar. Both British and German teams were aware of the tremendous operational and strategic advantage this new technology could offer. The German development team had the more advanced science and superior technology. Their radar was more accurate, had a longer range, and provided fewer false-positives. The German team—true to their cultural heritage—was striving to develop the best apparatus possible. The British team was smaller, less experienced, and had inferior technology. But from the outset, it adopted the motto: “Second Best Tomorrow”. This philosophy eventually allowed them to release an inferior but working radar earlier than the Germans thus winning the race and ultimately tipping the balance of power.

Cheap (often free) and simple software free of stringent SLAs is popping up everywhere. Most of us now get our breaking news from Google and personal blogs, case in point. We make free, long-distance calls on Skype (and don’t mind the low QoS), watch video on tiny iPods screens rather than high definition TVs, and more and more of us are using low-power cell phones that are just good enough to meet our surfing and emailing needs. For many leading companies, the distinction between good enough “beta” versions and commercially “best” products has blurred beyond recognition. (Gmail has finally come out of beta after more than 5 years.)

To be successful in commercial software development, one must fight the urge to gold plate by adding late stage functionality. One must also learn how to be firm regarding ad nauseum pressure for application re-writes, all in the name of making it the best.

Contrary to what the motivational posters profess, when it comes to shipping on-time, the pursuit of perfection can become your worst enemy. The same also applies to excessive QA and testing. In the end, even the most comprehensive white, gray or black box tests can only provide a projection of how your application will perform. The ultimate usefulness gauge are the real users. The earlier you release your product into the wild, the faster you’ll discover if it adequately fills a need.

As I have discovered on many occasions, building a good enough product and releasing it early enough is good enough for most customers—which is good enough for me.

 

© Copyright 2009 Yaacov Apelbaum All Rights Reserved.