Adam I. Gerard

Software Engineering and Mistakes

I haven't spoken much about programming, my experience engineering, etc. Today I'd like to!

Code Bugs


Having 10-50 bugs per 1,000 lines of code is considered “good”. Hardware rates are much lower (since hardware defects cause electrical shorts, fire, combustion, are carried on a person physically, etc.). Software is typically e-commerce heavy (involving financial transactions) and not typically life-threatening. Still, I was surprised to find the listed rates as industry standards.


From Has a software bug killed anyone?.

Engineers should carefully define each possible worst case scenario that might arise from people using one's code:

  1. What is the worst thing that can happen if this code fails?
  2. Is that something reversible?
  3. How can the likelihood of those worst case scenarios be mitigated for?

I don’t think the above process happens enough. As a result, it's often unclear what the effects are if a subsystem or system fails.

Lately, there's been a lot of high-profile news surrounding aircraft systems that have gone haywire and resulted in people being injured or killed.

Strategies like the preceding might help to save lives and improve overall engineering quality at many firms.

For example:

  1. Critical systems (where failure would result in loss of life) would be designated Failure Threat Level One.
  2. Dangerous or criminal systems (where failure would result in injury or a major crime) would be designated Failure Threat Level Two.
  3. Financial systems (where failure would result in irreversible financial loss) would be designated Failure Threat Level Three.
  4. Temporal systems (where failure would result in irreversible loss of time) would be designated Failure Threat Level Four.
  5. Annoyance systems (where failure results in a purely reversible, easily fixable, cosmetic but functional defect) would be designated Failure Threat Level Five.

No code I've written, nor any coworker of mine has ever written, has resulted in loss of life, injury, or irreversible financial loss as a result of a defect or error.

I personally stay away from aircraft, vehicle, weapon, and/or life support systems (or similar such systems) so that my Failure Threat Level never exceeds Three.

As a result the worst that could happen and could have happened from my code is Failure Threat Level Four (per the classifications above).

Personal Mistakes

I’ve had four production bugs (one CSS, one HTML, one SMS, and one memory leak on a public beta) that were deployed live on customer-facing resources. (Although I've also introduced a few bugs that were caught in QA testing.)

Three were fully functional but had some UI anomaly. All were corrected with no permanent loss of money, time, or anything else to our users. No immediate costs or harm occurred (being relatively minor bugs).

Four live-production software engineering bugs in 5+ years of engineering. And, all four were code-reviewed and tested by others. Still, I'm disappointed those happened!

According to the Failure Threat Level above, these issues would have Failure Threat Level Five (reversible but functional cosmetic defect) and Failure Threat Level Four (loss of time to fix a memory leak).

By contrast, the average number of bugs introduced is right around 10–50 / 1,000 Lines of Code. Anecdotally, I think that bug defect rate is true - even the best engineers would have maybe 1-2 bugs / 1,000 Lines of Code introduced into production after Quality Assurance testing and review.

Personal Accomplishments

  1. Two companies I worked for exited (were acquired) because of projects I worked on.
  2. Three companies I worked for raised money because of projects I worked on.
  3. Two companies doubled in terms of employees because of projects I worked on.
  4. Lives have likely been saved from health-related projects I consulted on or helped to build (improved clinic experiences, telemedicine, air quality and asbestos detection).
  5. Projects I worked on were within the sphere of national security and national scientific agendas (as designated by the Department of Defense, Department of Homeland Security, and Congress).

Profit Per Employee

You have to bring it and typically 2-4x your salary and benefits.

Most FAANG companies generate at least $125,000 in profit per employee (after salaries, benefits, etc.). Walmart generates about $6,000. (Clearly, very different business models.)


Despite certain cultural stereotypes, it's difficult to say that software developers don't work their asses off!

Cost Saving

I once made a code change in about 15-45 minutes that saved our 300,000+ users about 5 minutes each:

  1. Each user would log in at least twice a month (300,000 × 5 × 2).
  2. The service ran for at least two years (300,000 × 5 × 2 × 12 × 2).
  3. That's greater than 72,000,000 total minutes saved.
  4. There are 525,600 minutes in a year.
  5. That's greater than 136.98 years of time saved (72,000,000 ÷ 525,600).
  6. The GDP per capita is $62,794.
  7. So, my change (probably) created greater than $8,601,917.80 in time cost savings passed on to our users (136.98 × 62,794).
  8. In terms of raw business value-add, it’s more difficult to say though we eventually saw almost 100,000 more users (the app was not-subscription-based). We can infer though that the nearly 25% increase in users was partly the result of performance enhancements.