Real World Software Engineering – XXIV

On "why?"

People like to say "It's the 'what' and not the 'how'" as if they found the key to solving major problems in software analysis. I think "why?" is a more important question.

Real World Software Engineering – XXIII

On the intuition of risk of throw-away code

At times, while I'm creating software, I have a feeling (and it is just a feeling) that what I'm creating is going to the garbage. Being subjective, there is no way at the time to be sure if the preoccupation is warranted or not. I think that this feeling arises out of different factors:

- knowing that you are coding something for a requirement that brings little value to the end user. Apparently the business thought about the requirement, but you are not convinced.
- knowing that you are coding something for a requirement that is not firm in consensus.
- knowing that you are coding something for a requirement that is not well understood by the client. (The client may 'know' the requirement but not 'master' it. When that happens there's risk in the requirement.)
- knowing that you are introducing dependencies into other software that were not debated in the group/project.
- knowing that you are coding something for a requirement which is not explicit, that you think you are going to need but you are not so sure. That is, if you do not code it, there's the risk that there will be a 'bug' created saying that your feature is not complete.

Real World Software Engineering – XXII

On Knowledge and Planning

It is interesting to note the impression that it is sufficient for one to know what has to be done for it to be done. We just need to list what we need to do and do it. The plan becomes the charter that guarantees success. It gives the illusion that's all there is to do. Often the plan helps give direction, but the tasks not included in the plan are many. And so are the unforeseen events. A list of tasks helps one get organized and identifies what we think we should do in the near future. It does not guarantee success, however.

Real World Software Engineering – XXI

On the Fear of New Technologies

It appears that the fear of new technologies is a major criterion for not adopting them. This fear is often a product of ignorance of the new technology. Another factor for not adopting new technologies relate to 'peers'. That is, companies want to be as fashion as their peers, but do not want to dare too much. It is usually a miscalculation of risk, as they seldom take the time to assess the new technology.
I conclude that the introduction of new technologies is often related to a culture of innovation within the company. And that is a rare culture, despite the rosy discourses.

Real World Software Engineering – XX

On scope options

I'm impressed at how quickly some people make decisions on scope and design without taking the resulting customer satisfaction into consideration.
Why is this process so informal again?

Real World Software Engineering – XIX

On the Holy Grail

I frequently hear of bad expectations for software. In moments like these, expectations reveal how badly the speaker perceives software.
For instance, take the requirement that can do everything in one central location. Whenever I hear this I interpret the following:
- the person declared to be ignorant on the problem domain.
- the person is saying that the problem domain is complex, but he/she is lazy (or incapable of) to analyze it in detail.
- the person dreams of a reality where all his/her problems are gone with one magical software capable of marvelous deeds in the metaphysical dimension. Of course, the person is not aware of this dimension.
- the person wants a software with good quality and "usability" - mainly the UI (even though the person cannot articulate this thought).

Take an example from the "real world" for example: the speaker's dwelling. I bet the person has different rooms. I bet that in his/her house there are different objects that serve different purposes. I suppose different objects are related (sofa with TV, power outlet with electronics, pipes with faucents, hangers with clothes, table with food, etc). I bet he speaker is comfortable with using these different things for different purposes. Why does he/she have different expectations for software?
How can we - developers, architects and business analysts - deconstruct this Holy Grail of software?

(keyword: silver bullet)

Real World Software Engineering – XVIII

On Customers

Whatever you do, always listen carefully to the customers. Listen to all the customers. All their voices from all departments. Try to understand them. Empathize and understand their problems. Now think about your product. Think about how it addresses the problems.

Real World Software Engineering – XVII

One what versus how:

The mantra of trying to concentrate only in the what and not in the how only goes so far. At times it is possible and desireable to only think about the 'what'. The fact of the matter is that often the division between 'what' and 'how' becomes more visibly artificial. More often than not, the 'what' is some abstraction of the 'how' described in a 'more abstract' language. Often the semantics are not sufficient to convey relevant meaning for defining the 'what' and elaborating on the 'how' is mandatory. The wise Business Analyst will realise that a division between 'what' and 'how' is only artificial. It will realise that when the metaphor works it helps the requirement gathering progress. At times, however, it does not work. Then one is left with the painful but necessary task of refining the language. Oh, how I wish real people who work with real software knew Linguistics and Philosophy better.

Software Reuse is Bad For You - II

On Software Reuse

(Note: I don't actually mean that software reuse is bad for you)

"You" this time is the developer. Developers in this industry move frequently between companies. The idea is that software that is reusable, easy to extend, easy to integrate is more difficult to create (and easier to maintain). If a developer stays a short period in the company, why should he/she bother to spend the extra time to create software with these long-term qualities? More often than not these good qualities are 'invisible' and not immediately recognized by local management or the customer. My advise to local management? Check out which software or modules have less maintenance, look at the software and think about the reasons why. My advise to customers? Learn more about software quality. Ask tough questions not only about what will work for you tomorrow but years from today.

Real World Software Engineering – XVI

On Software Complexity

You know you work developing a complicated software when you don't quite know it inside out, but you somehow manage to work with it. This means the software could be simpler. And I'll bet there is problem with the language around the software. There is, the language used by documentation and the community around the software is not very clear.

Real World Software Engineering – XV

On Documentation

Documentation is one place where you can see the discrepancies between marketing and the real product (or maybe not).

Real World Software Engineering – XIV

I wonder how music would be if it were like software development. I think it would be something like this.

The Ottawa Brass Ensemble was a very good ensemble. One day, at the end of a rehearsal, Mr. Borremans, the Artistic Director, arrived with the good news:

- We'll perform the Neck Cracker for the official visit of Mr. Schneider to Ottawa next week.

Mr. Schneider was the Foreign Minister of Lichtenstein, the small European autonomous enclave. His unpredictable temper was notorious.

John, who played the tuba, was the first one to speak.

- But the Neck Cracker is a symphony for a full orchestra.

- Yes, I know. Don't worry, we have already ordered the remaining instruments.

The musicians had mixed feelings. This was their chance to grow into a full orchestra. And the Neck Cracker wasn't just any ordinary symphony. It was a masterpiece by Maître Dubois, the famous Modern French composer. Just hours before dying, Maître Dubois wrote this masterpiece for his mistress and hid it in a safe so that his wife wouldn't claim it.

- The concert is set to June 1st. We have lots of work to do.

Next Monday, April 3rd, was the first rehearsal. The musicians arrived and were greeted by Borremans and another man.

- May I introduce you to Monsieur Rousseau. He is our guest conductor for the June 1st concert. He is a prestigious conductor and I'm sure you will all be able to do a great performance that day. As you all know, Mr. Schneider is a high official, and his visit is very important to our representatives. This concert is very important and we have to execute it perfectly. The media is already speaking about it. There is no margin for errors. I'm confident you are competent musicians that will perform very well. With that said, let us welcome Monsieur Rousseau.

After much applause, Monsieur Rousseau began to work right away.

- Merci, merci, my friends. For this concert, we'll be employing the Presto Methodology.

The Presto Methodology was the latest buzzword in Orchestral Engineering. In it musicians were supposed to memorize the whole score and practice in invisible instruments. Each section would practice separately, instead of the classic whole-orchestra rehearsal all the time.

- Mr. Borremans, when will the other musicians arrive to complete the orchestra?

- We are already interviewing several candidates, who should arrive next week, if everything goes well with HR.

- And how will we practice without the other instruments in the meantime?

Mr. Borremans hadn't actually thought about that, but he needed to answer the question.

- I've consulted with Monsieur Rousseau, and we've decided that we'll be transitioning some of you to the other instruments. It is only a temporary measure until the other musicians arrive.

The musicians thought that was a bit odd, but were still excited to try out the Presto Methodology and the different instruments.

In the next day, the musicians arrived at the theatre and saw the list of sections with their names in them.

Looking at the list, John commented with Joseph:

- Did you realize that we'll need to learn a brand new instrument?

- Don't worry, John, it's only until the string players are hired. And it's not so difficult. After all, we are all musicians.

But M Rousseau announced shortly after:

- HR is still interviewing the new musicians. We realized that we may not have enough time, so we'll need you to be prepared to play the string and percussive instruments for the concert just in case.

The musicians were surprised. M Rousseau continued:

- But don't worry. We have already scheduled for a famous violin player to come and give us a Master Class on how to play the violin. I've also rented the movie 'The Fiddler in the Roof'. My daughter saw it yesterday and thought it could help us. I thought it was a wonderful idea. We want to equip you to be successful by all means, so feel free to order any books and expense them.

Now the musicians were worried. Most of them had only a few weeks to learn a brand new instrument, practice and perform the difficult Neck Cracker sounding like a professional orchestra.

The new rented instruments arrived. The musicians took them and sat down in their named chairs.

- Hey, Brown, where is the score?

Brown was the first-chair clarinet, and had been promoted to concertmaster.

- We have a small problem with the score. Don't worry. As you know, Maître Dubois locked it in a safe before dying. The grandson of Maître Dubois, Bob Dubois, had promised me that he would get it to me last Sunday. But he called me yesterday saying the lawyer of Maître Dubois' wife spoke to him. The wife's lawyer had a subpoena to take the score. But don't worry, our lawyers guaranteed me that we'll have it next week. Luckily, Bob Dubois heard his grandpa whistle it many times at his cottage. Just in case we have no access to the score, Dubois will be helping our contracted Score Consultants to rewrite the score. The Score Consultants will fly in from New York tomorrow. They will be brainstorming with Bob. I think we are in the right path. In the meantime, take the time to assemble your instruments, install the strings and familiarize yourselves with them.

The musicians had no alternative. They went on to get their instruments in order.

• M Rousseau, I think we have a small problem with the instruments here. We can't find the cellos in the order. They sent violas instead.
  
• Ask the section leader what to do. I'm comfortable with whatever decision she makes.
  

  Sandra said:
  

-We'll just put the cello strings in the violas. It should be alright.

- Hey, Brown, did you notice that the violins only have three strings? There was a flaw in the design of the violins by VonderBuick. I told the Procurement Department not to order the violins from them, but I guess they lost my memo in the paperwork.

- Yes, I noticed it too. Don't worry, Peter. The newer version of the violins by VonderBuick already fixed the problem. They promised me they will mail the newer versions next week with no extra charge, and we get to practice on the current violins for the time being. It's not much different playing in three or four strings anyways.

The next day was the brainstorming session with Bob Dubois and the Score Consultants from New York. Of course, they were all in pink suits and relayed lots of confidence in what they did.

• Mr. Dubois, the first thing that we should do is to write down the Vision Statement for the concert. Let's begin brainstorming, and my colleague will translate the keywords from the sticky notes into the Vision Statement. This is how it works: we say a word and you say the next one. Like ping-pong. Ready?
  

- Yes

- Concert?

- Beautiful!

- Music?

- Dream!

- Did you capture that, Joe? I feel that "Dream" is a keyword. Very important.

- Do-re-mi?

- The Sound of Music

- Wow, that's good. I think we are getting somewhere.

- Mr. Schneider?

- Wild horses!

- Beautiful, beautiful.

After one week of work, Dubois and the Score Consultants had a draft version of the score. The layers were still arguing over Maître Dubois' real score. Meanwhile, the musicians were trying to learn everything they could at the violin master class.

• Look, it's easy, the bow goes up, and then it goes down. You see? Up, down, up, down.
  

The next day, the Score Consultants and M Rousseau gathered the musicians to introduce the score.

• As you know, the Neck Cracker is a masterpiece. It's a modern symphony in four movements: Allegro, Vivo Misterioso, Lento, and Presto Senza Perdere di Vista la Chiave ad Una Vita Bucolica e Tranquilla Ricordandosi che Cosa gli Uccelli Fanno Quando Copulate Ma Non Troppo.
  

This was the score:

Allegro Maître Dubois

Vivo Misterioso Maître Dubois

Lento Maître Dubois

Presto Senza Perdere di Vista la Chiave ad Una Vita Bucolica e Tranquilla Ricordandosi che Cosa gli Uccelli Fanno Quando Copulate Ma Non Troppo

Maître Dubois

John was the first musician to speak:

• The score surely looks great. But it doesn't have notes in it. There aren't enough details for us to play it.
  

The Score Consultants explained:

• The Neck Cracker is a beautiful modern piece that has the spirit of innovation. Bob Dubois will agree with us that it captures well this spirit.
  

Monsieur Rousseau jumped in to avoid any confusion:

• The score is a powerful tool to convey the principles of the Neck Cracker. It's a starting point and this draft will allow us to move forward. The musicians will look at the pictures and just play the emotions that come to mind. Let's think outside the box and be a bit creative here. The requirements are clear. We need to leave a good impression on the public and on Mr. Schneider June 1^st. It needs to be a moving, powerful and beautiful piece played skillfully. By the way, how are the Presto Methodology-oriented practices going?
  

The musicians went home and practiced as if they had air instruments. For days they were supposed to look at the pictures and let the emotions flow. The following rehearsal, May 2^nd, began not without debate.

• M Rousseau, the first violins are feeling that we aren't prepared to play the piece. We just don't have enough time to learn to play the violin. It is not an easy instrument.
  
• What's the problem that you are having more specifically? Is there any note you can't play?
  
• Well, it's not that there are specific notes we can't play. It's that we just don't feel we have the right techniques and enough training to do so.
  
• Have you been to any of the NAC concerts any Tuesday night?
  
• Yes, as a matter of fact.
  
• Then try to seat in a row nearer the orchestra. Do you understand? Nearer the orchestra. I have brought Bob Dubois here. He will be helping us to perform the Neck Cracker as it should. He will be advising us in terms of overall quality. He has heard his grandfather, Maître Dubois, whistle it many times and we have no one more equipped to help us in that aspect. Ok, folks, I don't need to take more of your time. Let's rehearse now with the instruments. First movement, da capo.
  

And the orchestra played it for the first time. As anyone would expect, it sounded weird. The section leaders were trying to make their best to convey the tempo to their sections gesticulating a lot. Nobody knew exactly what time to end. M Rousseau shouted

- Stop! Stop! It's sounding awful! This doesn't sound like the Neck Cracker. Section leads, what happened?

The cello lead explained:

- We are not familiar yet with the violas. We need more time to practice.

-We in the viola section are just trying to follow the score. But this first draft doesn't have enough details for us to play.

M Rousseau said angrily:

-Look, it's supposed to be grand and happy and yet powerful and moving. You played timidly sounding like a rutting gazelle. Can't you guys just read the score? It says Allegro! Allegro! Da capo!

And for weeks the musicians went on air playing their instruments in section rehearsals, struggling to perfect the Neck Cracker. The first version of the score, which was supposed to be temporary, became the de facto version. Of the string instruments that were to be hired by HR, only one violin player arrived weeks after because of budgetary constraints. Once in a while Bob Dubois would interrupt a section rehearsal saying

• Ok, stop, stop. That is not what my grandfather used to whistle, and I'm sure Mr. Schneider won't like it this way. Tuba, will you please play more gently? Oboe, the melody goes like this: la la la la laaaa la laaaa. Get it? Let's try again.
  

The section leaders got together in the pub around the corner, as they usually did Friday nights. The 2^nd violin lead went on to talk about his idea:

• What if we get together and write down the score? We could capture the basics, like tempo, metrics, key signature, improvisation sections and so on. We'll lock Bob in a room for four days and just write it down.
  

That sounded like a great idea. They called M Rousseau to show the PowerPoint presentation. M Rousseau was worried that it was a big risk not to be rehearsing, that it could jeopardize the concert. He was the one who suggested bringing in the Score Consultants, who were paid $150 per hour. The section leaders, however, convinced him that it would be worst to continue with the current score. And so it was: the section leads were locked in a room with Bob for four days writing down the score by hand.

In the next rehearsal the musicians were much happier. Now they finally had a better understanding of how the Neck Cracker should be. But things were still not perfect. M Rousseau stopped in the middle of the third movement angrily:

• Arrêtez! Arrêtez! That's not like it! It's sounding awful. Where's the harpsichord?
  
• See left earlier, sir.
  
• How come? We still have two hours of practice to go.
  
• Mary told us that she takes 2 hours to tune the harpsichord before every practice. She already completed 8 hours of work today, so she left.
  
• Ok, you there. Yes, you from the bassoon. Go there and play the harpsichord.
  
• But I can't play the harpsichord.
  
• It doesn't matter. We need this part. We are a team.
  

A few minutes later, M Rousseau stops again:

• Arrêtez! Arrêtez! Harpsichord, what's wrong with you? I can't hear you. Can you play louder please? This passage is fortissimo!
  
• I'm trying, sir, I'm trying.
  
• Harp, the same applies to you. Step on those pedals, will you?!!
  
• Ok, sir.
  

After weeks of hard work and long practices, M Rousseau announces changes:

• We don't have much time left until the concert, and I feel that we are lagging behind the plan. From now on, we'll play twice as fast the piece, so that we'll practice it more in less time. Please double your metronomes when practicing at home. I also realized that we are playing too many notes. We'll switch from the diatonic scale which has 7 notes to the pentatonic one which has only 5. This should make it easier for us to play. I also decided to remove the third movement altogether. It was kind of a downer anyways. We don't want Mr. Schneider to feel depressed. Are you ready to write down the notes we are taking out? It's very simple: please erase from your score all C sharp and F sharp.
  
• Sir, but the second movement is in C sharp. That's the tonic.
  
• Oops. Ok, in that case, we'll remove all D sharp. Call it version 2 of the score.
  

Rob Dubois complained:

• My grandfather liked D sharps a lot. It was the first note he learned on the piano. I think you should not remove the D sharps from the score. And you can't remove the D sharps without removing also all the G sharp, since they are a perfect fifth apart.
  
• Ok, we'll leave the D sharp, delete all C sharp and F sharp, except when any of these two are preceded by a lower A sharp. Call it version 2b of the score.
  
• Will you please make up your mind? I wrote the modifications by pen and now I can't erase it in the score.
  
• Sir, can you please confirm? For version 2 we'll leave all G sharp, delete all C sharp and F sharp, except when they are preceded by a lower A sharp? What about the D flats and G flats? They are not technically C sharp and F sharps, but they sound the same.
  
• Yes, you are right. We are calling it version 2b now …
  

And so they went discussing on for two more hours until the end of the rehearsal. At the end there were many different versions floating around, and the musicians had to spend time comparing notes measure by measure, which was quite time consuming.

A few days before the deadline, the challenges were still many.

• Folks, we still have a long way to go. We still need to practice more, but don't have much time to correct the errors. We decided to cut some measures. Please write down all the measures that we are deleting: 2, 76, 197, and 203 through 209, 305 and 776. Let's call it the amended version 7.5c now.
• But removing measure 76 the piece won't be moving and powerful anymore.
• Sir, the violins are playing one note that spans through 16 measures. It is too long.
• No worries. What if you tie one more bow on top of the current one?
  

When the whole orchestra began to play, M Rousseau had to stop the orchestra again:

• Stop, stop, stop! Viola, what happened? Why aren't you playing?
  
• Oh, for performance reasons we removed some notes we thought were unnecessary. We also removed all pauses, since we weren't doing anything. We just thought we could be more efficient this way. We ended the piece before the timpani solo.
  
• No , no, that's all wrong! Are you playing version 7.5c with yesterday's amendment? Violins, why are you all playing pizzicato all the time?
  
• Oh, sir, it's because the security in the building now classifies our bows as weapons. We needed to leave them in the coat room.
  
• Trumpets? What happened? You got all attacks wrong!
  
• Sorry, sir. We refactored the score. The 5/8 meter was causing integration problems with the rest of the orchestra, so we changed it to be 3/8 plus half of 2/4.
  
• Let me get my calculator. [Beep beep] Hum, I guess that could work. Let's compare measure by measure with all the other sections.
  

While they were speaking, the light went out.

• What now?
  
• The power went off, sir. I guess the UPS didn't kick in. When the lights are out we can't read the score.
  
• But according to the Presto Methodology you were supposed to memorize it anyways.
  
• Well, we thought we had agreed to play with our custom version of the methodology.
  

Mr. Borremans began to think he would have to cancel the concert and refund the tickets.

• Folks, I was discussing with M Rousseau here, and we are thinking about putting an accompaniment track that we downloaded from the Internet in the background and for you just to pretend you are playing. Yes, we found a version of the Neck Cracker on BitTorrent. Strings: just rub some soap in your bow hairs. Woodwind: remove the mouthpiece. Harp: we were thinking about putting some laser beams…
  

The section leads were angry at the suggestion.

• We have practiced all these weeks and now you just want to substitute us with a recording? That's not fair! We are only a few days away from the big day!
  

After much debate, musicians convinced the conductor and the artistic director that performing live was the right thing to do.

Hours later Bob Dubois stood up and said:

• By the way, when are we going to practice with the choir?
  
• What choir? Asked Brown, the concertmaster.
  
• The big choir. The Neck Cracker must have a choir. My grandfather always spoke of a great choir singing in the Neck Cracker.
  
• "Yes, yes, I have a group of friends that sing beautifully. I'll give them a call tonight. ", said M Rousseau.
  

  Brown replied:
  

• I don't think it's a good idea to include a choir a few days before the concert.
  

Mr. Borremans jumped into the discussion, which took hours. It was too much risk to include a choir at the last minute. In fact, everybody knew the only reason Mr. Borremans did not go for it was the cost of paying for all the extra musicians.

Finally the big day arrived. June 1^st. Everybody was nervous. How would the orchestra sound?

Mr. Schneider was up there in the reserved mezzanine, looking rather bored. Instead of the traditional A, the oboe sounded an awful A flat for the other instruments to tune. After half of the orchestra began to tune in A flat, John, who had perfect pitch, tried to whisper rather loudly:

• Hey, hey, that's not an A! Stop, stop!
  

The oboe player noticed the problem and raised half a step. The violas, the cellos and the French horns also noticed the mistake and tuned in A. It was too late: the orchestra already had different tunings. M Rousseau walked in with confidence and joy and everyone applauded.

The concert began. It was a mess. But nobody noticed. After all, it was a modern piece. Mr. Schneider fell asleep by the 50^th measure. The public thought that it sounded alright for a modern piece. In reality, they couldn't tell. It sounded as good as any other modern piece. Despite being very weird, nobody dared to comment. As the end approached, Mr. Schneider woke up with the gentle poke of his assistant and applauded effusively. The concert was a big success!

Real world software engineering - XIII

Software will more often than not have a very low quality. Population, therefore, will continue to have difficulties using and trusting software. The reason is that companies continue to seek the highest profit possible per project and impose the shortest deadlines, compromising quality this way.

Real world software engineering - XII

Automate repetitive tasks. Don't develop "bad habits".

Real world software engineering - XI

"It's the documentation, stupid."
In my Real World Software Engineering Methodology (RWSEM), developers realise that software specification is an important document. This software documentation is there to protect everyone and to increase the quality of the end software. This software specification should be a document that is binding, useful, and meaningful. It should not be obsolete mid development nor at the support phase. It should be binding during these phases.
I don't know why this is so unpopular. Perhaps the idea is too simple. We need fashion methodologies in the companie's web site and in RFPs, that in practice translate into the well-known terrible Chaos-Waterfall.

Real world software engineering - IX

Bug resolution: think twice more than 'debug'.

Real world software engineering - VIII

In Real World Software Engineering, one may create code keeping in mind that the stuff may have multiple uses in different scenarios. Instead of closing the door to infrequent scenarios, why not allowing the developer to include other basic functionality - perhaps even unsupported - so that customers can apply the software where it is useful? I get upset when I want to use software that has very strict functionality and it doesn't work for me (I'm usually trying something complex... that doesn't work until I 'bend' the software).

Real world software engineering - VII

Real World Software Engineering acknowledges that when Project Management is disconnected from reality, when "planning" is a false sense of security for management to feel good, the stage is set to disorganization, 'late' software, and low quality. Expect chaos and stress.

Real world software engineering - VI

Note to self: when you are blocked on the server site (and that will continue to happen frequently), use JavaScript on the client side. I thought JavaScript was a necessary evil; now I think it saves the day.

Real world software engineering - V

Recipe for debugging:
- Think about the error.
- Think about what you are doing.
- Suppress the things that you suspect are causing the error. Check the result again.
- Force an error so that your suspicion is confirmed or denied.
- Throw some salt over your left shoulder.
- Look hard at the source files. Let the bug stand out. If needed, keep staring for hours at your screen, waiting for the bug to manifest itself.