Story One: A Memory
When I was a little boy and went to elementary school, the following story happened to me. During a break, I was in a classroom with several friends of mine. Suddenly, a flower fell from the shelf. At that very moment, the teacher entered the room and accused us of vandalism without looking into the situation. We were reprimanded and our parents were called to school. It was so offending and hard to understand that I remember this situation even today. Since then, I have often wondered what made the flower fall.
The most logical explanation would be — it just fell. It was simply growing, throwing new shoots and becoming bigger, stretching to the sun. One day, the projection of the flower’s center of gravity moved beyond the prop and it fell.
I don’t know why I’m writing all this. Perhaps, melancholy came with autumn and yellow leaves. Sometimes it seems to me that programming develops in leaps and bounds and new technologies are created so fast that one day people won’t be able to control all of this.
Story Two: Programmers Don’t Know Exactly How Their Programs Work
I remember quite well the first program I was paid for. It was a very long time ago. In those ancient times, there were no computers in my country. At least, they were pretty rare. A commercial organization has decided to produce their own computers.
It was an improved clone of “Specialist” based on 580 processor. They were going to create completely new software of their own for this machine. The first step was to write BIOS. I was the one to make the very first procedure: a RAM test. The produced computer was so unreliable that I could not use temporary variables in memory for the RAM test. In addition, I couldn’t store anything in RAM until it was tested. Actually, it’s quite obvious. But how could I write a memory test having an eight-bit accumulator A and six eight-bit general-purpose registers B, C, D, E, H and L? Moreover, the test should print how much memory has been tested in the upper right corner of graphical display: 8 KB, 16 KB, 24 KB… I worked two of three days on this program. There were up to one hundred lines of assembly code. After checking it all, I was really proud of myself and saved the program on a magnetic tape. When I came to the office to show my work, turned out the tape could not be read on the tape recorder. I had nothing to do but write the entire code from memory (!) on the “Specialist” at the office. When we checked, everything was working. Yay! I got some big money for the job.
In those old times, I understood ABSOLUTELY EVERYTHING in my program.
Since then, I have written programs on the assembler of 6502, Z80 and x86 processors. Some time later, I fell in love with Forth language. I also could do some things in FoxPro. Finally, the times of C/C++ came. They said the one who could understand MFC was the top-class expert. So I learned and mastered MFC. I wrote drivers for Windows 2000/XP/w7 and Linux. When it was necessary to make a website, I used html, css, js and php. Then I did some work with microcontrollers: Atmel, STM32, and then x86: MMX/SSE. And then a little bit more of C/C++: need boost and QT? Okay, let’s use them. At the same time, someone asked me to make a program for the Android tablet (Java+JNI+cpp) in Eclipse. I also made a system in PLD Altera and used Verilog language, test benches, and other hardware fun.
And you know what? I’m not sure that I fully understand how all these systems, programs and algorithms operate. Nothing left of my former confidence. According to my calculations, I had to learn and use up to three new technologies, programming environments, libraries and languages a year. I definitely wasn’t sure that I understood and knew everything. In fact, we usually understand only the basic principles…
Moreover, five years ago I thought I knew, understood and loved C++. Nowadays, when new standards like C++11, C++14/17 came, it’s really hard for me. My colleagues fix my code changing my typed variables to auto type… Now what? Do they really make code more readable? Here’s another piece of not very recent news: it is undesirable to use inheritance, and it’s better to use templates. This makes me really sad as I don't like templates.
I could probably hit rock bottom but private talks with other programmers with a glass of brandy revealed that I wasn’t the only one having such problems and being so outdated (not everyone is ready to confess the fact. It depends on how much alcohol they drank before). Some of them, just like me, do not fully understand someone else’s code. They do not fully understand the way our system works and don’t know what third party libraries we have set up and use. They don’t understand why there are 67 threads in quite a simple program and why it takes 350 megabytes of memory.
There’s another thing that disturbs me more in the story of high-level programming and it is the “magic line effect”. I have recently invented this term myself. It’s not about bugs that are certainly always there. It is about one line of code that fundamentally changes program features.
For example, we are writing a program using QT libraries. We have our own class inherited from QGraphicsView. We use our class, place QGraphicsScene on it, then add various QGraphicsItem and live happily but slowly. Finally, we add a magic line to the code:
QGLWidget* gl = new QGLWidget(); this->setViewport(gl);
Our application becomes pretty fast as now it uses hardware acceleration of OpenGL to render graphics.
— Hey! Programmer! Do you know what OpenGL is?
— I’ve heard of it, but never used it.
— Me neither. But now we use it and it works! Look how great things are now!
— Hurray! But it’s just one line of code… and two or three additional DLLs in the installer.
In a week or two. Listen, it’s not so good anymore. We have some problems with the media player and QGraphicsVideoItem… We should do something about it. In a week, another magic line appears:
— Hey! Programmer! What the hell is this?
— Well… I don’t know for sure but I guess the translation mode of OpenGL queries in DirectX (Angle) is used and it seems like things have become better. The CPU is not so loaded and it seems that video runs without errors.
— Okay, let’s see what QA will tell us…
Trouble No.1: Programmers do not fully understand how their program works. Is there any way to understand? How much time is necessary for this? What about time-to-market? Is it the time of “technological singularity” when new technologies appear faster than we can understand and master them?
Story Three: Complete Misunderstanding
Three years ago, our company was lucky to join a very promising project. Well, we thought it was promising. A company manufacturing equipment has decided to equip its machines with an electronic control board. We had to quickly develop and produce the board plus software for the microcontroller.
To tell the truth, I was really happy about this project. First, I was proud to work for the national company. Secondly, it seemed that the project was quite long-term as the range of machines was wide and the customer wanted to keep up with the times and produce competitive equipment.
However, there were some things to be mentioned:
- The Customer had absolutely no idea what microcontroller software was and how boards were produced. He even said that they could buy a line for installing boards if necessary and produce them themselves. Imagine how great his intentions were?
- As for us, we did not quite understand how their equipment worked and the area of equipment control was pretty new to us. It was supposed that the customer requirements on the algorithm of controller operation would be given to us after we sign the agreement.
I don’t know why but somehow we charged them a minimum as the project was long-term and there was lots of work ahead. The customer gave his verbal consent.
Then, some strange things began to happen. As it turned out, there was just one engineer on the customer’s side. He scrutinized the algorithm, described it himself, often called us and wanted to know how things went. In addition, I felt that we were doing something wrong. Our algorithms obviously lacked math. There was not enough of control engineering, proportional–integral–derivatives and other tricks of automatic control systems. We tried to move development towards real control engineering but their reply was that it was too complicated this way. The strangest thing was that their engineer was the only person interested in algorithms on the customer’s side. When we finished the works and were demonstrating produced boards I realized what was going on. The engineer thought of the board as of any other part of an aggregate turned on a lathe or bent on a bending machine.
This meant that they wanted a drawing with indicated types of material, as well as sizes and allowances. There was a detail: we had to place it on its seat and that was it.
Well, what can I say? I tried to talk to them and explain that software was not a detail, that long testing on real hardware was necessary. That it could turn out the implemented algorithms might require adjustments or changes in time constants, response time, etc. My phrase that software might have bugs that needed to be debugged faced a wall of misunderstanding. Turned out, our bugs were our problems. ALL bugs, 100% of them, should be removed before we submit works. After that, the engineer carries out his testing and accepts works. That’s it. The project is finished. No more changes to the software are foreseen and the product can be passed to series after the pilot batch.
Needless to say that the customer’s team has carried out the testing with the help of one engineer in one (!) day. In a week, he called us and asked to fix two time parameters in the algorithm.
Wow, that really hurts. We have put lots of effort into this project but we never understood whether the work had been carried out the right way. We never saw the machine under the control of our board.
It all ended as suddenly as it began. After some time, the customer has calculated that the production of printed circuit boards was not profitable in our country. The project was closed. Perhaps, it’s for the better, who knows…
Let’s take a break from the described case. I want to highlight an important thing: every programmer knows that his program has or might have bugs. We do write tests and perform continuous integration but lots of us have a list of outstanding problems in the bug tracker even for the product that was released long time ago. Sometimes a program comes to the stage of a “fatal test loop”. It is the situation when new adjustments to the program lead new unknown bugs. QA returns software for debugging, then developers fix them and send back to QA who return the program for debugging again and again. If the customer is adequate, he is ready to put up with some bugs that are sometimes called features not to escalate the situation.
On the other hand, the customer who has never faced the process of software development understands the process in a completely different way. For lots of people, a program is something they can see and touch. That is, it’s only a part of GUI. It is something we can see with our own eyes: buttons, forms, dialogs. If there’s a board, it’s important to see how it looks like and hold it in our hands. They don’t know what’s behind it, and don’t know what logic and algorithms are in there. That’s where the misunderstanding comes from.
Trouble Nr2: The customer almost never can check the quality of the developed software, even if he has access to the source code.
If we now consider the two described troubles as a whole, we will see the situation when programmers do not fully understand how their programs work while the customer cannot check the result. What will come out of it? What future awaits us? How many under-products will be around us? Will it be dangerous to enter high-rises, pass over bridges calculated in engineering CAD? Will we be afraid to fly and drive robotic cars? Can we trust medical devices? Should we be afraid to log in to online banking? Software products are everywhere around us. Perhaps, I wouldn’t be so scared if I wasn’t a programmer…