An old irony in New York is the ubiquity of the ‘gourmet deli’. It is hard to find a deli that doesn’t proclaim to be gourmet. It is so commonplace that the word gourmet has lost all of its original meaning and perhaps taken on the opposite meaning. A similar phenomenon happened with the word engineer resulting in a dilution of its meaning. I recently heard someone describe themselves as a ‘domestic engineer’. While presumably tongue-in-cheek, the overuse and perhaps abuse of the term engineer has precipitated its loss of meaning and value. Other pithy examples include ‘sanitation engineer’, ‘sales engineer’, or ‘quality assurance engineer’.
Many will argue that title doesn’t really mean all that much and for the most part I agree. However being an engineer used to be more than a title: it also defined your role, your education, and your liability. I’m biased because I have a degree in Electrical Engineering and have gone through the rigors of the discipline culminating in the 8-hour long marathon Engineer In Training exam. One of my professors would state this disclaimer at the start of his Communications (where you design a wireless transmission network) class: “There are two things you won’t be able to do with a degree in Electrical Engineering: wire a house and fix a TV”. We all laughed at the time but it took time for the real message to sink in. His point was that engineering is a professional discipline that focuses on theory and design. We learned why things work and how to predict the behavior of systems we design based on first principles. It is not a technical trade that teaches you how to do perform specific functions. So we may not know how to wire a house, but it didn’t matter because under the hood we knew all the underlying theory.
The curious thing about engineering is that we often get confused as people who build things. The world of software has reinforced this stereotype, which I think needs to be addressed. At our core engineers are problem solvers: people who use mathematics, physics, and other scientific fields along with analytical skills to solve practical design problems. A civil engineer designs bridges and transforms building architectures into safe, viable structures. Electrical engineers design microchips, computers, wireless networks, power stations. Aeronautical engineers design aircraft. The list goes on and on. Note that in all these domains an engineer does not build these things as that is the domain of manufacturing or construction. Engineering is about design and prototyping that culminates in an explicit and detailed plan that describes how to build or manufacture the end product. The epitome of this philosophy comes from Qualcomm who revolutionized the integrated circuit (IC) business by focusing only on the IP surrounding the design of wireless ICs and eschewing manufacturing altogether. Other companies like ARM Holdings have since followed suit.
So where does that leave software? The challenge with software is that it is so widely adopted and the tools so easy that almost anyone can program these days (and in fact is now being taught to primary school students [1]). Knowing how to program is different from being a programmer, which is different from being an engineer. For example mathematics is taught throughout primary and secondary education but no one in their right mind would call themselves a mathematician unless they have a degree in mathematics. Most people I know with a math degree (including myself) do not refer to themselves as a mathematician. The same is true of physics: we learn a fair amount of physics but we don’t call ourselves physicists. The same really should apply to software: just because you know how to write software doesn’t mean you are a software engineer (a term I don’t particularly like). To be an engineer your understanding needs to be more than how to write software. You also need to understand data structures and algorithms, system design, resiliency and redundancy, analysis, numerical methods, etc. Without this foundation there is little guarantee that in production a system will operate at a level of reliability and accuracy that is expected of the system. This isn’t so different from a lay person building a house. While the house may stand and be livable, there is little analysis surrounding its structural stability, how much load it can bear, etc. I’m close to throwing out the baby with the bath water, so where does that leave us?
We are now truly in an information age, where even the most mundane products have become datacentric. This means that a product or service cannot exist without data to drive it. I’m not talking about Salesforce.com where you enter in data and retrieve it later. I’m talking search engines, mapping tools (Google Maps), recommendation systems (Amazon, Netflix), digital personal assistants (Siri), etc. I call this datacentric product development. Finance has been doing this for years under various guises. It is particularly prevalent in portfolio management, trading, and risk management, not to mention applications like creditworthiness or fraud detection. The term Financial Engineer embodies the role, which involves equal parts analysis, modeling, and development. These people are typically part of business groups and not part of the greater IT organization, which is usually designated a cost center. What is remarkable is that the role of Financial Engineer embodies this notion that being an engineer is about applying analytical methods to solve problems. The solutions just happen to be written in software. As datacentric product development becomes more widespread, being just a programmer will not be sufficient.
Organizationally I think this model is what technology product companies need to move towards. Product development needs more engineers and fewer programmers. What this means is that a single role must be responsible for understanding the business domain, assessing the problem, designing a solution, and illustrating that it works. The toolset of the engineer is more than just a Java IDE. It also includes working in languages like R or Matlab to analyze data and test hypotheses. In many cases this will also mean building a production system, while in other cases it may mean handing off to a vendor that specializes in building software. In some ways I’m trying to dial back the clock to the “good ‘ol days” but not for reasons of nostalgia. It’s really about recognizing what it means to be an engineer and what datacentric product development needs. (In a separate article I’ll share my thoughts on what I think an optimal organizational structure looks like when you adopt this model.)
The last question is what do we call this legion of skilled engineers? I’ve already professed my distaste for Software Engineer as I think this term confines the discipline to just software, which I’ve argued is only a small portion of the overall capability a person in this role needs. I could cheat since the organization I’m building is in the financial industry and stick with Financial Engineer but that still leaves an unnamed army to fend for themselves. Instead the emerging field of Computational Engineering [2] embodies the characteristics that I think datacentric product development needs. The reason is that the Computational Sciences are known for solving domain-specific problems using analytical and numerical methods, which of course includes writing software. People in these fields face the same challenges of those in product organizations: scaling, performance, reliability, accuracy of the model, etc. The term has not been widely adopted for Internet and business technology, but it is not difficult to see how it applies here as well.
If you’ve bought this argument and are interested in being a part of a new breed of engineer, don’t hesitate to contact me.
References
[1] Code Club: After-school group teaches children how to become programming whizz kids
[2] Graduate Education for Computational Science and Engineering
Cartesian Faith 
In Buenos Aires (Argentina) it’s called “Information Systems Engineering”.
Here, Computer Science & Engineering is separated in four degrees:
– Information Systems Engineering (sometimes only called “Systems Engineering degree” or “Systems Analysis degree”): 5 years degree. What you describe.
– Informatics Engineering: 5-6 years degree. Basically what you would call computer and software engineering.
– Computing Sciences: 5 years degree. Research oriented degree, focused on things like algorithms, discrete math and graph theory.
– Technical Programmer: 2-3 years degree. Designed to train programmers.
The “American” view of computing education really affects the expectation of local students, most of them english speakers that had been on the internet for quite some time. They expect to learn only “american computer science” in all of those degrees, so they get dissapointed when they have to study Organizational Sciences in ISE, Electrical Engineering in IE or very abstract math in CS. And they don’t want a “technical” degree, which is closer to a bachelor in computer science, they want their degree to say either “Engineer” or “Computing Scientist”.
Some universities changed their degrees to fit this expectations, IE degrees removed a lot of their Electrical Engineering content, and CS degrees added software engineering and practical programming classes.
Continental Europe has a similar system.
LikeLike
Thanks for sharing your thoughts. The degree distinctions you mention seem like a good way to organize it. Basically you have research (Computer Science), applications of computer science (Software Engineering), and then applications of computer science in conjunction with other physical/social sciences (Computational Engineering). Here things are further complicated since people graduating with a Computer Science degree are often hired as programmers. The variation in degrees and focus across programs makes it extremely difficult on the hiring side. I really think some form of standardization needs to be introduced (like traditional engineering has) along with greater clarity in the types of roles in industry. As data seeps into every nook of business and as everyone is required to know a little programming (like everyone needs to know a little math), this need will only become more acute.
LikeLike
As an outsider, the “rise” of Data Science seems to indicate some kind of specialization process already taking place. Something similar happened with “Interaction Design” or “UX Design”, which used to be in the scope of the programmer and now it’s a young and independent design dicipline.
LikeLike
Good point about Data Science. I’ve spent a lot of time brooding over the term Data Science. My assessment is that the term Data Science and those who call themselves Data Scientists are different from what I’m calling Computational Engineers. The reason is that if you look at the literature the role of the Data Scientist centers around “telling stories through data” (sorry, I don’t have a citation handy). This is a valid exercise and useful in say a Marketing department where you need to understand consumer trends, etc. To accomplish this task clearly a lot of the tools overlap with product development and engineering but the focus is totally different. If I were to phrase Computational Engineering using a similar structure I would say it is about “solving problems with data”.
LikeLike
I have always admired the non-American degree programs. Can’t say I understand why in the US we’ve got a fascination with this rigid 4-year template for all fields.
LikeLike
Because of the Bologna Process, Continental Europe had already implemented 4-years degrees equivalent to the Bachelor, and here (Argentina) they implemented the so called “middle degrees”. For example, after completing 3 or 4 years of a 5 or 6 years degree, you earn an Analyst’s degree (either in Systems, Computing or Informatics). Some english speaking countries accept this middle degrees as equivalent to bachelors, like New Zeland or Australia, but here they only mean “you are half way”.
There is resistance to this process, because of tradition. Engineers are as much respected as Lawyers and Doctors, with similarly long degrees. It used to be common, and it still is for more “traditional” engineering diciplines, to be called “Engineer Smith” or “Engineer Jackson”. The idea that you could be an “engineer” with a 4-years degree is kind of strange and rejected around here.
Tradition is strong in local engineering. I’m finishing a 5-years long degree in Information Systems Engineering (Computational Engineering), and I had to take year-long classes in Technical Drawing (actual pencil, paper and drawing board) and have to do “Recorded Professional Practice” like civil and mechanical engineering students.
LikeLike
What do you make of the seemingly prevalent distaste among programmers for “technically” or “theoretically” trained programmers? We’ve all heard the anecdotes about the CS/math major who is terrible at, say, JavaScript, compared to the English major who shows more competence. Personally, I feel these anecdote arise from a mismatch in expectations rather than a mismatch in skill sets. If that’s true, how does one counter the push to “just make things work”?
LikeLike
I agree that it comes from a mismatch in expectations. At the beginning of a product lifecycle pragmatism and making things work is usually what is needed. Once the parameters of the product are understood and the trajectory stabilized then you need to start focusing on the design. I think people with more theoretical or research-based degrees don’t always get the opportunity to balance pragmatism with design/theory whereas for the English major or others coming in from outside fields they are driven by pragmatism. The flip side is that people without formal training in say algorithms and data structures (myself included many years ago) tend to suffer from lazy reinvention of the wheel (which adds overhead) and can potentially plateau at a sub-optimal level. At the end of the day you need people like my idealized Computational Engineer who can do both. The danger is that those who never learn the theory will get stuck in a commoditized role.
LikeLike
If one asks whether datacentric development is the goal, one needs to ask why it is that neither academic nor cube drone coders use high normal form relational databases? For non-data centered applications, such as games, using files is fine. But for any industrial strength application, then datacentric should lead to Dr. Codd’s model. Yet, both sorts of coders (nearly) always opt for [un|de]normalized data dumps in various SQL databases.
The usual justification: RDMBS engines aren’t fast enough to process the necessary joins, and the data just looks better in Excel spreadsheet structure. Now that multi-processor/core machines and SSDs the norm, there’s little reason for multi-user datacentric applications to be built just like your granddaddy’s COBOL/VSAM systems of 40 years ago.
LikeLike
Pingback: The myth of the missing Data Scientist « Cartesian Faith
I Like these line of your article We are now truly in an information age, where even the most mundane products have become datacentric. This means that a product or service cannot exist without data to drive it.
software product development
LikeLike