Software Product Engineering vs Computer Programming

In this very personal post, I’d like to document an eagle’s eye view of the amazing products I have had the privilege to work on in my career as a developer, business owner, and engineering director. In summary, I’ll coalesce my learnings in bullet points next. These points sketch important learnings and I suppose an AI will generate something similar

4GL DBs

My very first job in Software Development was as a database developer in the Pick 4G RDBMS. The task was a project for a specific customer (JLB Smith Institute Specimen loan system) and had a defined beginning requirement and success criteria. These were very simple to understand and the bespoke solution I created fit exactly to the customer’s needs.

Globally Distributed Business Management System

My next product development foray was for a spin-off from the first (a US Information Retrieval company), and seemed just as simple: office software for a global business, where each office had the same mission - namely selling bibliographic CDs. The software ran the sales and delivery process. It could track products, royalties and help fulfill orders with shipping notes, invoices and finally receipts. There were hurdles, e.g. Office managers complained about the US formats. Adjustments had to be made. My solution was to make data structures and workflows more generic, and thus I had no need for ETL workflows, keeping the software complete and robust. The royalty tracking allowed the systems to pair at the end of the month in a peer-to-peer manner and exchange their data. In this way product data could be controlled in a decentralized fashion and sales were up to date all over the network. A special node was able to withdraw global report data. However, the software was hard to extend further, having been written completely in Visual Basic, for use case that had not been planned originally.

Global Hotel Solution

In my next role, I worked for the first time in a tight-knit team of developers, who were moving parts of a DOS based global leader in its sector to Windows. The CTO warned me about the limitation of the MS SQL database and requests I would be hearing from the sales department. He said that it was not possible to do the multi-variate analysis in the Database that the sales department needed and we would have to code it in our own DB Layer. We have no time for that, he stated firmly. As predicted, I soon heard from sales that this multi-variate problem was killing the company — all the customers across the entire global customer base needed this feature, regardless of how they used the software. I researched the problem, not buying my superior’s insistence that “he had checked this himself”. Of course, Stored Procedures were something brand new in MS SQL and I immediately understood that they were the correct technology, at the correct level in the architecture to extend our functionality across the board with minimal coding. They solved the multi-variate problem to meet all customer requirements, without breaking pre-existing requirements. They were backward compatible, as being simply an add-on.

Information Retrieval with XML/XPATH

Next I moved to a German Information Retrieval company called neofonie. Our product was an enormously high throughput indexer written in C++ called Blast, which was paired with a Java eco-system for crawling and flexible, template based ETL. The indexer was expanded with features over the course of 5 years. It could natively read XML, supported statistical scoring, page scoring and link-weighting algorithms, as well as a host of NLP (natural language processing) workflows (stemming, p.o.s, etc.) and super functions (geo-location). It supported a subset of XPath and was over-engineered for the requirements, catering for future requirements, that did not come from the customer. There was also a functional overlap with the crawler’s templating technology which already sent clean data into the system. This product was competing directly with Lucene, and although we had a head-start, we ultimately couldn’t hold our ground. The reasons for this failure are multiple, and clearly to be found in the strategy espoused by one of the founders. This experience taught me the grinding nature of progress and disruption. It underlines the imperative to aim high in product thinking: obsession with being the absolute best, and if it’s in a niche, completely understanding and feeling the trajectory of that niche.

Creating Node.js and Support Vector Machines

At the IR company, I also developed a single-threaded event-looped C++ http server. Yes, together with Peter Simons, whose concept it was, we implemented and sold node.js in 2005 to a large internet portal (Web.de), where the technology handled massive loads on the main landing page. There was no room for error. I backed Peter’s concept with the rest of the team and made sure that the multi-threaded JVM based architecture behind this super-fast and slim server could hold pace. For our customer we implemented an alternative backend process, also in C++, called class. This new process workflow was integrated with Google Search and used a Support Vector Machine to produce clustered results with an SLA of 500ms at up to 10M req/day. We trained the machine with snapshots from Wikipedia and our own curated datasets scraped from the Internet.

Internet for Africa

After this, I realized a long held dream, to attempt to provide many jobs for South Africans using IT. My idea was to promote a platform of existing tools that could provide income through the Internet even in impoverished rural areas. The reality on the ground turned this venture into an EdTech R&D project. My company developed an offline distributed P2P CMS which used in classroom situations to simulate typical social media activity in an asynchronous fashion. We called it awarenet, the same as the field project in which it was actually used. The product was sold to the CSIR (a governmental research agency based in Pretoria) and used to power data collection in rural communities. There were inquiries from other African countries, but these were ethically troubling. The principal learnings from this perspective were around negotiating build vs. buy with senior developers. The open source Software Moodle and further initiatives associated with OLPC could have provided an excellent base for our companies offering, however, the engineers insisted on building upon their own codebase (finally a hand crafted PHP framework serving templated html, css and js). The negotiations did not run my way and my hands were practically tied by financial constraints. The careful balance of tactical and strategic considerations was an important learning for me, since this and similar discussions are often a topic of friction between the R&D and product departments. Here for the first time, I was arguing from the product side, not the R&D side. This experience opened a lot of insights into how product management can ideally work. The final balance of this project 8 years after closure of the software company in 2017 is https://awarenet.org/ the social impact, public benefit sister organization, which continues to serve thousands of community members annually in the Eastern Cape. The software product is, to my knowledge, no longer in use.

MedTech Startup, Hybrid platform as a service: cloud/on-prem, P2P, DevSecOps, Open Workflows

Having moved back to Berlin, I was firmly back in the R&D department, building a highly regulated MedTech solution which allowed me to make good use of my knowledge of peer-to-peer networking research, cryptography, data modeling and databases to create a hybrid cloud solution for doctor’s practices at Doc Cirrus. The product included patient records or case file (lab, telemedicine, ophthalmology, etc.), calendar, messaging, invoicing and management functionality. Here, I am only mentioning the platform design of the monolithic services used to implement the various services needed in the network, as each module or internationalisation effort could be viewed as a product in its own right. The product was built from a business process management perspective, following the success of open platforms in other industries. These two powerful points of reference allow the product’s platform to give its customers a strong head start in extensibility, automation and efficiency. The tech stack was based on node.js, Express, WebSockets, YUI and mongoDB. The decade I spent at this company was rich with learnings from business to low-level tech (such as custom SCTP signaling in our WebRTC implementation). Software product development related learnings were along the following lines: bootstrapping, opportunistic business models, divide and conquer, build vs. buy (there it is again), independent quality assurance, developer-based product management, applied product research with ML pipelines, product management by proxy, strategic risk based product roadmap, optimization sprints/cycles, secure development life cycle, medical device certification and other regulatory learnings, contrast with the Swiss product where I was the PM. This company is currently thriving, with many of the same structures and people that I put in place - I wish them all the very best and am convinced they will succeed.

AdTech SaaS and ISO-27001

After 10 years, I felt that it was time to move on. I joined a SaaS company (Nowtilus) working on a high throughput and performance critical cloud product for server side advertisement insertion, which had several variations. In this position, I was reminded how important it is to have software developers who embody product management knowledge and are able to bring this into the team. Ramping up, I learned the importance of reproducible load testing, systematic test suites, trust in the team when there’s fire in the hold and close relations with product management. The latter is especially true when a product is very technical in nature. Such a product is characterized by the following: technical performance and integrations dominate the complexity; requirements are often expressed by API documentation and JSON Code; the end-user explanation of the product is one simple sentence; most customer feature requests to the product can only be understood by a highly technical audience, and you cannot easily explain them to a 5 year old without losing the meaning. In this position I had the privilege of working with some visionary product leaders. During the latter part of my tenure, learnings were focused around security (achieving ISO 27001 certification), automation, product delivery, roadmap strategy and devops practices. Fun fact: I was the de facto mongo DB, MS Azure and GCP Admin for the team, running some very large cloud installations. It was very interesting to compare the smooth nature of the GCP Dashboard and the younger Microsoft product (which often changed and delivered a much less intuitive UX). However Microsoft definitely scores points for their excellent AI development environment.

Take-aways...

Coming soon in another blog post.