Ownership and Human Involvement in Interface Design

Good interface design is a complex engineering challenge with many dimensions. In this article, I explore the key dimensions of Ownership and whether a Human is involved.

Introduction

When I joined Temenos in August 2017, I came across a paper written by our Chief Enterprise Architect, John Schlesinger. The paper, written more than ten years earlier, discussed application integration. Having only a bit of experience in integration at the time, I struggled to completely understand the points made. The most striking point in that paper that I was struggling with was the "forbidden scenario," accompanied by the simple diagram shown in Figure 1.

John argued that this leads to tight coupling between the two resource managers and essentially makes the source resource manager a "hostage" to the target resource manager. In engineering terms, the nonfunctional characteristics of the source system are now a direct function of the nonfunctional attributes of the target system. For example, if the target resource manager takes 2 seconds to respond, these two seconds are added to the latency of any operation the source resource manager serves.

Figure 1: Forbidden Scenario of Application Integration [typo was in original source]

Too many questions came up: What about all these web services (SOAP and REST) I have been exposed to for so many years? What if I need two applications to exchange pieces of information both ways and do this very fast? Why is this scenario forbidden? Does it create some hidden implementation problems? And many more. The rest of the paper answered some of the questions.

Over the years, I have had the opportunity to discuss the concept in detail with John and other colleagues, tried discussing the ideas with many bank architects, and also met friends and other people at random developer events.

It took me several years to clear my mind of the concepts and convert them to an abstraction that guides my thinking when talking about interfaces. I see this confusion everywhere today, even in people holding extremely senior positions in large institutions. This article aims to establish a simple framework for classifying Interfaces and understanding some important concepts before implementing them.

Two Key Dimensions Defining the Type of Interface

In computing, an interface is a shared boundary across which two or more components of a computer system exchange information. The exchange can be between software, computer hardware, peripheral devices, humans, and combinations. (Source: Wikipedia)

Focusing on software, engineers will usually group/classify interfaces based on different dimensions. It can be grouped by domain, system, synchronous or asynchronous, protocol, integration pattern, etc. All these dimensions are essential for implementing an interface. But they don’t tell you in the first place how to make these choices. When should you decide to use HTTP or gRPC? Can I use Kafka or MQ for a given use case? Are there any differences I need to know if I make a particular choice?

I will introduce two key dimensions that, in my opinion, help a great deal in making educated choices when thinking about a specific interface: a) Whether a human is involved in the interface and b) Ownership of the systems involved in the interface.

Taking the two dimensions discussed in this section, we can classify all interfaces into four categories, shown in Figure 2.

Figure 2: Types of Interfaces

The terms Inter-Process Communication and Application Integration are widely used in the industry. I learned the term Interaction from John and my involvement in Temenos Architecture over the last few years. I had the opportunity to dive deeply into the concepts and the difference in treating them as a separate concept. ChatGPT provided me with the term Experience Orchestration after I explained what I was looking for. This concept is quite prevalent today in organizations using distributed architectures with a common user experience layer that touches multiple heterogeneous (often headless) applications behind the lines.

Understanding which type of interfaces you are dealing with is an essential first step in understanding how to implement them effectively. Understanding the forces at play for each type of interface helps understand the trade-offs.  

Inter-Process Communication (System-to-system; Single owner)

Inter-process Communication (IPC) is an old concept. Many complex applications are broken down into components (processes), and these components exchange data via programmatic APIs. In a monolithic application, this is often done using transactions and a single database.

IPC is a bit more complex in distributed applications built on microservices. When you have multiple components with maybe different databases, you need a different approach. Components can communicate in many ways, e.g., via the database (which is usually really bad), synchronous calls, or asynchronous mechanisms (messages or files).

In IPC, there is usually a single owner of the entire application architecture (a lead developer or lead architect). This owner can work with teams building different components and facilitate collaboration, often around common standards and protocols. Life is much easier when you have the luxury of common standards and protocols. If you have a challenge, the team can collectively find a solution or switch to a different standard or protocol offering different features.

I will not go into details about all the options possible for IPC, as there is usually a chapter on this topic in every book about application architecture or integration. Technologies such as database sharing, RPC, REST/SOAP APIs, various types of message queues, and Pub/Sub technologies are all possible options for IPC.

My opinion (architecture is all about opinions) is that IPC between two components that don’t involve humans must be done with an asynchronous technology that uses a broker to persist messages. This is the only way to achieve the essential characteristic of loose coupling, which greatly enhances the availability and scalability of the application. This is also the answer to the "forbidden scenario" point made in the introduction. The forbidden scenario has three main problems: a) transaction management problems, b) latency problems, and c) reliability problems.

If the application has been designed from scratch, it will likely have homogeneous components. If this is true, then IPC is even easier. Components can be designed to understand messages from other components and immediately transform them into a command.

Application Integration (System-to-system; Multiple owners)

When an interface needs to be built between two applications with different owners, without any human involvement, we have the Application Integration scenario. Application Integration is similar to IPC in some respects; for example, the asynchronous broker-based choice I would make in IPC, I would also make for Application Integration for more or less the same reasons. However, in this case, there is another reason to avoid synchronous technologies: ownership and separation of responsibilities.

When you have to integrate your application with another one, there are two main facts you need to consider: a) Your knowledge of the other application and how it works is usually low or even nonexistent, and b) Your control of how the other application behaves is again low or nonexistent.

The most robust approach to application integration (again, a personal opinion!) is the approach shown in Figure 3.

Each of the two applications to be integrated provides a public interface. The public interface should be a contract. This contract can be a B2B agreement between the two application owners. However, each application can also define and publish its external interface and commit to backward/forward compatibility, versioning, etc. Whatever the approach, the contract must not change, no matter what changes are introduced to the application. Each contract needs to include a flow that allows the application to turn a local event (which adheres to the application’s data model) into a business event (which follows a data model that the entire enterprise can understand—hence "business").

Figure 3: Integration via Contracts

The interface can be built after the two contracts are agreed upon. One or two adapters may be needed, and a component that facilitates collaboration between the adapters, using the broker as required. When designing the interface, it is critical to define these components and designate owners for them. Usually, these owners will have to be paid in some form to build and maintain the interface. Either of the two application owners may agree to own the interface, or a 3rd party can own the interface. Parts of the interface may also be owned by different parties (e.g., each application owner may own its adapter, and a 3rd party may own the collaboration logic and the broker).

The role of the broker is vital here. The presence of the broker, along with clearly defined ownership, makes troubleshooting of errors trivial. For example, when a message is lost, you can start investigating at the broker. If the message is there, then the producer is immediately declared innocent. Following this line of thinking, it is quite easy to identify the source of any issue and take the necessary action. You will also identify the owner for the corrective action without needing negotiations during times of trouble.

Sometimes, one of the two applications may not want to play ball. There are two common scenarios where this happens:

a) The application owner is too big to enter contract negotiations. In this case, the application owner will provide their public contract, and whoever wants to integrate must find their path. Sometimes, the application owner may be so big that they may decide unilaterally to modify the contract, usually giving some advance notice. The interface owner needs to be able to accommodate these changes.

b) The application owner is no longer available. This is a frequent scenario with legacy or abandoned applications. For example, you may have to integrate with a legacy application offering a TCP interface with a proprietary format. Again, it comes down to the interface owner to sort out the mess. The good thing in this case is that the application owner will usually not resurrect and do random changes, so the complexity needs to be managed only once.  

Interaction (Human-to-system; Single owner)

Now, I am shifting the attention to the user interface. When a user interface needs to be built within the scope of a single application (i.e., the application and the user interface share the owner), there is again the convenience of choosing standards and protocols, as well as a collaborative environment to facilitate change management.

What makes this interaction pattern very different from machine-to-machine communication is the presence of a human who has some capabilities (e.g., using the brain to understand an unknown error and asking for help) and some requirements.

The human/user wants the user interface to be available when he needs it. For example, an internet-facing application must be available ideally continuously (without downtime). If a user interface is meant to be used while in the office, then it’s sufficient for the system to be available during working hours. The gradual transition to working from home and flexible working hours for some occupations may change the availability requirements for certain user interfaces.

Then, the human/user wants to interact with the system conveniently. We call this "usability"; much research and expertise is available on making a usable user interface. A crucial part of usability, which is essential for modern interfaces, is responsiveness. When I started using computers about 25 years ago, it was okay to do something and wait a few seconds for the system to process your request—even minutes. Today, if I click a button and the screen takes more than 3 seconds to respond, I am filled with anxiety. After 5 seconds, this becomes rage. The human/user wants feedback as soon as possible. The communication must "feel" synchronous (even if the underlying implementation is not). It is difficult to define exactly what level of latency is acceptable. More than a decade ago, Amazon found that every 100ms of user interface latency costs them 1% in sales.

Another characteristic of Interaction is that the human/user manages the transaction (which is managed by the contract in the Application Integration). Thus, feedback must be timely and easy to act upon. Suppose we are triggering a simple straight-through action from a user interface that only touches one component of an underlying application and is completed in a few hundred milliseconds. In that case, we expect the user interface to inform us when the action is completed. If we are triggering a long-running workflow, we need to be notified, e.g., what the next step is, when it will be completed, and how we will be notified. If an error occurs and remediation is required, the human/user must be told how to trigger the remediation process.

Regarding opinions, the best way to implement a user interface is via an HTTP-based synchronous protocol. There is a straightforward argument to support this opinion—the Internet. Other mechanisms like gRPC or simple servlets are good ways to implement Interaction as well.

Experience Orchestration (Human-to-system; Multiple owners)

Many engineers/organizations implement Experience Orchestration the same way they implement Interaction. However, the nuances of ownership mean that there are better ideas than this.

If you implement Experience Orchestration with a direct API call to another system that you do not own, you immediately become hostage to that system. Your availability now is contingent on the availability of the 3rd party system. The same applies to latency. Your error messages and subsequent actions may depend on what information you receive from the 3^rd party system.

In my opinion, the best approach to implementing an Experience Orchestration interface is to use the approach outlined in Figure 4.

Figure 4: Implementing Experience Orchestration

Essentially, you need to build a self-contained application around the Experience interface and use this application to control/enforce the requirements of Interaction I have described in the previous section.

To control what you present to the user, your self-contained application must build its own API. You also need a persistence layer to synchronize the data you need and store your own state. These two parts of the application are essential to controlling availability and latency; they also allow you to manage your error messages, suggested actions, and other elements of your user experience.

If your Experience is complex enough, you will need a business logic layer to code the desired behavior and a workflow engine to manage long-running processes. While these two components may not be as important as API and persistence, they should be in place in enterprise-grade applications. The workflow could implement its user interface for managing tasks. A neater (but more complex) approach uses workflow engine events to integrate workflow tasks with the other existing user interfaces.

When you follow this approach, you exchange information with other applications following the system-to-system principles described earlier. Within your self-contained application, you can use any technologies you prefer for your Inter-Process Communication.

Summary

An important part of architecture is overlaying different views and examining reality from different perspectives. Applying the concepts described in this article to a fairly standard generic architecture diagram yields the view in Figure 5. Any mid-to-large enterprise will likely have all four types of interfaces I have described implemented at various places.

Figure 5: All types of interfaces co-exist in an Enterprise

In my day-to-day work, I come across business requirements for "Interoperability" or "Ability to Integrate." Some of my colleagues and I in the past casually answered that our software offers APIs and generates events, allowing interfaces to be built. While this answer is correct, it is only the tip of the iceberg. There is much more complexity in interfaces that most stakeholders often underestimate. Hopefully, after reading this article, your understanding of what an interface may entail will become more pragmatic.