Initial structures for Service as Software, Part 2

I have been wanting to draw the structure of how a Service as Software could work on a high level. To kind of identify the challenging parts and to get an overall sense of the “big picture”. That’s what we’ll be starting today by expanding on the previous post.

Lets get going and first dive deeper into our tech service. The one performing the actual service delivery. Delivering a non-specified IT service using AI agents. We could maybe draw it like this:

We have agents on top of software running a specific process. And at this level we notice, there isn’t exactly anything new is there? One could do this with more traditional automation methods and many have. Key points that AI agents bring to the table are: Reasoning, Acting and Price. Agents can do things “out of the box” that previously would have required long periods of planning and coding. Capabilities that would have cost hundreds of thousands just a few years ago. Cababilities that can now be invoked with the cost of a few tokens. But agents do also change the playground, as they add a degree of unpredictability. As in, they are nondeterministic. Which is an aspect we need to come to terms with. Unpredictability is something one does not wish for their business processes, but something that can also be managed. Which we will try to do, as this is also the key, to unlock the kind of productivity and business capabilities we have not witnessed before.

Key players are the agents that everyone of us needs to be familiar with:

And angit LLM in the middle. Forgot to add it to the picture. The point is, get familiar. Agents are at the core of what we will be building together.

The situation in the first picture is probably where most companies are aiming at. Leveraging AI in spesific points to automate processes. Also trying to come up with ways to govern agents and AI. Or at least this was my perception when visiting the Google Cloud Summit Nordics in June. But we do remember the MIT report some time ago. Reporting 95% of AI pilots failing to deliver financial impact.

Still, at this level the approach is fairly simple from an automation point of view. There are multiple new tech aspects to consider like LLM security and new attack vectors that are new and important. But these can be considered additional layers around the actual automation logic. And the automation logic itself is easy to define if one knows the process well and the business operations on top of it are in an explicit form.

MIT report listed reasons as vague objectives and business integration issues. Also data quality issues and technology mismatch along with talent shortage. To me these sound like classical challenges in an automation project. Maybe with a bit heavier emphasis on data. But, I would bet that many of those cases failed already in the definition part. As this is the hardest part. Where one needs to convert the tacit business knowledge into an explicit form in order to automate it. And this is a classical problem. Where there are no correct anwers on how to do it. I would suggest one needs to focus more on the business part and listen to the business experts. Focus less on the tech parts.

But the reality is of course more complex. The business experts involved with the process could be busy, under resourced to the project, new to what is being done and simply threatened by the change that is coming. In these situations, respect is essential. The automation project might be a harsh reality for them, with big changes. Which is also a challenge for the automation project. We would need to get the technical side to communicate fluently with the business side, come up with clear explicit definitions and be able to push the changes through to actual benefits. We all know the realities of this.

I feel we need to emphasise the following:

Automation capabilities have grown and become simpler and cheaper to implement. The key is to have the disclipine to scope it well enough. It’s still ok to leave the hard part out of scope and do them the traditional way. The difficulties of the definition phase still exist, but due to new automation capabilites, we should now have more time and budget to focus on the definitions. Atleast we should, If we are to succeed.

Now, I will try to silence the “automation advocate” that seems to live in me. I just happen to be passionate about the subject and the benefits of productivity. We should do more to enable them…

Back to the main subject

Where the business logic would “locate” in our figure? Lets also take account of the new security considerations by adding some pre-prosessing and post-prosessing steps around the agents:

I might have added a steps too many. Sanitation, pre-prosessing and sec could be all under a single security step? Well, I draw these out of sync with my writing, so lets just go with it. Point is, these are necessary. And similar steps between the agents depending on the roles and data involved. But where is our actual business logic?

Part of it is of course built into the process and the softwares. And the “to be automated” part of the business process, as in, the business operations that are performed by humans on top of the process will be coded to the agents. How, depends on the business process. Part could be deemed determenistic in nature and coded to the process, software or as an additional determenistic automation part on top of the pre-existing process setup. Then rest coded to the agents, be it a form of workflow memory, few shot examples, episodic transaction memory and the functions and tools to perform the actions necessary. Agent collaboration structure and context engineering play a large part aswell. But before going deeper into those, we still need to go higher as I want to see “service as software” as a whole before diving into the details. Lets continue in part3.

Thank you for reading.

P.S. a couple of people have asked me along the lines “when will we see some code?”. At the moment, I feel that the few hours I have available for this blog is better used for conseptual thinking. So, there’s still some distance from any actual code. Also, I am writing this in hopes of contacting with people who ponder similar stuff. Please feel free to contact me in linkedin.

And see you next time.