Building Workflows in PHP with Ecotone
Almost any business requires Workflows. The type of Workflows we will need to build will depend on the Business Domain we work in. This may be fully automated flows like uploading images, resizing and storing them, or flows which require manual actions at some step like verification, signing or acceptance for example.
Workflows can easily get complicated and existing tooling more often than not create hard coupling between the Application level code and the related Framework. Therefore our code lose the clear business intention, and becomes mix of technical and business concerns. This as a result creates confusion in the code and makes Workflows much harder to understand and maintain, that they actually are.
Ecotone Framework takes approach of pushing the focus on the business side of the things. It allows us to create even most complex Workflows without the need to extend or implement a single Framework related class. This way business intention behind the Workflow stay crystal clear.
Yet before we will dive into how Ecotone can help us in building Workflows, let’s first understand what high level kind of Workflows we will be dealing with.
Stateless vs Stateful Workflows
We do have two main types of Workflows — stateless and stateful.
Stateless Workflows
- Stateless workflows operate without retaining any state from previous interactions, it acts based only on the given input.
This makes them ideal for things like Data validation and manipulation, Image processing or ETL (Extract, Transform, Load) scenarios.
Stateful Workflows
- Stateful workflows can remember previous inputs, decisions, or steps in a process, allowing for complex, multi-step operations that depend on earlier outcomes and time based actions.
This makes them ideal for things like Order fulfillment, Customer Service Ticketing or Document based Workflows.
Of course we can combine those two types together, in order to build hybrid solution. An good example of this could be Credit Card Approval Process - where first we would fetch and enrich Customer additional details (Stateless) and after that we would kick in confirmation and verification process (Stateful).
We will now discuss an example of Stateless Workflow. Yet if you are unfamiliar with concepts likes Command and Events, it will be good to first read some basic information on this matter.
Synchronous Stateless Workflows
I want us to start with most simple workflow type which contains of several predefined steps done synchronously. This will give us taste of how components can be easily connected together and will build foundation before we will dive into more advanced scenarios.
We will focus on Image Processing Workflow which will containing of three steps:
- Validating Image (Checking if file extension is correct)
- Resizing Image
- Uploading Image (To some external storage)
For connecting each step together we will be using Message Handlers.
Each Message Handler will be representing specific step in our Workflow which connected together will create “Image Processing Workflow”.
Ecotone provides way to create Workflows using input — output ports (Message Channels) in order to determine the next steps. Each input - output port can be easily switched to work synchronously or asynchronously.
Beginning of the Workflow — Command Handler
To kick off the Workflow we will first send “ProcessImage” Command from our HTTP Controller:
This as a result will trigger our Workflow starting from “validateImage” Command Handler and then will pass the Message forward.
- We define Command Handler attribute which we will be able to trigger using Command Bus. This will be our entrypoint to the Image Processing Workflow.
- Command Handler allows us to define outputChannelName which is a Channel to which result will be sent. We will discuss more deeply in a minute.
- The result of this method execution will be passed to the “outputChannelName”
Part of the Workflow —Meet Internal Handlers
As our Command Handler defines output channel “image.resize”, we need to define Message Handler which will receive the Message from this Channel. For this we will use InternalHandler:
- Input Channel Name state from which Channel this Handler will receive Messages from
- Output Channel Name states where the result of this method execution should go to
- The result of this method execution passed to output channel name
We’ve used in here Internal Handler, not an Command Handler.
Internal Handler is a Message Handler which is not exposed via Command Bus. This means it’s used only internally within our Application. As this is step within the Workflow we don’t want to expose it to be triggered directly and Internal Handlers help us achieve that.
Internal Handlers are meant to make our code explicit by stating that they are internal part of the Workflow.
If needed however, we could connect Command Handler with Command Handler. It all depend on our context.
Messaging way to connect things together
Each Message Handler, whatever it’s Command Handler, Event Handler or Internal Handler is connected through Message Channels. In case of Internal Handler we’ve stated that it’s input Channel is “image.resize”.
By stating input and output channels we define our Message flow. So “resizeImage” Message Handler will be executed whenever Message will arrive on “image.resize”.
Ecotone connects components using underlying Messaging architecture based on Enterprise Integration Patterns. This provides great abstraction for connecting different parts of the system with ease.
This way Messages can flow between connected Message Channels, and thanks for declarative configuration application level code stays decoupled from the Framework and Messaging concerns.
All things connected together
Let’s now add the last part of our Workflow “uploadImage” and see how it all connects together:
ImageProcessingWorkflow can be triggered just as it’s already, as there is no need for any additional configuration. Message will flow between different Handlers just as defined using declarative configuration with Attributes. This is really powerful way of connecting components using decoupled Messaging communication.
These three methods marked with Attributes define our Workflow, without extending or implementing any Framework specific classes. All this happens thanks to Ecotone’s Declarative Configuration combined with underlying Messaging architecture.
To see the code for described scenario, we can refer to Ecotone QuickStart repository. Examples on how to test this code will also be found in this repository.
Asynchronous Stateless Workflows
In a lot of business scenarios, we will be having situations where part of the Workflow will need to be handled Asynchronously. The need for that may come from handling given part of Workflow being too time consuming, need for queuing due too big resource consumption, or making some integration more reliable and resilient.
This is where often Workflows gets really complex and hard to follow. The issue is that most of the tooling treats asynchronicity as additional tooling, rather than part of our Workflow model. As a result Application have to pay the price of accidental complexity to maintain.
Ecotone makes asynchronicity first class-citizen thanks to Message Channels. We can easily introduce Message Channels to make code execution synchronous to asynchronous. As this is done via declarative configuration, we still write the code like it would be synchronous. Therefore the application level code stays clean and easy to follow, no matter of the execution model.
So let’s now make our time consuming tasks like “image.resize” and “image.upload” done asynchronously.
To do this we will introduce Asynchronous Message Channel (Queue), before calling Resize Image Message Handler:
To make our flow asynchronous after image is validated, we will add Asynchronous attribute to the resize image step:
It’s important to understand that asynchronous and synchronous are just correlations between steps, and it’s up to us how we want to split the workflow:
If we want we can make the upload Asynchronous too, we do it simply by adding the Attribute for upload image Message Handler:
Therefore our flow will looks like this now:
In Ecotone it’s not the Message that is marked as Asynchronous, as Message is just an Data Record. Instead it’s the Message Handler that states that it will handle given Message in asynchronous manner.
Defining Asynchronous Message Channel
So we’ve stated using the Attribute, that given Message Handler should be executed Asynchronously. However we have not defined Asynchronous Message Channel that should be used.
We did provide the reference name of our Asynchronous Message Channel inside the attribute, which in our case is “async”.
Now let’s define specific implementation of Message Channel for this reference:
In above example, we’ve chosen to use Dbal Message Channel, which is Database backed Message Channel.
If we would switch the implementation here, our Message would be stored using different implementation. For example we could provide RabbitMQ, Redis, Amazon SQS etc. To find our more, refer to the documentation.
To see the code for described scenario, we can refer to Ecotone QuickStart repository. Examples on how to test this code will also be found in this repository.
Testing Workflows
It’s crucial to have good support for testing Workflows. As it’s really easy to end up with tests taking ages, or ones that are hard to understand and maintain. Therefore the aim is to have reliable test suite, yet the test suite that is actually fast to run and easy to maintain.
Ecotone comes with support for Testing which allows us to:
- Isolate the Workflow or part of it which we want to run under test
- Execute the Workflow in tests like it would be done in Production
- Keep the tests quick and easy to follow
Ecotone Lite allows us test our functionalities in a way that our production level code works. Therefore the confidence those tests provides, is really high.
Besides that it allows for Bootstraping the test for given set of Classes, which allows us to isolate the Workflow from the other part of the system. This way depending on our need, we can include or exclude given functionalities from our Test Suite.
To test our Image Processing Workflow we will use EcotoneLite:
Without going into too much details what is important here is that within this simple test:
- We’ve created isolated scenario, where we can test Image Processing Workflow without triggering any other parts of the System
- We test asynchronicity with In Memory Channels which are quick. And In Memory Channel in Ecotone behaves like any other Asynchronous Channels, therefore they serialize and deserialize the Messages
- Even so we include asynchronous communication in the test, we still run it in a single process. This makes it easily write, debug and provide replacement In Memory / Stub implementations.
To find out more about Ecotone’s testing capabilities, we can refer to related documentation. To see the test code in action, you can refer to Ecotone QuickStart repository and take a look on tests catalog in each example.
Stateful Workflows
As we have seen not all Workflows need to maintain State. In case of step by step Workflows we mostly can handle them using stateless input / output functionality. However there may a cases, where we would like to maintain the state, the reasoning behind that may come from:
- Our Workflow branches into multiple separate flows, which need to be combined back to make the decision
- Our Workflow involves manual steps, like approving / declining before Workflow can resume or make the decision
- Our Workflow is long running process, which can take hours or days and we would like to have high visibility of the current state
For Stateful workflows we will be using Sagas.
Saga —the Stateful Workflow
Saga is basically a Stateful Workflow, which keeps the state of previous executions in order to make further decisions.
Saga communicates via Messages, it can subscribe to given set of Messages and publish new ones as a result.
We will build Order Processing Workflow.
- The process will start after Order was placed and will trigger an automatic payment.
- If payment was successful then the order process will marked as ready to be shipped.
- If payment fail however it will retried after one hour.
- If the retry has failed, the Order will will canceled.
Our Saga will be started when the Order Was Placed, this will kick in our Process:
Our “startWhen” method will be triggered whenever Order Was Placed event will be published. As a result, it will return new instance of Saga, which will be persisted in our Storage.
We can provide Custom Storage for Saga using Repositories. We can also use in-built repositories that integrates with Doctrine ORM, Eloquent Model or Ecotone’s Document Store.
Our Saga publishes the Event “OrderProcessWasStarted” and we can subscribe to it to trigger an action. So let’s do it now and trigger an Payment action, after our Saga is started.
We could inject here CommandBus to trigger the Payment. However in the example above we have leveraged output Channel to avoid using infrastructure level code in our Business Objects.
As taking an Payment involves calling external Service, it’s good to isolate the execution by making it Asynchronous. This way any potential failures will not affect storing our Saga in the database.
Making Message Handler Asynchronous helps in isolating failures and enabling safe retries. To find out more, refer to previous article on the subject.
Our Payment Service responsible for taking Payment could looks like:
Let’s add Event Handler for the Happy Flow in our Saga, that will mark the Order as ready to be shipped, when payment was successful.
As the Event contains of order Id, Ecotone will know which Saga should be loaded for this Event.
Ecotone provides a lot of options to map given Event/Command to specific Saga/Aggregate instance. More information can be found in related documentation.
We can expose this state to the Customer using Query Handler marked directly on the Saga.
This is the benefit of State based Workflows, as they hold information about current state of the Workflow, which can be exposed outside.
Let’s now handle our failure scenario, where Payment has failed and we should retry it after one hour.
We’ve marked our our Event Handler with Delayed attribute, to delay the execution by one hour. This way after payment has failed we will another retry after one hour.
And if maximum volume of retries is exceed, we will consider Order as cancelled.
Saga is a Stateful Workflow, as it’s running on the same basics as Stateless Workflows using Input and Output Channels under the hood. The difference is that it does persist the state, therefore the decisions it makes can be based on previous executions.
To see the code for described scenario, we can refer to Ecotone QuickStart repository. Examples on how to test this code will also be found in this repository.
Summary
Ecotone supports building Workflow using underlying Messaging architecture by providing Input / Output Pipes strategy, which on the high level works like seamless chaining of Message Handlers.
Besides that provides declarative configuration, which allows us to keeps our business code clean from extending or implementing framework related classes.
There are more features that Ecotone provides out of the box, which are out of scope of this article. Therefore if to explore more scenarios like below, please read related documentation page:
This all together create robust solution for building Workflows in PHP that stay clear about it’s business intent. Building Workflows does not have to be complicated with good underlying model.
