2016-08-26

Beauty of #microservices: part 4 from agile development to agility of enterprise systems

1 Introduction


This blogpost is inspired by several blogposts about microservices and it is based on the blogpost [REF1] “Architecting #cloud-friendly application architecture #apparch (inspired by #microservices)” http://improving-bpm-systems.blogspot.ch/2015/04/architecting-cloud-friendly-application.html which uses other blogposts about microservices http://improving-bpm-systems.blogspot.ch/search/label/%23microservices

See also the previous blogposts of “Beauty of #microservices” topic.

2 Agility during the whole application lifecycle


This blogpost is named after my 10 years old presentation “From agile development to agile evolution of enterprise systems” ( see http://www.slideshare.net/samarin/from-agile-development-to-agile-evolution-of-enterprise-systems ) with a small change to introduce the word “agility”.

This done because of one of Matthew Kern’s blogpost “Fixing Agile with Architecture” (see https://www.linkedin.com/pulse/fixing-agile-architecture-part-4-matthew-kern-cea-cissp-issap-pmp- ). Obviously, architecture works with a longer timespan, e.g. the whole application lifecycle ( see also http://improving-bpm-systems.blogspot.ch/2014/03/enterprise-patterns-ear.html ) and application development covers only the delivery of the first production version. Thus architecture must guide the application development.

Considering that the TCO of a custom-developed application is only 20% for the implementation of its first production version then the rest 80% must be taken seriously. A real-life example: the cost for a small modification (changing an UI field from free-text to drop-box) in a custom-developed ERP is 20 KCHF and the implementation time is 3 months.

The agile development does guarantee that prototyping and implementation of a new application can be done rather quickly. But, the agile development does not guarantee that the production version of an application can be changed very quickly. Just compare – change a wheel of a parked car and change a wheel of the moving car.

The table below compares the level of frequency of changes for various application building blocks ( see the part 3 -- http://improving-bpm-systems.blogspot.ch/2016/08/beauty-of-microservices-part-3-employ.html ) during different phases of the application lifecycle.



Building blocks with the low level of changing frequency may be kept as microservices, services or even be embedded into a monolith. Building blocks with the medium or high level of changing frequency must be kept as microservices.

Thus the latter can be changed quickly and safe (without damaging the running application) because each new version of microservice assemble is a different process-template (a formal description of the process). Each process-template may be enacted as several process-instances.

The distinction between process template and process instance is very important. The lifecycle of process-template is controlled at design-time. The lifecycle of process instance is controlled at run-time. A process instance is created, maybe suspended & resumed and finally terminated. Many process instances from various templates may co-exist at the same time as shown below.



3 Conclusion


Microservice architecture, together with BPM, resolves two fundamental problems of the agile development:
  1. building blocks (which are discovered by BPM) are to be implemented à la agile development and 
  2. some building blocks are to be implemented as microservice thus enables agility of enterprise systems (quick and safe changing of production applications).
Thanks,
AS

2016-08-24

Beauty of #microservices: part 3 employ #BPM to tame the service granularly beast

1 Introduction


This blogpost is inspired by several blogposts about microservices and it is based on the blogpost [REF1] “Architecting #cloud-friendly application architecture #apparch (inspired by #microservices)” http://improving-bpm-systems.blogspot.ch/2015/04/architecting-cloud-friendly-application.html which uses other blogposts about microservices http://improving-bpm-systems.blogspot.ch/search/label/%23microservices

See also the previous blogposts of “Beauty of #microservices” topic.

2 Service granularity


A recent article from Zhamak Dehghani of ThoughtWorks https://dzone.com/articles/taking-microservices-home says “Service Modeling is Difficult” and “Find your Granularity”.

Of course, the service granularity is difficult, because it is usually done bottom-up, i.e. in the solution-space without systematic understanding of the problem-space.

What could be a good enough description of a particular problem-space? Ideally, the following is necessary to analyse objectively a particular problem-space:
  1. An domain-of-interest (e.g. business unit, group of processes, etc.) whose unsatisfactory behaviour is qualified as a problem.
  2. An explicit and machine-executable model of this domain-of-interest.
  3. Data those are sufficient to simulate the behaviour of the domain-of-interest with its model.
To build an explicit and machine-executable model of a particular domain-of-interest, this domain-of-interest must be considered as a system. Considering the following definition of the concept ‘system’ “set of interacting discrete parts organized as a whole which exhibits (as the result of interaction between the parts) some emergent characteristics indispensable to achieve one or more stated purposes”, those “discrete parts” have to be identified first. In other words, it is mandatory to find something that will allow to state: “this domain-of-interest is a system of ”.

Typical business artefacts that are used to describe a particular business are: capabilities, functions, services, processes, roles, data, etc. Is it correct to state that a domain-of-interest is a system of capabilities? System of functions? System of services? Not sure – they (capabilities, functions or services) can be explicit, but they can’t form a machine-executable model to simulate the behaviour of domain-of-interest.

So far, it is correct to state that a particular domain-of-interest is a system of business processes ( see http://improving-bpm-systems.blogspot.ch/2014/03/enterprise-as-system-of-processes.html ).

Each business process is an assembly of the following artefacts: events, human activities, automated activities, services, processes, data, documents, rules, roles, records (audit trails), reports and KPIs. All those artefacts are designed in accordance with their domain because they are specific for a particular process or a group of processes. Thus, domain-driven design is automagically followed.

For example, data structures comprise some data necessary for a particular activity and those data are sub-sets of an enterprise data model (extra blogpost will be written on this). Roles are around a particular activity – who can execute this unit-of-work, who can supervise it, who can consult it, etc. Those roles are composed from the enterprise-wide organisational and functional roles.



All those artefacts are units-of-functionality with single responsibility.

All those artefacts have well-defined interfaces.

The majority of those artefacts are stateless.

Thus there is only one step away from the microservice architecture – just wrap those artefacts as services and deploy them separately.

Some of those artefacts in some BPM-suite tools are already microservices! In other BPM-suite tools, some of those artefacts are in the same JVM (see “programmable monolith” in http://improving-bpm-systems.blogspot.ch/2015/12/typology-of-platforms.html ).


3 Conclusion


The use of BPM (which is a top-down analysis of a domain-of-interest) resolve the following difficulties of the microservice architecture:
  • granularity of microservices is defined in accordance with the domain-of-interest, thus “domain-driven design” is achieved
  • relationships between microservices become explicit and they are known at the design-time
  • the number of relationships between microservices is reduced thus complexity is reduced as well

Thanks,
AS

Architecture is about WHY

Yet about definition of architecture which may be more understandable by some audiences.

MASTER definition

architecture, noun
totality of fundamental concepts or properties of a system in its environment embodied in its discrete parts and relationships, and in the principles of its design and evolution

Another definition

architecture of a system-of-interest, noun
description why this system-of-interest is designed as shown in its blueprint

Such a description must be
- explicit
- computer-executable (i.e. formalised)
- multi-viewpoints
- understandable by all the stakeholders of the system-of-interest

A blueprint of a system-of-interest is only one of the architecture viewpoints.

Thanks,
AS


2016-08-06

#IoT as a system of digital contracts (thanks to #blockchain, #BPM, #ECM and #cryptography)

Ability to work together is one of the essential enablers for the civilisation. At present, countries, companies and individuals can work together. Considering the current huge interest to the IoT, this blogpost outlines how to enable the things also working together with other things and people.

From the three following definitions of IoT, the last one is the better fit with this blogpost goal.
  1. Wikipedia states that “the internet of things (IoT) is the network of physical devices, vehicles, buildings and other items—embedded with electronics, software, sensors, actuators, and network connectivity that enable these objects to collect and exchange data”
  2. The IoT European Research Cluster (IERC) definition states that IoT is “A dynamic global network infrastructure with self-configuring capabilities based on standard and interoperable communication protocols where physical and virtual “things” have identities, physical attributes, and virtual personalities and use intelligent interfaces, and are seamlessly integrated into the information network.”.
  3. The Alliance for IOT Innovation (AIOTI) report http://ec.europa.eu/newsroom/dae/document.cfm?action=display&doc_id=11810 says “The IoT represents a concept and a paradigm that considers pervasive presence in the environment of a variety of things/objects that through wireless and wired connections and unique addressing schemes are able to interact with each other and cooperate with other things/objects to create new applications/services and reach common goals.”
A typical pattern for working together is the following – establish a contract, carry it out and close it. Considering that the things understand very well digital information, a contract must be digital to be used by the things.

As explained in “Digital contract as a process enables business in the digital world (thanks to #blockchain, #BPM, #ECM and #cryptography)” http://improving-bpm-systems.blogspot.ch/2016/07/digital-contract-as-process-enables.html, a digital contract is an explicit and machine-executable process between several business-entities, primarily, things and people.

For example, a new future household fridge will have as minimum four types of contract simultaneously: 1) with people who are living a particular household, 2) with a producer and service company for this fridge, 3) with some online shops to order various food, and 4) with all other things within a particular household to achieve together some goals of energy consumptions.

In general, the things are contracted to cooperate with other things and people differently:
  • orchestration of several “slaves” by one “master” (also known as strong coordination)
  • choreography of two or more business-entities (also known as contractual coordination)
  • goal-achieving by a group of participants, like a football team (also known as weak coordination)
Of course, the same thing holds different roles in different digital contracts.

Fortunately, digital contracts as explicit and machine-executable processes are powered by BPM which already knows how to handle all these complexities.

But, can BPM-centric digital contracts provide necessary processing for high data volumes are being generated? Yes, with the help from microservices. Let us consider that all automation activities in a process are implemented as microservices (which are potentially externalised from a business process engine).

Firstly, each process (actually a digital contract) may be executed in a partially independent environment – only a common part would be a blockchain with all the records (or audit trails).

Secondly, each running process can anticipate what microservices will be required and can prepare secured and individual instances of microservices (with one-time passwords).

Imagine a football match. The venue starts an individual “customer” process for each person on the stadium. Knowing the customer, such a process prepares microservices which are the most probably used by this person.

Thanks,
AS

2016-07-27

Digital contract as a process enables business in the digital world (thanks to #blockchain, #BPM, #ECM and #cryptography)

In this blogpost I aim to outline how a combination of modern technologies (including the blockchain technology, BPM and cryptography) can enable good practice for carrying out business in the digital world. Such a combination enables 1) the establishment, 2) the execution and 3) the imposition of digital contracts digitally, quickly, surely and (potentially) legally. Thus a digital contract enables business can be carried out fully digitally, with less risk and fewer resources.

Related blogposts:


1 Starting from good business practice


One of the major prerequisites for doing business is a guarantee that all the risks relating to working with somebody else (potentially a stranger) are properly addressed (mitigated, hedged, etc.). A recurrent pattern found in good business practice comprises the following actions:
  1. document, as a legally binding contract (which is voluntarily approved by all the business parties), all the related business relationships, responsibilities, conditions, activities, performance indicators, schedules, conflict resolution procedures, etc.;
  2. impose the execution of this contract on all the business parties;
  3. impose agreed liabilities on some business parties.
Let us try to combine several modern technologies to implement this good business practice such that it
  • is fully digital (without need for the mandatory involvement of middlemen such as lawyers or escrows),
  • is legal (admissible in courts) – of course, after necessary governmental decisions,
  • is universally accepted (unrelated to location and nationality), and
  • has negligible cost and overhead.
One of these technologies is the blockchain technology which enables the creation of immutable, tamper-proof and transparent records https://www.linkedin.com/pulse/summer-reading-viewing-update-blockchain-arno-laeven ). Despite the existing hype around it ( see https://theconversation.com/blockchain-really-only-does-one-thing-well-62668 and https://medium.com/@pavelkravchenko/decline-of-blockchain-hype-and-rise-of-a-common-sense-8de5789a794d#.olmj0go1e ), the blockchain technology is mandatory for doing business in the digital world but alone it is not enough.

Potentially, digital contracts will further develop and properly implement promises of “smart contracts” blockchain-based applications which are rather overhyped ( see http://www.coindesk.com/smart-contract-myths-blockchain/ ).

2 Digital contract as an explicit process


The key of doing business properly is a contract which is an agreement with specific terms between two or more business parties in which there is a commitment to do something in return for a valuable benefit known as consideration.

A good contract must have several factual elements. They are listed below together with some modern technologies and methodologies which enable those elements in the digital world. In general, any promise, commitment, document, written or oral (e.g. by Skype) communication between business parties (which are involved in a contract) is secured by the blockchain technology. The last is acting as immutable, tamper-proof and transparent storage of records.
  • a commitment to perform – is enabled by business processes (and BPM) as a formalised description of some activities to be carried out;
  • a valuable consideration (which can be a commitment or payment in some form) – is enabled by business processes (and BPM) as a formalised description of some activities (i.e. payment) to be carried out;
  • a time or event when activities must be initiated (to meet given commitments) – additional detailed characteristics about scheduling and coordinating activities to be carried out; scheduling and coordination can be formally expressed by various coordination techniques (see http://improving-bpm-systems.blogspot.ch/2014/03/coordination-techniques-in-bpm.html );
  • terms and conditions for performance, including fulfilling commitments – can be formally expressed with various decisional business rules (also known as decision management) and behavioural business rules.
  • demonstrated performance – can be collected and treated by fact-based performance indicators, business performance reporting and fact-based predictive analytic;
  • an offer – is actually part of the contract how it is proposed by one business party, e.g. service provider; thus an offer can be formally expressed with all the mentioned about technologies and methodologies;
  • an acceptance of that offer which results in a meeting of the minds – various cryptographic technologies and document management help to enforce integrity of documents, identity and authorization of business parties and confidentiality of all documents and communications.
Thus a contract can be formally expressed as an explicit business process which comprises:
  • pertinent business assets (goods to be delivered, payments, etc.);
  • agreed business activities to be carried out by the business parties and necessary inputs, outputs, guidance and resources for those business activities;
  • planned coordination between activities – which business party is doing what, when and for how long;
  • some business events – things that happen or take place which are considered important for this contract; they can serve as stimuli for starting of some activities or changing the way of contract fulfilment;
  • business rules used – various formally-defined logic;
  • audit trails – collection of records in accordance with good business practices;
  • agreed performance indicators – how well all the business parties are fulfilling their commitments. 

The table below shows primary (P) and secondary (S) relationships between contract concepts (rows) and process concepts (columns).


It seems that a contract can be expressed as an explicit business process with a level of formalism which is understandable by all the business parties (thanks to popular visual notations for coordination and business decisions).

3 Digital contract as a computer-executable process


By definition, a contract is a description of expected behaviour of all the business parties. With a classic contract, each business party is accountable for its behaviour and no one has a full and objective overview. In some cases, to mitigate this risk or to speed-up the execution of a particular contract, a lawyer or a notary may be assigned to conduct the execution of activities of a particular contract.

Having digital contract defined formally as an explicit process, there is no need for extra human to conduct the execution of a contract. Such a “conductor” can be a robot (i.e. a computer, actually some software tools, known as BPM-suites) – let us call it “digital” lawyer (or arbitr).



Actually, not a robot or computer, but some software tools, known as BPM-suites.

Thus, all the business parties may agree that a particular “digital” lawyer can be the conductor for a particular contract. Another option is that each business entity assign their own “digital” lawyer and those “digital” lawyers have to reach agreements about everything that happened with a particular contract.

In addition to “digital” lawyer, a “digital” notary may be required to record everything that happened (i.e. audit trails) in a private storage  ( see privacy concerns at http://www.americanbanker.com/news/bank-technology/banks-privacy-concerns-shaping-blockchain-vendors-strategies-1090411-1.html ) and deposit related digital signatures (or hashes) into a public storage implemented with the blockchain technology.

Also, a “digital” lawyer may act as a “digital” escrow if accepted by all the business entities.

It must be clear that any complex contract (therefore any contract as a process) does not cover all the exceptional situations which are usually addressed by a court.

An interesting “by-product” of contract-as-an-explicit-and-computer-executable process is the ability to simulate (or imitate) the execution of a digital contract under the various scenarios (e.g. WHAT-IF scenarios). Thus the business entities can be fully informed about any potentials traps in a contract they are supposed to approve.

From the BPM point of view, digital contracts coordinates two or more cross-business-entities co-processes. Without a digital contract those processes (see the left part of the illustration below) have to coordinate the joint work by themselves. While adding a digital contract (see the right part of the illustration below) makes the coordination explicit and easier to implement (that is important for small contracts).

  


4 Digital contract lifecycle


There are several phases in the contract lifecycle:
  • three sequential phases: preparation, execution and closure; 
  • one overarching phase: monitoring, and 
  • one discretionary phase: exception handling. 
Contract preparation is a typical document-centric process for drafting, validation and approval of a particular contract. Note that a digital contract may be composed (or assembled) from several parts, for example:
  • the business entity’s general requirements for contracts;
  • contract-specific requirements, e.g. consulting vs goods, local vs international, and
  • contract-specific information (including the identity of contractors).
Obviously, some of those parts can be approved once and re-used many times. Additionally, the whole contract and some of its parts may be simulated with some scenarios to estimate associated traps, risks, benefits and losses.

Contract execution is a formal enacting of a particular digital contract-as-a-process by some software. This software acts as a “digital” lawyer and it is responsible for coordinating activities which are assigned to related business entities. For example, a buyer must pay to a vendor an agreed amount of money and then this vendor must deliver some goods to this buyer. Some activities may be assigned to the “digital” lawyer as well.

Also, the “digital” lawyer may act as a “digital” notary to keep the records of everything that happened with a particular contract.

Contract closure is an additional activity to ensure that all the business entities agree with the execution of a particular contract.

Contract monitoring is a permanent activity for controlling that a particular contract is executing correctly.

Contract exception handling is a set of activities to be initiated if something went wrong with a particular contract. Such situations may involve a “digital” judge.

5 An example of doing business fully digitally


In this example, a buyer wants to purchase some digital assets (a personal copy of a digital book or a car renting certificate) from a vendor. Both of them can use only digital money. Because the buyer and the vendor don’t trust each other, they engage an independent “digital” lawyer to conduct this purchase via a digital contract. This “digital” lawyer explicitly acts as an escrow and a notary as well.

The desired sequence (also known as happy path) of activities is the following.

Activities during the contract preparation phase:
1. Vendor and Buyer: Engage a “digital” lawyer via a standard digital contract
2. Vendor: Propose contract
3. Lawyer: Validate contract
4. Buyer: Accept contract
5. Escrow: Seal contract

Activities during the contract execution phase:
6. Buyer: Transfer money to escrow
7. Lawyer: Announce payment to vendor
8. Vendor: Deliver goods
9. Buyer: Announce acceptance of goods
10.Lawyer: Transfer money to vendor

Activities during the contract closure phase:
11.Lawyer: Close the contract

The same sequence with more details.

1. Vendor and Buyer: Engage a “digital” lawyer via a standard digital contract
  • They engage a certified “digital” lawyer via a standard digital contract (which is digitally signed by some legal authority).
  • The standard digital contract and identities of three business entities forms an engagement contract.
  • This engagement contract is digitally signed by three business entities (with their private keys).
  • A digital hash of the signed engagement contract is saved as a record (the lawyer acts as a notary). 
2. Vendor: Propose contract
  • The contract is digitally signed by the vendor (with his/her private key).
  • A digital hash of the signed contract is saved as a record.
3. Lawyer: Validate contract
  • The contract is validated by the “digital” lawyer.
  • The validation report is digitally signed by the “digital” lawyer (with its private key).
  • The contract signed by the “digital” lawyer (with its private key).
  • Digital hashes of the signed contract and validation results are saved as records.
4. Buyer: Accept contract
  • The validation results are analysed.
  • The contract is digitally signed by the buyer (with his/her private key).
  • A digital hash of the signed contract is saved as a record.
5. Lawyer: Seal contract
  • The contract (which is already signed by buyer and vendor) is signed by the “digital” lawyer.
  • A digital hash of the signed contract is saved as a record.
6. Buyer: Transfer digital money to the “digital” lawyer
  • The agreed amount of digital money is transferred by the buyer to the “digital” lawyer.
  • This transaction is saved as a record.
7. Lawyer: Validate the payment
  • The “digital” lawyer (acting as a “digital” escrow) validates that the right amount of money has been transferred. 
8. Vendor: Deliver digital goods
  • The vendor change the ownership of the digital assets to be sold in accordance with the contract.
  • This transaction is saved as a record.
9. Buyer: Announce acceptance of digital goods
  • The buyer confirms the reception of the digital goods.
  • This transaction is saved as a record.
10. Lawyer: Transfer digital money to vendor
  • The “digital” lawyer (acting as a “digital” escrow) transfers the digital money to the vendor.
  • This transaction is saved as a record.
11. Lawyer: Close the contract
  • The history of the contract is prepared and shared with the buyer and the vendor.
  • This history is saved as a record.
Again, this is only a happy path without any problematic situations. Important that all the procedural formalities can be implemented exactly in accordance with the legislation of a particular country. Because of the detailed audit-trail which is saved as records, any disputes about a particular contract may be resolved by a “digital” judge.

6 Conclusion


Both characteristic, “explicit” and “computer-executable” of digital contracts are equally important. For example, smart-contracts are computer-executable but they not explicit because they are pieces of code which is not understandable by some business parties. Also, some traditional laws contain typical process diagrams (see http://improving-bpm-systems.blogspot.ch/2014/07/practical-process-patterns-laap.html ) to become more explicit, but they are not computer-executable.

Thanks,
AS








2016-07-02

Enterprise Architecture (#EntArch) as a #SystemsApproach applied management discipline

The goal of this presentation is to show how the use of the systems approach to address typical enterprise challenges
  • an algorithm to generate an enterprise’s blueprint
  • different people in similar situations come to similar solutions and possibly bring innovations
  • an algorithm to build a bigger enterprise from smaller ones