Application Governance with the WSO2 Governance Registry

We are doing a webinar on "Application Governance with the WSO2 Governance Registry" on Thursday, August 23. Here is the abstract that is published on the WSO2 site. Refer [1] for more information.

At any given time enterprises have several development teams working on a range of applications, all at various stages - from development, to quality assurance, to production. Finding a single product, with the capabilities for storing these applications and managing the lifecycle of each artifact across different environments, is a challenge.

The WSO2 Governance Registry is the central repository management product of the WSO2 SOA platform. It is a single product that utilizes its lifecycle management functionality to move configuration artifacts and descriptions (such as Services, WSDLs and WADLs) between development, test, and production environments.

Join Janaka Ranabahu for this webinar as he examines two features in the WSO2 Governance Registry: the powerful asset definition language (RXT) and lifecycle configuration model. Based on standard XML configurations, these features enable users to define any kind of asset or lifecycle model with ease. Additionally, Janaka will demonstrate how to combine these features along with the other functionality in the WSO2 Governance Registry to govern an application across several environments.

During the demonstration, he will

• Introduce an application asset using the RXT configuration.
• Define a new lifecycle to manage these artifacts.
• Show how you can use the lifecycle management functionality in the WSO2 Governance Registry to move artifacts across multiple environments.

[1] http://wso2.org/library/webinars/2012/08/enterprise-use-case-webinar-application-governance-wso2-governance-registry

Performance improvements in WSO2 Governance Registry 4.5.0

WSO2 Governance Registry 4.5.0 is scheduled to be released this summer. This version of Governance Registry has many new features(some are carbon core features) and lots of improvements as well. In this release we have concentrated more in improving the overall performance of the product. There are few areas that we have focused our attention on and this post focuses more in one of those areas which is performance related to governance artifacts.

Governance Metadata and RXT

"Governance Metadata" and "RXT", these keywords can be considered as the base of governance in WSO2 G-Reg. Lets look at what these keywords describe. 

• Governance Metadata - We have used the term "Governance Metadata" to describe basic governance artifacts like "Services","WSDL","Schema" and "Policy". We have used this term because we store metadata related to web services. Metadata of a web service is what a client needs to know to interact with it. 
• RXT - We use the term RXT or Registry Extension Type to describe configurable governance artifacts. A configurable governance artifact is a new feature that we introduced in WSO2 G-Reg 4.1.1 and with that, users have the capability of defining their own asset model in G-Reg. Configurable governance artifacts are defined using a XML configuration language and those can be added from the management console.

All of these artifact types are called governance artifacts. WSO2 G-Reg exposes an API called the Governance API which can be used to handle these governance artifacts using external code. We have done a significant improvement in performance when handling these governance artifacts using the Governance API.

Artifact ID

The people who have worked with the WSO2 G-Reg's Governance API will know that, each governance artifact has a unique ID called the Artifact ID. The artifact ID is there to identify a governance artifact even after its unique properties get changed. 

Before G-Reg 4.5.0, we used a resource to keep track of all these artifact IDs. We had a resource called the artifact index where we stored all the artifact IDs and the corresponding paths as properties of that resource. This was a bottleneck when it came to highly concurrent scenarios.

In G-Reg 4.5.0, we have identified that the artifact ID should be something unique to all the resources. Therefore we have introduced a new column in the registry database (REG_RESOURCE.REG_UUID) to track these IDs. Now each resource in G-Reg 4.5.0 has a UUID and this gave us the ability to remove the usage of artifact index resource. We have noticed a significant improvement of performance when handling governance artifacts in G-Reg 4.5.0. 

So my final conclusion is that G-Reg 4.5.0 has far better performance than its predecessors when it comes to handling governance artifacts. 

Default transition UI - Part 02

WSO2 Governance Registry lifecycle model has many extension points. One of them is called the transition UI. In one of my previous posts, I have given an insight to what transition UIs are. This post is an continuation of my previous post.

WSO2 G-Reg distribution includes a single transition UI. This transition UI is used by the default service lifecycle for its service version functionality(more information can be found here). This transition UI does the following.

The name of the default transition UI is "pre_invoke_ajaxprocessor.jsp".

The main functionality of this UI is to take user input for the versions of services and their dependent resources, so that they can be passed to the service version executor.

It is displayed only for services. It will ignore all other media types.

It will list down all the dependencies of a service resource. These dependencies are gathered recursively. Hence all the dependencies of that service and its dependent resources, will be displayed in the UI.

Parameters can be passed to these UIs. The lifecycle configuration model supports adding parameters to a transition UI element and these parameters are added as URL parameters. Please note the section after the "?" in the following example.

The "pre_invoke_ajaxprocessor.jsp" accepts 5 different parameters and all of these are optional.

1. preserveOriginal - Setting this parameter will allow users to instruct the server to keep the old resource or whether to delete it. If this parameter is set to "false", then the underlying implementation will delete the resource after a state transition. This parameter is set to "true" by default.
2. viewVersion - This parameter is there to instruct the jsp page to avoid displaying the content of the page. If the viewVersions is set to "false", the underlying implementation of the jsp page will do an invokeAspct() call including parameters like "preserveOriginal". We use this parameter to pass the value of the "preserveOriginal" parameter to the server without displaying any additional UI to the user.
3. displayMediaType - This parameter is used to override the default media type which is checked by the jsp page. By default, this UI checks whether the resource has a media type equal to "application/vnd.wso2-service+xml"(which is the default service media type of WSO2 G-Reg). By setting this parameter, users can override the above mentioned value. 
4. showDependencies - Instructs the jsp page to not to show the dependent resources of the service. If this parameter is set to "false", the jsp file will only show an input field for the service resource in the custom UI(it will not give input fields for any dependent resources).
5. currentEnvironment - Defines what is the current environment of the service(or its dependencies). The value of this parameter should be a path in the registry. If this parameter is set, the jsp page will not show input fields for resources that are not in the given registry path. 

Default transition UI - Part 01

In one of my previous posts, I discussed about few improvements that we did in the lifecycle configuration of WSO2 Governance Registry 4.1.1. In that post I explained about one of those points, which is the executor interface. Today lets have some insight into the other improvement we did, which is the transition UI.

What is a transition UI?

We use the term "transition UI" to describe a portlet that is displayed as an result of an lifecycle action. In other words, transition UIs are used to get user input at runtime, as a part of a lifecycle transition. To get some clear idea about it, lets look at the following use case. 

Eg:-

Say that I'm a user who is going to govern all my services in WSO2 G-Reg using its lifecycle management capabilities. As a part of the governance process, I need to keep my services in different locations of G-Reg based on the lifecycle state of a service. As a part of this operation I also need to add a version to my services.

Adding a version to a service that is been moved to different environment is somewhat a tricky operation. The version that needs to be added, should be given by a user at runtime. For this, we need some kind of a mechanism to give user inputs for a lifecycle state transition.

We introduced the transition UI feature to achieve the above, which is to give user input for a lifecycle state transition. There are few key things that a user should know about transition UIs.

Transition UIs are registered for an action/event in a particular state. The registration happens through the lifecycle configuration.

If the lifecycle configuration demands the same transition UI for multiple actions/events, then it has to be re-registered.

Only one transition UI can be given for a action/event in a particular state.

Parameters can be passed to transition UI. These parameters are defined in the lifecycle configuration and they are considered as static parameters. Hence they can not be changed at runtime.

The input values that are given through the transition UI are passed to back-end and they can be read using an executor.

Transition UI appears below the lifecycle portlet and there are no popup windows.

There are few transition UIs that we ship by default. I'll be covering about them in another post.

WSO2 Governance Registry 4.5.0 - Lifecycles

At the time of this post, we are preparing for the release of WSO2 Governance Registry 4.5.0. This new version has lots of new features and performance improvements. From this post, I would like to mention one such improvement that we did for G-Reg Lifecycle management feature.

Lifecycles in a nutshell

WSO2 G-Reg has a powerful lifecycle model with lots of flexibility. In WSO2 G-Reg, users have the ability to define their own lifecycle model using a simple XML configuration. The default lifecycle configuration of G-Reg has the ability to define states,check items,permissions,transition UIs, pre and post transition actions. A detailed description about the lifecycle model of WSO2 G-Reg can be found in this post.


In G-Reg 4.5.0 we have introduced the ability to audit lifecycle operations. This means we capture all the lifecycle operations of a particular resource. Following are some key points when it comes to lifecycle auditing.

Lifecycle auditing is turned on by default.

Auditing can be turned off from the lifecycle configuration. This can be done by setting audit=false in the lifecycle element in the lifecycle configuration.

Lifecycle auditing happens per lifecycle. Based on the configuration, all the resources associated with that lifecycle are audited.

Audit logs are kept as a resource in G-Reg. These resources are stored under, "/_system/governance/repository/components/org.wso2.carbon.governance/lifecycles/history" collection. A resource is created for every resource/collection that is associated with a lifecycle. The name of this resource is calculated by replacing all the "/" with "_" of its full path.

Audit logs are kept as a XML file. A sample audit log is as follows.

Lifecycles in WSO2 Governance Registry 4.1.1

Today I thought of taking you all through the lifecycle improvements that we did for WSO2 Governance Registry 4.1.1. We introduced a new lifecycle model with the WSO2 Governance Registry 4.0.0 release. This  model had many improvements when compared with the older versions(like G-Reg 3.6.0 and earlier versions) and had lot of flexibility. A detailed description about this model can be found here. This post focuses more towards the improvements that we did from G-Reg 4.0.0 to G-Reg 4.1.1.

The main improvement we did was to add couple of extension points to the lifecycle model in G-Reg 4.0.0. The lifecycle model in G-Reg 4.0.0 had extension points like "validations" and "permissions" which could be used to customize the behavior of the lifecycle runtime execution. Extension points like "validations" are a good method to run some custom code at a check item click or prior to a transition but our lifecycle model lacked the ability to attach an extension after a transition. So, for that we have introduced the following extension points in G-Reg 4.1.1 release.

1. Executors
2. Transition UI

With these new extension points, our lifecycle model has the ability to handle various kinds of validations and executions. In brief, our new lifecycle model allows users to plugin their custom code before and after a state transition. A detailed post about "Executors" can be found in here. I'll be covering the about the  "transition UI" improvements in another post. 

For more details about WSO2 Governance Registry 4.1.1, please go through our documentation which can be found here. Also you can find more details about the lifecycle model in WSO2 Governance Registry using this link.

Default Executors in G-Reg

This post applies to G-Reg 4.1.1 and upwards.

As you all know, WSO2 Governance Registry supports adding extensions to the execution logic of a lifecycle.This blog post covers about one particular extension point which is called an executor. This post discuss about the default executors that we have included in the Governance Registry distribution. To get more information about executors, please refer this.

We have 3 default executors in G-Reg. From this post, I'm planning to give an overview about these executors.

1. Service version executor
2. Demote action executor
3. Copy executor

Service Version Executor

The main functionality of this executor is to version services and dependent resources of services(wsdl,schema and policy) when a lifecycle promote action happens. This executor was introduced  to G-Reg with the service version functionality implementation. To get an idea about this executor, we need to have some idea about the service version functionality of G-Reg.

Service Versioning

In G-Reg, we have introduced the ability version services and its dependent resources as an part of lifecycle management of the service. With this improvement, a service in G-Reg are kept in 3 environments when it is governed using the default service lifecycle.

1. Trunk
2. Branches/testing
3. Branches/production

Each of these environments are correspond to a lifecycle state in the default service lifecycle. The default service lifecycle in G-Reg has 3 states.

1. Development
2. Testing
3. Production

The service version functionality happens in the following way.

1. At creation, the service resource(and its dependent resources) will be placed in "trunk" environment. The path structure of the service resource is like - "trunk/services/{namespace}/{name}". We limit the creation of 2 services with the same name and namespace in this environment. 
2. All the services in "trunk" environment are on "development" state of the lifecycle.
3. When a "promote" action happens, a UI is displayed to the user to enter the new versions of the service and its dependent resources(This is done using the "transition UI" functionality of G-Reg which I will be covering in another blog post).
4. After completing the above action, the service and its dependent resources are moved to "branches/testing"environment. The path structure of a service resource in this environment is like - "branches/testing/{namespace}/{version}/{name}". In this environment, we create a collection with the version that the user has given and because of that, we allow users to keep multiple versions of the same service in this environment.
5. With the above step, we do some additional processing and update the dependencies with the new paths and also update the version filed in the service to reflect the new version. 
6. Promotion from "Testing" state to "Production" state is similar to step 3,4 and 5. All the services and the its dependencies are moved from "branches/testing" to "branches/production" environment. 

The service version executor is the one who does steps 4,5 and 6 when a promote transition happens. It is written in a way such that a user can change the environments from the lifecycle configuration. To get some idea about this, we need to look at the configuration of the service version executor.

Here you will notice several "parameter" elements. We focus our attention on 2 of these which are,

1.  currentEnvironment
2.  targetEnvironment 

The target environment parameter is the one who decides what is the next environment that the service is placed on. In other words, these 2 parameters say that, take the resource out from "/_system/governance/trunk/{@resourcePath}/{@resourceName}" and place it in "/_system/governance/branches/testing/{@resourcePath}/{@version}/{@resourceName}" path. By changing this configuration, an user has the ability to change the environments based on their requirement.

Also one important fact to note about this executor is that it is media type aware and it works only for services. 

Demote Action Executor

The demote action executor in G-Reg 4.1.1 does not do any copying of the resources to the previous environments. We have identified that the demote action requirement varies from one user to another. The ideal demote action would be to redirect the user to the resource in the previous environment but that is again challenging because there can be multiple versions of the same service in the previous environment. 

Therefore, we have introduced a demote action executor to avoid changing the lifecycle state when the demote action is performed for service in a particular state. In other words, this executor does not do anything apart from keeping the same lifecycle state when the demote action is preformed. 

Copy Executor

The idea behind the copy executor is to do a basic copy of resource from the given location to another. This executor has the ability to work with any media type and it is not media type aware. This executor does a basic copy operation(registry API copy operation) of that resource and it does modify the content or make associations like the service version executor. 

The configuration element of the copy executor is somewhat similar to the service version executor. 

Here you will find 2 parameters called the "currentEnvironment" and "targetEnvironment". These parameters work in the following way. The copy executor will copy any resource that is inside "currentEnvironment"  to the corresponding path in the "targetEnvironment". 

Eg: - Say that you have a resource in the following path.

    "/_system/governance/e1/foo/bar/xyz".

When the user applies a lifecycle (that is configured with the copy executor with the above configuration parameters) to resource "xyz" and do a promote operation, the copy executor will copy that resource to the following path.
"/_system/governance/e2/foo/bar/xyz"