How to use Ready! API to test APIs published in WSO2 API Manager

In this post I'm going to explain on how we can use the Ready! API(SmartBear Software) to test API's that are published in the WSO2 API Manager.

WSO2 API Manager

WSO2 API Manager is a 100% opensource solution for designing and publishing APIs. It also has the capability to create and manage a developer community. The core of the WSO2 API Manager is created using the WSO2 Enterprise Service Bus, Identity Server and Governance Registry components.

Ready! API

Ready! API is a product that is developed by SmartBear software which includes a platform of tools such as SoapUI NG Pro, LoadUI NG Pro, Secure Pro and ServiceV Pro that can be used to effectivly test API functinality. A more detailed description about these products can be found using their product page.


This post focus more on the Ready API Plugin usage and not WSO2 API Manager product functionalities. If you are new to WSO2 API Manager, I suggest you to go through our product page and documentation to get an idea about what WSO2 API Manager is and how to use it.


Ready! API has an extension point where people can create a plugin and alter the behavior of the product. WSO2 Team has created a plugin(named WSO2 API Manager Plugin)  for Ready API which helps the users to Integrate and Test their APIs that are visible in the WSO2 API Store. This blog post focuses on describing the features and advantages of the plugin.

WSO2 API Manager Plugin

Installation

The WSO2 API Manager Plugin is published in the Ready API Plugin repository. Hence installing the plugin can be done using following a simple set of steps. 

1. Navigate to the Plugin Manager.
2. Click on  "Browse Plugin Repository".
3. Select the "WSO2 API Manager Integration" Plugin and click Install.

The following screenshot shows the above mentioned steps.

Usage

Once the plugin is installed, there are multiple ways to get the advantage of the plugin.

1. Create a project by importing a new API.

2. Add a new API to an already existing project.

When using both of the above mentioned options, you would see a prompt to enter the API Store connectivity details. What we need to do is to fill in the details and press the "OK" button. Note that the tenant domain field is not required if you are connecting to the super tenant store as the plugin will connect to the super tenant store if the field is not specified. 

After the above step, the plugin will connect to the API Store(give by the user) and list out all the APIs in its API list prompt.

The user only has to select the API(can select multiple APIs) and whether to create a test suit and a load test for the API. The plugin will connect to the API Manager instance using the REST APIs of the API Publisher and Store and will create a project by reading the swagger definition of the API(or APIs). The final outcome would be something similar to the following.

As you would notice, the plugin reads the Swagger definition and create a project with the sample requests. The plugin also creates an OAuth2 authorization profile with the name "wso2-api-manager-default" so that the end users would only need to give the OAuth2 token that they generated from the API Manager Store.

Once the API import is complete, the usage of the plugin is really simple. The user only needs to give the OAuth2 token to the "Access Token" field and fill in the necessary request parameters before invoking the sample request.

The response would look like what is stated in the image below.

As shown in the above steps, you could import any number of APIs from the API Store and test them using the Ready! API. Similar to the project created in SoapUI NG Pro, there would be projects created on LoadUI if the user has selected that option when importing the APIs.

Summary

The WSO2 API Manager plugin in the SmartBear plugin repository was developed to have a seamless experience when testing the APIs published in the WSO2 API Manager using Ready! API Platform. In summary the current plugin supports the listing of APIs in the given Store, creating sample requests for the selected APIs by reading their Swagger definitions and create test suits and load tests if that option was selected and many more features.

For more information about the plugin usage, please refer to this article and the Github project for the plugin.

What Ready! API integration brings to the WSO2 API Manager Platform

WSO2 API Manager is a complete free and opensource solution that has a rich set of features for API Creation, Publication and the complete API Lifecycle management with features to secure the API invocation. All these features come from the WSO2 platform and from the core features used in the products such as WSO2 Enterprise Service Bus, WSO2 Identity Server and WSO2 Governance Registry. In addition to that, the API Management platform utilizes the power of the WSO2 Data Analytics server for the API statistics analytics.

The above features of API Manager focuses more towards the development and run time aspects of the complete API development lifecycle. Apart from the above main states of a API development lifecycle, the other most important state is the Testing state. 

From this post, I will talk about the WSO2 API Manager and its integration with the Ready! API(by SmartBear) and what capabilities it brings to the WSO2 API Manager platform.

I'm assuming that everyone who reads this post would have some level of understanding about WSO2 API Manager and how it operates. I'm not going to explain the basics of WSO2 API Manager from this post. For more information about the API Manager product, please visit the product home page.

Ready! API

Ready! API is a platform of API testing tools by SmartBear which combines the power of SoapUI NG Pro, LoadUI NG Pro, Secure, ServiceV, and API Monitoring in AlertSite into a single bundle to give its users the power of functional testing, performance testing to post-deployment monitoring. For more information about Ready! API, please visit the Ready! API product overview page.

Ready! API Integration

Lets look at what Ready! API brings to the WSO2 Platform.

API testing is one of the key aspects of the API Development lifecycle. Any API Management solution would have integrated tools for API testing but most would lack the ability to conduct a complete API testing that a commercial tool would give. Most of the embedded tools would only have the capability of doing basic testing to get an idea about the request/response as well as to try out the different resources exposed from the API.

With the above points in play, it is important that the commercial API testing tools in the market could integrate with your solution to give the users the ability to carryout testing at will. We have done the same by making our API Management solution integrate with Ready! API, to give the ultimate API testing experience.

WSO2 Ready API Plugin

We have developed a plugin for the Ready! API to integrate with WSO2 API Manager and discover the APIs published in the Store. The plugin helps its users to select the APIs that needs to be imported using its inbuilt API Store browser and does all the work from creating the project, creating the sample requests to creating the load tests in LoadUI.  The plugin has been deployed to the SmartBear plugin repository so that Ready! API users would have the best user experience when using the WSO2 Ready API Plugin. For more information, please visit the SmartBear plugin repository and the github project of the WSO2 Ready API Plugin.

Conclusion

The development of the WSO2 Ready API Plugin has helped the WSO2 API Management platform to move to the next level by being able to integrate with a commercial API testing platform. The advantages of the Ready! API platform, including its core components of SoapUI NG Pro, LoadUI NG Pro, Secure, ServiceV, and API Monitoring in AlertSite have strengthened the API development lifecycle of the WSO2 API Management Platform. 

Generate custom reports using WSO2 Governance Registry

WSO2 Governance Registry has many features that helps users to store and manage different types of data including Service metadata, WSDLs, Word Documents, PDF Documents, Text Documents and may more. It also has features that help users to navigate and search for any resource that is stored in the Registry/Repository. Apart from the tree view and metadata search of resources, WSO2 Governance Registry has a feature where users can define custom report generation for any types of artifacts.

In this post, I'm focusing on adding a custom report that helps users to generate reports about services and policies that are stored in the Governance Registry.

Support for Report Generation

WSO2 Governance Registry has the capability of generating reports for metadata and activity search results. With this feature, a user has the ability to export the search results as a PDF, Excel or HTML report. A sample report that was generated using a metadata search result is given below. 

As you can see, this is a basic report that is generated with the search results and there is no easy way of adding more fields and changing the report layout. But the WSO2 Governance Registry has an extension point where you can add your own report template and add your own data set and generate reports using that report template.

In this post, I'm going to add a custom report that lists out all the Services and Policies that are stored in the Governance Registry. The report contains the following metadata about Services and Policies.

• Name
• Version
• Description
• Relationships(Dependencies and Associations)
• Lifecycle state

The generated report is structured in the following manner.

Custom Reports

WSO2 Governance Registry uses the JasperReports library for generating reports. A user can define his own Jasper Report template(extension .jrxml) and a java class that injects data into that report and add it to the Governance Registry.

I have created a custom Jasper Report template and a GovernanceReportGenerator class that extends the org.wso2.carbon.registry.reporting.AbstractReportGenerator class to inject data into that reporting template.

Reporting template

The structure of the reporting template is as follows. I have used the iReport Designer  to design this reporting template. 

The report template is written in a Generic manner so that it will generate different tables for each of the data type. In this sample, I'm only sending data related to Services and Policies. The report template expects data sets which have sub data sets included in them.

GovernanceReportGenerator

The GovernanceReportGenerator class is responsible for fetching the data from the Governance Registry and creating the data collection structure which is expected by the report template. The GovernanceReportGenerator class uses a JRBeanCollectionDataSource as the datasource of the report. The class does two main operations. 

1. Fetch the pre-defined governance artifacts from Governance Registry. This class fetch data related to Services and Policies using the Governance API of the WSO2 Governance Registry.
2. Create the Bean structure for the JRBeanCollectionDataSource. The datasource is structured in the following manner. For all the JRBeanCollectionDataSources We pass a list of corresponding beans. 

The full source of GovernanceReportGenerator.java is as follows. All of these classes have been defined as inner classes of GovernanceReportGenerator.java. 

GovernanceReportGenerator.java

How to Add The Custom Report

We need to do few things in order to add this report generation to WSO2 Governance Registry. If you have not used WSO2 Governance Registry before, download the distribution using this page, unpack the distribution and start the server. More information on how to install and start the server can be found here.

1. First we need to add the compiled .jar file to the server. We can do that from the Extensions menu in the Management Console.
   
   
   
   
   
   Once the extension is added, you can see it in the list view.
   
   
   
   
   
2. Next we need to add the Report Template to the Registry Server. This can be added as a regular resource. Therefore I have added the governance_template.jrxml to the  /_system/governance/customReports/governance_template.jrxml location.
   
   
   
3. The report template expects a logo resource. I have added the logo.png to the same location where I have added the Report Template. I have given system/wso2.anonymous.role read permissions for this resource too.
   
   
   
   
4. Then we can add the report to the Registry Server. We can do that from the Management Console of the server. When adding the report, you need to specify the following.
1. Report Name - A unique name for the report.
2. Template - The resource location of the .jrxml file. In this scenario I have given /_system/governance/customReports/governance_template.jrxml as the report location.
3. Report type - I have specified PDF. This can be changed later if required.
4. Report class - Need to give the full qualified class name of the class we added in the initial step. For this sample I have given org.wso2.carbon.registry.samples.reporting.GovernanceReportGenerator as the full qualified class name. Once we define the class name we can click on the Load Attributes button. That will load the additional attributes that we have defined in our GovernanceReportGenerator.java.
5. Description - The description that needs to be added to the generated report. This is a custom attribute that I have defined in my source. If we leave this as empty, then a default description is added.
6. Report Title - The tile of the generated report. This is also a custom attribute and leaving it empty would set a default title.
7. Report Logo - The  location of the logo that we have added previously. This is also a custom attribute and if left empty, the Report Generator will look for a logo under /_system/governance/repository/images/logo.png location.
   
   
   
5. Once the above steps are completed we see the added report in the reports list.
   
   
   
6. Now, by clicking on the Generate link, we can create a report of services and policies with above mentioned reporting template. You also have the option of scheduling the report generation.

Complete Source

The complete source of this sample can be found in my Github location using this link. You are welcome to improve this sample.

Loading data into WSO2 Governance Registry using an Excel data sheet

Storing data as files is one of the basic features of a Registry/Repository. Some advanced repositories have the capability of loading data into itself by reading various types of files including text, CSV and Excel. This post describes about how we can use an Excel sheet to load data into WSO2 Governance Registry.

The WSO2 Governance Registry has many extension points that can be used to alter its behavior.It also has a set of clients that can be used to access data in the repository without the use of the Management Console. A sample that uses the Remote Registry API can be found using this link.

In this post I'm going to describe how we can use the Governance API of WSO2 Governance Registry to create a set of governance artifacts by reading the content of an Excel file.

The Data Set

I have used the Reception to Year 2 books list which is listed in here, as my data set in this post. The data set contains information about a set of books with the following set of information.

• Author(s)
• Title
• Series
• Publisher
• Published Year
• Annotation

Governance Client

I have written a Remote Governance Client using the Governance API of WSO2 Governance Registry to read the data from the Excel files and add them to the Governance Registry. More information about the Governance API can be found here.

The Governance Client is written to map the data in each row of the Excel sheets into 2 separate Governance artifacts.

Book Publishers

RXT configuration is as follows.

Books

RXT configuration is as follows.

The Governance Client does the following.

1. Read any Excel file in the pre-defined resource location.
2. Read through each file and get the column headers.
3. Use the properties files to map the column names to the Governance Artifact.
4. Create a Governance Artifact for each row and add the row data to the attributes.
5. Add the Governance Artifact.

After executing the Governance client, the book and publisher artifacts are added to the Registry. Sample output from the Management Console is something similar to the following.

The content of a book is like this. We also create an association to the book publisher from a book.

The complete Governance client code can be found at the end of this post. Let me highlight the important code segments. Also one of the important facts to highlight is that this Governance Client code is not a generic code and only covers this scenario. I only did little effort to make some parts of the code generic but with little more effort, this client can be made fully generic.

This code segment is used to read the Excel cell values and store them against the governance artifact field. There is a check to see whether this is the books Governance artifact because we need to add a relationship to the book publishers Governance artifact.

The following code is used to create the Governance artifact and add it to the Registry.

The Complete Code


The final code segment I used can be found in my git repo using this link. It contains the Java source as well as the RXT configurations too.  In order to make it easy for the readers, I have added the Governance client Java code below as well.

If you want to try out the complete sample, then these are the steps that should be followed.

1. Download and extract a wso2greg-4.6.0 pack.
2. Start the server.
3. Login to the Management Console and add the RXT files to the Registry. Read through the docs to if you are not familiar on how to do it.
4. Change the following constants in the RegistryResourceImporter.
• CARBON_HOME - should point to where the wso2greg-4.6.0 pack is unzipped.
• SERVICE_URL - Should be changed if you are not running with the default ports.
• USERNAME - Should be changed if you change the default admin user name.
• PASSWORD - Should be changed if you change the default admin password.
5. Execute the client and the check from the Management Console.

RegistryResourceImporter.java

Insert data into multiple tables using WSO2 Data Services Server - Introduction

The WSO2 Data Services Server (DSS) is an enterprise-grade, lean, 100% open source, cloud-enabled, multi-tenant data service hosting and management platform. From the next series of posts, I'm going to explain several ways that we can use the WSO2 DSS to insert data into multiple tables.

Before starting to go into detail, let us setup a simple environment to demonstrate our scenario. I'm going to use WSO2 Data services Server 3.0.1, which is the latest release at the time of this post. If you haven't downloaded it yet, you can download the product distribution through here. I will be using MySQL as the database during these posts.

In this introduction post, I'll be guiding you step by step through the process of creating a sample database to creating a sample data service that inserts data into two tables.

First, lets create the database.

Creating the sample database

1. Create a sample database.

I have named this database as 'order_sample'.

2. Create the tables. 

I am creating 2 tables in this database. "orders" and "order_items". Im using the following SQL statement to create the database tables.

Creating the sample data service

1. Copy the MySQL connector jar

Before starting the server, we need to copy the MySQL connector jar to $DSS_HOME/repository/components/lib directory.

2. Start the server and login

The DSS server can be started using the wso2server.sh/wso2server.bat file located in $DSS_HOME/bin folder. For more information on configuration of the Data Services Server, please refer to the documentation. 

3. Create the sample data service.

To create a data service we have to navigate to the following location, from the management console.
Home > Manage > Services > Add > Data Service > Create

Note: Please refer the documentation for more detailed description on how to create a data service

I have given the name "SampleInsert" as my data service name.

Next we need to give the data source information. I have given the connection details to the database we created earlier. 

Then Im going to construct my queries that needs to be executed. I have two queries, one to insert data in to the order_items table and another one to insert data to orders table.

Following is a snapshot of the "insert_order_items" query which I use to insert data to the order_item table. We need to pass 2 parameters for this query which are the order id and the item id.

The following image shows the "insert_orders" query which insert data to the order table. Even though this table has 3 columns we only pass values to "creation_time" and "created_by" columns. The "order_id" is an auto generated value from the database. Since we need this auto generated value for the  "insert_order_items" query, we enable "Return Generated Keys" in this query. As a result of this query execution, the auto generated id will be returned to the client.

Next I'm going to define the operations that needs to be exposed from the data service. I expose 2 operations, "addOrder" and "addOrderItem".

Now we have finished creating our data service. I'm going to test this using the "TryIt" functionality of WSO2 DSS.

Following is the request and the response of the "addOrder" operation for the DSS.

Then we can use the above generated id to invoke the "addOrderItem" operation.

We can check the database to see whether our tables got updated correctly.

Conclusion

This post explains the basic method of inserting data into multiple tables using WSO2 DSS. Note that this method is not executed in a single database transaction. From my next post, I'll explain on how to insert data into multiple tables in a transactional manner.

Return JSON response from WSO2 Data Services Server

WSO2 Data Services Server is  an enterprise-grade, lean, 100% open source, cloud-enabled, multi-tenant data service hosting and management platform. This post explains how we can configure the WSO2 DSS to return the response in JSON format.

During this post I will be using WSO2 Data Services Server 3.0.1, which is the latest release at the time of this post. If you haven't got it already, we can go to the Data Services Server homepage and use the download link there or you can use this link to download it directly.

I will not be explaining any of the basics about Data Services Server since all of the information can be found using the documentation.

I will be using the ResourcesSample that is shipped with the DSS distribution to demonstrate this functionality. This sample explains how data can be exposed as a REST resource using WSO2 Data Services Server. What I'm going to explain is how we can get the response of these services as a JSON String.

1. Change the axis2.xml and axis2_client.xml

We need to add the following parameter to the axis2.xml and axis2_client.xml in the DSS server.  These files are found in CARBON_HOME/repository/conf/axis2 directory.

<parameter name="httpContentNegotiation">true</parameter>

First, lets change the axis2.xml. I have added this parameter to the parameters section in the axis2.xml. Note that this parameter is not there by default and we have to add that.

Next, we need to set the value of this parameter in axis2_client.xml to 'true'. Note that this parameter is there in the axis2_client.xml and we only need to change the value.

2. Deploy the samples

Use the instructions given in the samples guide to deploy the samples.

1. Stop the currently running Data Services Server instance if any. This is recommended in order to clean the current sample database and redeploy the services.

2. Remove the current sample database and existing sample data services.
# cd CARBON_HOME/samples
# ant clean

3. Deploy all samples.
# ant

4. Start the Data Services Server
# cd CARBON_HOME/bin
# wso2server.bat|sh

3. Run the sample 

Im going to use CURL to send a request to the DSS server. I will be using the same command that is described in the DSS documentation.

curl -X GET http://localhost:9763/services/samples/ResourcesSample.HTTPEndpoint/product/S10_1678

As as result, I get the following response from the DSS server.

Now let us try to get the same response in JSON format. In order to do that, I'm going change my request and add the following header to the request

"Accept:application/json"

Now my new request looks like the following.

curl -X GET -H "Accept:application/json" http://localhost:9763/services/samples/ResourcesSample.HTTPEndpoint/product/S10_1678 

Now you will observe that the same response is sent as a JSON string.

We can follow the same approach get a JSON response from the other invocations that is described in the DSS sample.