This article discusses three separate methods to contact, request information, and parse the data returned by a RESTful web services. There are several source code highlights within this document and the complete ATEasy project is attached at the end of the article. The example project consists of three tasks demonstrating various methods of access.
The first task in the example retrieves data from demo services in both JSON and XML formats. The second task in the example implements the OAuth protocol to contact a commercial website. And the final task in the example specifies a method to access OSLC Requirements Management V2 Services provided by the Rational DOORS Web service from ATEasy.
The code examples discussed are located within the example program (see link below). |
Making an HTTP Request
In the first example, we will use the .Net WebRequest class that is included in the System.Net namespace. This class allows ATEasy to programmatically access web services similar to the way a web browser accesses them and is the basis for the more advanced service requests found later in this article. The simplest, default HTTP method is GET where the client application requests that a server provide the data at a specified URL. In the example below, an request which acts as a client is created using the Create() method and and then immediately sent with the GetResponse() method.
request : System.WebRequest
ws : System.WebResponse
! Send an HTTP GET request to the JSON test server
request=WebRequest.Create("https://jsonplaceholder.typicode.com/todos/1")
ws = request.GetResponse()
By default, GetResponse() is synchronous and will report certain failures such as a timeout; in case of a timeout, the requested URL doesn't respond and there is no meaningful data in the ResponseStream. If the request was successful, the response from the server is recorded in the WebResponse object as well as in the ResponseStream property of the WebRequest object. The next step is to read the response data. The exact method would depend of the service requested and the formatting of the content. In this example, the server will respond with JSON-formatted text. HTML and XML are also covered in this article. This example reads the JSON and appends it to the log after the test has completed.
!Using StreamReader to read entire chunk
streamReader = new mscorlib.StreamReader(ws.GetResponseStream())
! Read line by line and print to log
repeat
sLine=streamReader.ReadLine()
Append sLine
until sLine=""
NOTE: If HTML is printed to the ATEasy test log, the text may be interpreted as HTML. This could have unexpected consequences such as causing some data to appear missing or may change the format of the test log. Avoid this by printing the HTML contents to a file for review, or by changing the Log's PlainText property to True, or replacing HTML tags as demonstrated in Test 1.4 of the included example.
Anything beyond a simple GET request requires the user to modify the WebRequest client object before sending it off. Once the object is created, all of the properties of the request are assigned to their default values and can be changed if they are read-write properties. Test 1.3 of the example demonstrates changing the request to a POST. After Create(), the ContentType is specified as JSON, the Method is changed to POST to update the items specified in the body, and the Timeout is extended. Immediately after this, the body of the request is populated with the values on the web server I am try to change This is done by write my JSON values to the RequestStream of my WebRequest. It is not demonstrated but a header can be added in a similar manner.
! Modify the header/body and make a POST request
request=WebRequest.Create("https://jsonplaceholder.typicode.com/posts")
request.ContentType="application/json; charset=UTF-8"
request.Method="POST"
request.Timeout=20000
try
streamWriter=new mscorlib.StreamWriter(request.GetRequestStream())
sLine="{\"title\": \"Marvin\", \"body\": \"Test\", \"userId\": 1}"
streamWriter.Write(sLine)
streamWriter.Flush()
catch
endtry
After this, GetResponse() can be used to send the request to the server. This code shown above is the equivalent of the following HTTP request. You can try this out by putting the following code into the DevTools console of your favorite browser.
fetch('https://jsonplaceholder.typicode.com/posts', {
method: 'POST',
body: JSON.stringify({
title: 'Marvin',
body: 'Test',
userId: 1,
}),
headers: {
'Content-type': 'application/json; charset=UTF-8',
},
})
.then((response) => response.json())
.then((json) => console.log(json));
Promise {<pending>}
VM288:13 {title: 'Marvin', body: 'Test', userId: 1, id: 101}
These are the basics of HttpClient web requests and responses. The example includes additional examples of reading HTML and XML as these have generic data parsing tools available.
The Root services document
The root services document contains information about your organization's specific DOORS instance. ATEasy uses the information contained within this document to negotiate authentication to access the protected requirements as well as primary service catalog which all sub-catalogs and services are contained within.
The first Test in the "Accessing Rational DOORS" task, "Access RTN DOORS from Sample File", reads through a root services document that has been saved locally as an XML file. Because this root services document is not located on a web server, no special authentication is required to access and read it. We use a System.Xml.XmlReader to parse through the document.
xmlReader = XmlReader.Create("public_rootservices_copy.xml", settings, context)
Once the file is open, the example parses it node to node from beginning to end using a while loop. This example looks specifically with a node named oslc:ServiceProviderCatalog.
while (xmlReader.Read())
select (xmlReader.Name)
case "oslc:ServiceProviderCatalog"
if xmlReader.HasAttributes
sCatalogLocation=xmlReader.GetAttribute(iAttributes)
endif
endselect
endwhile
This test proves the ability to parse through a root services document and is built upon in subsequent tests.
ServiceProviderCatalog and ServiceProvider
In the previous section, we searched a root services document for a node named olsc:ServiceProviderCatalog. This node is the entry point for discovering services within the DOORS webservice. The olsc:ServiceProviderCatalog node in the root services document has an attribute which specifies the URL to the primary service provider catalog. This primary service provider catalog and all other catalogs contains a listing of Service Providers and/or child Service Provider Catalogs.
See also: IBM, Access OSLC services from IBM Rational DOORS
OAuth Credentials and Tokens
To access any services beyond the root services document, transaction requests must be authenticated with OAuth, an open authorization protocol (see also: IETF, The OAuth 1.0 Protocol). In addition to the location of the primary service provider catalog, the root services document contains information required for access with OAuth.
In the second Test of the example, Access RTN DOORS from RootService w/ OAuth, we retrieve the OAuth URLs from the root services document, provide our credentials and retrieve an access identifier. This exercise requires a functional DOORS Web service which has been configured for web / automated access. Your DOORS administrator will need to provide you with a consumer key and consumer secret. The second Test has a header section which the end user must fill with their own implementation-specific information:
!============================================================================
!This data is generic, configure the server and port before running this demo
sDwaServer="doors9501.com"
lPort=8443
sConsumerKey=""
sConsumerSecret=""
sRootservices="http://"+sDwaServer+":"+Str(lPort)+"/dwa/public/rootservices"
!============================================================================
- sDwaServer is the URL of your DOORS web service
- lPort is the port of the DOORS web service
- sConsumerKey is provided by your DOORS administrator
- sConsumerSecret is provided by your DOORS administrator
- sRootservices is the address of the root services document. I assumed Rational DOORS Web Access version 1.4.0.2 or later.
With the correct information, the Test will retrieve the jfs:oauthRequestTokenUrl, jfs:oauthUserAuthorizationUrl, jfs:oauthAccessTokenUrl and use these URLs along with the user-provided information to load a log-in screen (via a pop-up Internet browser). After the user has put is his or her DOORS credentials, the Internet browser will respond with an identifier: a six digit query string value. This identifier is the access token required for access to documents beyond the root services document.
Depending on your organization's needs, the log-in process described can be optimized. For instance, instead of an external browser an ATEasy form can launch the log-in screen and automatically record and store the identifier.
An example: Crawling through your Database
All access attempts beyond the root services document must make use of the OAuth access token (identifier). The final Test in the example demonstrates an algorithm for crawling through the DOORS web service. The purpose of the crawler is to record the URL of all service provider catalogs and service providers within the database.
ATEasy keeps track of a list of to-be-searched URLs and already-searched URLS. Each URL in the to-be-searched list is accessed and parsed for service provider catalogs and service providers. The URLs of these provider entities are added to the to-be-searched list as long as they do not already exist in either of the lists. As URLs are searched, they are moved to the already-searched list.
Initially, the root services documents is the only URL in the to-be-searched list.
When the Test completes, the user can examine the already-searched list to see all of the service provider catalogs and service provider URLs. While this Test is not particularly functional in itself, it demonstrates how a user may programmatically search for and locate a specific requirement with no previous knowledge of the database architecture.
ATEasy Example Code