Showing posts with label DXL. Show all posts
Showing posts with label DXL. Show all posts

Wednesday, 23 December 2015

Do not give your manager a DOORS DXL Reference Manual

After reading my last post my manager came back to me with a DXL he wrote in one and a half days.

"Here is my DXL utility which can be used to perform requirements analysis on a module:" (source)


Requirements Analyzer tool UI

His selection was taken from various white-papers on the subject matter - such as that one from NASA - Automated Analysis of Requirement Specifications.

This tool creates a view in the module for each theme. It creates a compound "or" for all the keywords against either Object Text or all attributes.

Additionally only users will "manage database" can edit the themes.

You can see this UI in action in this video:


Well that's pretty cool!

Sure there is place for improvements, and it's not bullet proof, but that's not the point. Imagine what a Requirement Engineer could do in DOORS 9 if given a bit more time...

Maybe something similar to the use case I mentioned in my last post: use Watson Natural Language Classifier Service from Bluemix.

The story!

Once you have your requirements filtered you can have a list of rules your device must implement.
If you continue watching above video you will see how I created a rule from a very quick UI I wrote in DXL (source).

UI for Rule Creation in IoT Real-Time Insights

Code short and simple but shows you how to get schema information so you can:
  • view condition items
  • assign rule to schema 
If rule is successfully created its ID is stored in ruleId string attribute.

Of course, instead of extra attribute with value it could store an external link to:
https://iotrti-prod.mam.ibmserviceengage.com/api/v2/rule/{ruleId}

It would support 'follow' action, but user name and password which are apiKey and apiToken from VCAPS are not easy to remember ;)

As a exercise you can extend this example to enable action selection. In this way you will create a complete rule with action which can trigger alerts.

Traceability is Power

If you are working with DOORS 9 I'm pretty sure you are familiar with concept of Suspect links and changed objects

OOTB DOORS 9 let's you check information about in-links or out-links to open modules or to all modules.

Well that's great but with OSLC links, or even without links (if you select to stay with extra attribute in last chapter)?

With DXL you can easily do it, just retrieve your rule from IoT RTI and compare to what you have. See the last part of the video to see it working.

Suspect Linked Rule

As promised sources for

Conclusion

If you do not like challenges do not give your DXL Reference Manual to your Manager, he might come back with some great ideas!

Friday, 18 December 2015

2015 Blog Summary


It would be too easy to dismiss previous posts on "DOORS 9 and IoT" as disposable fun with no real value.

Yes they are bite size examples - deliberately so when you think about it - but at the high level they set out to prove a number of very serious points.

Those points are:


  1. There are no barriers preventing you from using DOORS 9 with BlueMix and IoT services today.
  2. There is no requirement for complicated, over-engineered solutions or add-ons. Forget that. You already have the most important piece - DOORS 9 itself.
  3. You can jump right in and immediately start to explore and take advantage of these new and exciting technologies.

So we thought it would be helpful if we took a step back and tried things from a different angle.

This time the aim is to walk through an end-to-end scenario. In doing so we hope to demonstrate how best practice RM in DOORS 9 can be used alongside IoT to sustain ongoing efficiency, improvement and success.


Scenario for a Story

Consider a company that makes a device.

Now it isn't particularly important what this device does: it could be large or small, shiny or matte, there may be one of them or there may be thousands, it could sit in your home or it could move around on water, roads or the air.

However one thing for sure is that it is complicated, takes time to build and involves both hardware and software.

Now the company feels very strongly about this device. It is the market leader. It makes them money. Of course they are going to be emotional about it.

The company isn't stupid either. They realise that over time this device will need to change in order to continue market dominance.

Change may come for any number of reasons such as simple aesthetics, available of new materials or increased regulation.

Change may also occur because life has a frustrating habit of throwing up problems.

In other words the company's device is built to exacting standards but it can still fail.

To help deal with this failure the device includes sensing, monitoring and communication capabilities. In real-time there is a continual flow of information back to the company and naturally the company wants to act fast in diagnosing and fixing any problem.

This historical data can also be used when considering a new variant of the device. The company wants to identify small changes that can be made incrementally and hopefully at low cost.

Phew... that took a while to set the scene. Might have been better to use a GIF to present a "Star Wars" crawl, or a Mad Men sales pitch.

So let's gets back to DOORS and IoT.


How DOORS 9 fits into this story?


We may have already mentioned this but the company isn't stupid and this means their requirements are written and managed in DOORS 9, so let's start there...

There has been a lot of work over the years into what makes a good requirement.

For example if you google "nasa requirement quality analysis" you will find excellent whitepapers on natural language processing. See Automated Analysis of Requirement Specifications by Wilson, Rosenberg and Hyatt.

This essentially boils down to scaffolding the requirements process with tools that maximise the conciseness, consistency and overall quality of the statements used to describe what a thing should do.

There are more involved utilities available, but using DXL it is a relatively straight-forward exercise to parse your module looking for keywords and/or symbols. A filter can be applied to focus the attention on those that are ambiguous or likely to cause confusion.

You could even consider using a service such as Watson Natural Language Classifier.

On November 1st I've shown how to use DXL HttpRequest to connect to Clouding, so one shall know how to use it with DXL, no matter layout, attribute or utility.
So what's stopping our fictional company from feeding Watson NLC with their DOORS 9 data using DXL or NodeJS and OSLC with aid of a very first example I presented? With HttpRequest this company could use Watson NLC to classify requirements as according to:

  • imperatives
  • continuances
  • directives
  • weak phrases
  • options


With this said there is no reason why the same utilities cannot be used to identify requirements that are essentially "rules" in their own right. This act alone helps system engineers focus the attention on the rules that will need added to Workbench or Real Time Insights.

In terms of our device this means that it would have a thing dedicated to capturing and reporting a specific behaviour or variable.

When RTI detects that a boundary condition is breached an alert is made.

The rule in RTI that controls this behaviour should be able to be extracted from a good requirement.

XXX must not exceed a maximum of YYY units.
So not only can we use DOORS 9 to increase the confidence in our requirements – but we can then use those requirements as a basis for development of things and the subsequent reporting capabilities.

Did I forget to mention DOORS 9 can easily create a rule (or action, alert, whatever you need and the API lets) in Real Time Insights or IoT Workbench (currently an experimental service)?
Well, you know the drill use your DXL HttpRequest to POST some data to a URL.
URL you found in this blog, in a post not only telling you how it fits into IoT World but also telling you where is the IoT Real Time Insights HTTP(S) API documentation, good luck finding that with any search engine (well apart from link to this post).


A thing in the wild


So the requirements are filtered and rules identified, implemented and introduced via things in the real world. Sooner or later something bad happens.

A message is sent to RTI from the thing.

We can easily report on this information from DOORS 9 - and contextualise it in terms of the requirements that perhaps could be a problem. Requirements are linked through to design, test case and source code - so impact analysis of change can be investigated and more importantly executed.

The Layout DXL columns posted earlier show this particular scenario playing out in DOORS 9.

Closing the lifecycle circle


But remember there was another use case.

The company feels it is time to consider a new variant of the product but they want to ensure maximum benefit from minimum change.

We’ve seen how errors can be handled but what about other intangibles. This is where IoT can really begin to drive benefit from the unstructured data.

What if the device was a car that was intended to get mpg at mph. Accelerate to mph in ss.

What if it never achieved that? The information to hand allows you to cross-check weight of passengers, softness of brakes, engine performance to determine areas of weakness.

Is this climate based – were environment factors to blame? Is the car configured wrongly OOTB for the geographical area?


Conclusions

Hopefully my 2015 posts will have raised your awareness and interest in IoT.

It represents fantastic levels of potential and my aim was show that in a few relatively small steps you can begin to unlock that potential now.

At this moment you have everything to need in order to take advantage of DOORS 9 and the GB/TB of data it sits on top of.

Any further advice I could give is best summed up by IBM Fellow John Cohn when he stresses "The importance of play":




Tuesday, 15 December 2015

DOORS 9 direct IoT support - build-in challenge #1

Monday morning is a good time for reflection and quick chat with coworkers before throwing yourself into work cycle.
In this way I had quick chat session with my manager yesterday. That ended up with challenge#1 (DOORS and IoT) and later on I had another quick chat my friend from DNG team, that ended up with challenge #2 (DOORS 9 itself).

Since challenge #1 is about IoT let's leave details of the other one for another post- but I like the idea so I will definitely post something in that subject.


Challenge #1

It was simply; "DXL triggers and IoT."

Well in my last post I used external tool written in NodeJS to publish DOORS 9 TRS feed to IoTF broker.

You need a full DOORS Web Access stack with some extra settings (as TRS is not enabled by default) to be able to get TRS feed which you will consume with your external application before posting that to IoT Foundation broker.

That's how complex last solution was
DWA Stack used in last post

There are no events in TRS so the last example was querying for changes every 10 seconds. That's far from real-time in IoT terms.


DOORS 9 Triggers

Have a look at DXL Reference Manual if you are not familiar with concept of triggers in DOORS 9.

Triggers are as close to real-time events as it can be in RM tool. Yeah but DOOR 9 doesn't have MQTT client.

Does it have to have it?

Well NO!

Looking at Internet of Things documentation, you can find topic on HTTP(S) Connectivity for Devices. If you read my previous post you should be familiar with HttpRequest. So your trigger DXL have to do is send a HTTP POST request to
<target server: org_id.internetofthings.ibmcloud.com>/api/v0002/device/types/{DeviceType}/devices/{DeviceID}/events/{eventID}

Looking at the URL format you can guess some event values will be set automatically:
{
"device_type": {DeviceType},
"device_id": {DeviceID},
"evt_type": {eventID},
"timestamp":
{
"$date": 1450194956425
}
All you need to add is a evt itself.

So your entire trigger DXL will look like

Module m = current Module
Buffer msg = create
msg = ""
msg += "\"id\": \"" getResourceURL m"\", "
msg += "\"content\": \"" name m "\", "
msg += "\"group\": \"Open\""
msg += "}"

string ioturl = "https://org_id.internetofthings.ibmcloud.com/api/v0002/device/types/DeviceType/devices/DeviceID/events/trigger_update"

HttpHeader h = create
string auth = ""
toBase64_("use-token-auth:YOUR_TOKEN", auth)
auth = auth[0:length(auth) -2]
add(h, "Authorization", "Basic "auth)
add(h, "Content-Type", "application/json")

HttpBody b = create
setValue(b, msg)

HttpResponse resp = HttpRequest(HttpPost, ioturl, b, h)
delete b

if (!null resp && resp.isOk)
{
// no one really needs to see it
  print "got it " (!null resp ? resp.code "" : "null") "\n"
  delete resp
}

delete h
delete msg


That's all you need to do publish events in real-time to IoTF!

Conclusion

Again HTTP Request proved to be a very powerful and useful perm. Using it the initial stack was reduced to very simple form:
Simplified stack
Final stack in DOORS to IoTF communication

DOORS 9 has a potential in it, you just need to know how to use it.

Wednesday, 2 December 2015

DOORS 9 IoT Report

Let's think about following user story:
"As a Requirement Engineer I would like to know how many errors my devices report"

In DOORS 9 module one can visualize this as:
Example report in DOORS9

So we would like to get a number of errors of given type reported by each of devices.

Device Error Reports in Sample App

First I need a device which can send some error codes. The IoT Starter Application mentioned in one of previous post publishes three kind of messages

  • touchmove - user interaction with screen
  • accel - orientation of the device and its position
  • text - a text provided by user

I extended that application to be able to send error codes.
 

Pressing "Send Error" button and selecting error code sends a MQTT message to IoTF broker.

Typical error message looks like;
{
device_id"doors9"
evt_type"error"
timestamp:
{
$date1449059192155
}
evt: 
{
errorCode"20"
}

}

For a compiled version of this application follow this link.
Now I could start updating my module.

Design for Analytics


Requirements in my DOORS module have a "ErrorCode" integer attribute which links a requirement to a error code reported by my device.

Additionally I'm using DOORS module level attributes to store values I do not need hardcoded in my DXL. Those are:
  • Authorization (string)- so I do not need to calculate Base64 on each call
  • DeviceType 
    (string) - which of my devices this module describes
  • Organization (string) - my Bluemix organisation name
With all information in place I can write some simple layout DXL (which can be convert to attribute DXL later on to improve performance)

Layout DXL 

I want a specific type of event from all devices of a given type from my organization. So I need to use a historic query for a device type:

buf = "https://internetofthings.ibmcloud.com/api/v0001/historian/" org"/" deviceType "?evt_type=" eventId

This will return a JSON string with a list of events. If you do not have a JSON parser ready you can try to parse this data with Regular Expressions. Please remember this is a very simple example and in real world one shall not attempt to parse JSON with regular expressions.

My main worker code looks like:
if (!null obj) {
  int ival = obj."ErrorCode"
  if (ival == 0) halt
 
  string val = obj."ErrorCode"
  Module m = module obj
  string auth = m."Authorization"
  string dtype = m."DeviceType"
  string org = m."Organization"
 
  if (!null m && auth != "" && org != "" && dtype != "")
  {
    Buffer b = getData(auth, org, dtype, "error")

    if (!null b)  {
      string s = stringOf b
      Regexp re = regexp2 "\"device_id\":\"([^\"]*)\"[^}]+.[^}]+{\"errorCode\":\"([^\"]*)\""
      int i = 0
      string device = "", code =""
      //  temporary skip to hold names of devices which reported
      Skip erSkp = createString
      int allErrors = 0
      int numDevices = 0
      while (!null s && re s && i<100) {  // i is just a guard, we know there is no more then a 100 results in one page
        device = s[match 1]
        code = s[match 2]
        int ireported = 0
        // if code matches attribute value
        if (code == val)
        {
          allErrors++ // icrease number of errors
         
          if (!find(erSkp, device, ireported)) {
            put(erSkp, device, 1)
            numDevices++
          }
          else {
            ireported++
            put(erSkp, device, ireported, true)
          }
        }
        s = s[end 0 +1:]
        i++
      } // while
      // clean up
      delete b
     
      // report
      if (allErrors != 0) {
        for ireported in erSkp do {
          device = (string key erSkp)
          displayRich "Device with Id {\\b "device "} reported an issue " (ireported == 1 ? "once" : ireported" times")
        }
      }
     
      delete erSkp
    } // null b
  } // module setup
} // !null obj

You can find full DXL here.

Conclusion


Above layout DXL works fine when there're not so many devices. Once there will be more of them we no longer want to see how many times each device reported given issue. Thus the DXL could be rewritten to show something like:
Example report


As you saw in this example I'm using layout DXL but I think for a better understanding and feedback, one should consider writing a utility DXL.
Maybe that utility could provide its own UI for easier navigation?

In example above there's no need to send a HttpRequest for each object... It is enough to make one call per 100 (max page) events returned by query and write a little more complex Skip management. That however would require to make one top level Skip, but I'm sure you all know how to do it.

Sunday, 29 November 2015

Moving DOORS9 towards real-time IoT sensors

Last time I shown how easy it is to use historian from your IoT enabled device data in DOORS using IoT Foundation HTTP API. I thought 'it cannot be that hard to show real-time data', I know there are no MQTT callbacks in DXL, no JSON parser (sure we can write one, but what's the point?).

Sounds challenging. So let's do it!

Let's play!

If you have spare 15 minutes please have a look at wonderful presentation by Dr John Cohn (@johncohnvt) - "The importance of play".

Let's create a very simple HTML static page you can open in any (recent!) browser and which will connect to IoT Foundation.

Very (and I mean it) simple HTML page

The page will have just two p elements, one for static info, and one used dynamically from JavaScript. It will additionally include 3 scripts; jquery, paho and MQTT connection helper class.
The entire page code is simply:
<!DOCTYPE html>
<html>
    <head>
        <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
        <script type="text/javascript" src="jquery.min.js"></script>
        <title>MQTT listener</title>
    </head>

    <body onContextMenu="return false">
        <p>Please do not close</p>
        <p id="pStatus"></p>
    </body>
        <script type="text/javascript" src="realtime.js"></script>
        <script type="text/javascript" src="mqttws31.js"></script>
        <script type="text/javascript">
          var realtime = new Realtime(org, apikey, apitoken, device_type, device_id);
        </script>
</html>

MQTT connection

As you know IoT devices communicate over MQTT, lightweight device to device protocol implemented in many languages. Among which there are many JavaScript implementations available, which you can find at http://mqtt.org/tag/javascript. Personally I'm using Paho because it is very easy to use with IBM Bluemix services.

Using Paho is straight forward, you need to create new Messaging.Client object providing host, port and clientId.
For newly created client you should provide two message callbacks:
  • onMessageArrived - function(msg)
  • onConnectionLost - function(e)
client = new Messaging.Client(hostname, 8883,clientId);
client.onMessageArrived = function(msg) {
  var topic = msg.destinationName;
  var payload = JSON.parse(msg.payloadString);
  data = payload.d;
};

client.onConnectionLost = function(e){
  console.log("Connection Lost at " + Date.now() + " : " + e.errorCode + " : " + e.errorMessage);
  this.connect(connectOptions);
}

Then you construct connection options object which will be passed to connect function of Messaging.Client. This will start connection with IoT MQTT Broker.
You have to specify:
  • timeout - number
  • useSSL - boolean
  • userName - string - as usual this will be API Key for your organization
  • password - string - and API Token  
  • onSuccess - function() - executed when connection was successful
  • onFailure - function(e) - execeuted on failure
var connectOptions = new Object();
connectOptions.keepAliveInterval = 3600;
connectOptions.useSSL=true;
connectOptions.userName=api_key;
connectOptions.password=auth_token;

connectOptions.onSuccess = function() {
  console.log("MQTT connected to host: "+client.host+" port : "+client.port+" at " + Date.now());
  $('#pStatus').text("MQTT connected to host: "+client.host+" port : "+client.port);
  self.subscribeToDevice();
}

connectOptions.onFailure = function(e) {
  console.log("MQTT connection failed at " + Date.now() + "\nerror: " + e.errorCode + " : " + e.errorMessage);
  $('#pStatus').text("MQTT  connection failed at " + Date.now() + "\nerror: " + e.errorCode + " : " + e.errorMessage);
}

console.log("about to connect to " + client.host);
$("#pStatus").text("about to connect to "+client.host);

client.connect(connectOptions);

Once you establish connection with IoT MQTT Broker you should subscribe to some topic.

var self = this;
// Subscribe to the device when the device ID is selected.
this.subscribeToDevice = function(){
  var subscribeOptions = {
    qos : 0,
    onSuccess : function() {
      console.log("subscribed to " + subscribeTopic);
    },
    onFailure : function(){
      console.log("Failed to subscribe to " + subscribeTopic);
      console.log("As messages are not available, visualization is not possible");
    }
  };
  
  if(subscribeTopic != "") {
    console.log("Unsubscribing to " + subscribeTopic);
    client.unsubscribe(subscribeTopic);
  }
  subscribeTopic = "iot-2/type/" + deviceType + "/id/" + deviceId + "/evt/+/fmt/json";
  client.subscribe(subscribeTopic,subscribeOptions);
} 

DOORS9 HTML window

Now it's time to display this very simple page in DOORS 9:
bool onB4Navigate(DBE dbe,string URL,frame,body){return true}
void onComplete(DBE dbe, string URL){}
bool onError(DBE dbe, string URL, string frame, int error){return true};
void onProgress(DBE dbe, int percentage){return true}

Module refreshModuleReff = null
void moduleRefreshCB(DBE x) {
  if (!null refreshModuleReff) {
    refresh refreshModuleReff
  }
}

string surl="c:\\iot\\realtime.html"
DB iotUIdb = create("do not close")
DBE iotUI=htmlView(iotUIdb, 100, 50, surl, onB4Navigate, onComplete, onError, onProgress)
DBE t = timer(iotUIdb, 1, moduleRefreshCB, "ping")
startTimer(t) 
realize iotUIdb 

You probably noticed I'm using timer functionality (page 48 in DXL Reference Manual), that's because layout DXL is run when the module is being refreshed, redrawn, etc. This simple code ensures it is refreshed every second.
There is a lot of space for improvement in above DXL; you can add a toggle to check if user wants refresh, you can add noError/lastError block before calling refresh refreshModuleReff. 

Now save above code as %doors_home%/lib/dxl/startup/iot.dxl and restart DOORS. This will create a top level window which can be accessed from any DXL context in your DOORS 9 session.

Layout DXL

Finally we can add some worker code. This will be using hidden perm which has following syntax:
string hidden(DBE, string)

string property = obj."accel_property"
if (property != "none") {
  noError()
  refreshModuleReff = current Module
  string strval = hidden(iotUI, "realtime." property "()")
  string err = lastError
  if (!null err) { halt }
  real val = realOf strval

  //enable blow if your Module is similar to the one from last post
  //obj."iot_data" = val
  display strval
} 

I put whole code from "MQTT Connection" chapter into realtime.js and created additional helper methods to get required
 
this.yaw = function() {
  if (data != null) return data.yaw;
  return -101;
}
this.pitch = function() {
  if (data != null) return data.pitch;
  return -101;
}
this.roll = function() {
  if (data != null) return data.roll;
  return -101;
}
var data = null;


Now the module is displaying real-time data from a device connected to IoT Foundation. It has 1 second delay, maybe you want to synchronize data object when it's retrieved from realtime.js.

Well afterwards I didn't need to write a JSON parser in DXL!

Conclusions

DOORS reading IoT real-time data is easy and possible! It was even simpler then I thought initially ;)

Remember in this blog I'm just giving examples what can be done, it's up to you to extend it with unsubscribe, change device type and device depending on current module. There are many thing you might want to add.

Tuesday, 17 November 2015

DOORS in IoT World

Last time I shown a quick example how to use HTTP request perms (DOORS functions) to get data from Cloudant database.
Today let's go a little step further and try to get more recent data.

Before you will continue with DOORS connection to IoT Foundation API please have a look at IoT Foundation Starter application. There you will learn how to create a simple IoT device, deploy a IoTStarterApp to your mobile, and start talking IoT! That's something I'm not covering in this blog (yet).

IoT connectivity

IoT devices communicate over Machine 2 Machine connectivity protocol - MQTT Message Queue Telemetry Transport. This protocol is extremely lightweight protocol much lighter than HTTP. MQTT becomes more and more popular, not only in IoT world but also in mobile data exchange, enven Facebook Messenger uses MQTT.

DOORS and IoT connectivity

MQTT protocol requires message broker and message client. From DOORS client point of view IBM IoT Foundation can be a message broker, but what with MQTT client? Well at the moment I do not see easy/best solution, one can try writing a MQTT client in DXL, or using some external library OLE interface. However MQTT messages are great for a real time data which can feed IoT Real-Time Insights or IoT Foundation.

OOTB DOORS can consume a historic data. In order to get a historic data from your device you can use IBM IoT Foundation HTTP API and you can do it using HTTP perms available in DOORS.

You can get events for:
  • every device from your organization - so all device types, all devices.
  • all devices of selected device types in your organization 
  • a single device
Each of possible historic data queries has a number of filters available.

Authorization 

IoT Foundation HTTP API uses a Basic Authentication in combination with HTTPS. You need to create  API key for your IoT service.

IoT API Key generation

When authenticating API Key is used as user name and its associated Auth Token as the password. Those should be provided in HTTP Authorization header in all requests.

Following DXL will add set authorization header:
  HttpHeader h = create
  string auth
  toBase64_(apiKey":"apiToken, auth)
  auth = auth[0:length(auth) -2]
  add(h, "Authorization", "Basic "auth)

toBase64_ is a perm which will convert a string into its Base64 representation. It it adds a newline character to the resulting string thus is it removed before use.

Historic query

Since I'm going to use my script in Layout DXL I'm not going to query for devices or their types. Let's assume I know already all these information (or it can be stored in module level attributes) so I will focus on a single query on specific device. From documentation I know I need to send a GET request (with empty body) to a URL:

https://internetofthings.ibmcloud.com/api/v0001/historian/${org}/${type}/${id}?filter

where:
${org} is organization
${type} - device type
${id} is a device ID

filter will let me narrow down response:
top - selects a number of top most events - selecting top=1 is almost equivalent to getting data in real time ;)
start/end - interval of historical data to query at
evt_type - narrows down selected events
sumarize and sumarize_typeString - allows performing aggregate functions of selected events.

My layout DXL will select an average from last 100 'accel' events over a selected attribute:
Buffer buf = create
buf = "https://internetofthings.ibmcloud.com/api/v0001/historian/" org"/" deviceType "/" deviceId "?top=100&evt_type=accel&summarize={"attr"}"

'accel' is one of the events sent by IoTStartApp if you use different IoT device, then please change that accordingly.

Proposed usage scenario

The scenario here is that you have requirement(s) for an Android device that is being measured in the real world. Those measurements may mean that error messages are passed back and ultimately you would want to report against these from your original requirements. The attribute "accel_property" in the following code is the key that identifies the thing and measurement that you are interested in (and is relevant to the requirement).

DXL IoT column in DOORS module


Whole DXL is:
Buffer getData(string apiKey, apiToken, org, deviceType, deviceId, eventId, attr)
{
  Buffer buf = create
  buf = "https://internetofthings.ibmcloud.com/api/v0001/historian/" org"/" deviceType "/" deviceId "?top=1&evt_type=" eventId "&summarize={"attr"}"
  HttpHeader h = create
  string auth
  toBase64_(apiKey":"apiToken, auth)
  auth = auth[0:length(auth) -2]
  add(h, "Authorization", "Basic "auth)
  HttpResponse resp = httpRequest(HttpGet, tempStringOf buf, null, h)
  delete h
  delete buf

  if (!null resp && resp.isOk)
  {
    HttpBody b = resp.body
    Buffer respBuf = create
    respBuf += b.value

    delete resp
    return respBuf
  }
  else {
    if (!null resp) {
      display "error getting response " resp.code""
      delete resp
    }
    else {
      display "connection error"
    }
  }

  return null
}

if (!null obj) {
  string val = obj."accel_property"
  if (val != "none") {
    Buffer b = getData("API KEY", "Auth Token", "org", "type", "id", "accel", val)

    if (!null b)  {
      string re = stringOf b
      int l,o
      if (findPlainText(re, ":", l, o, false)) {
        display val " last value " re[l+1:length(re) -3]
      }
      delete b
    }
  }
}

As you can see I'm using any fancy JSON parser, my query returns a single variable so it can be easily extracted from a string. 
I used an extra enum attribute I created for my module so each of my requirements can select different variable from query.

Conclusion

DOORS can be really easily connected to IoT Foundation and access its data. With help of htmlView DBE one can add an extra DXL window displaying IoT data in Real-Time.
There is a lot DOORS can do for you!

Bonus lines of DXL for Real-Time data

"Turn your mobile phone into an IoT device" is an interesting extension to the article on m2m I showed you in the very beginning of this post. In compare to preview post on IoT starter application it has "Step 6. Create a Bluemix app to visualize sensor data". Where you can see how to deploy IoT sample NodeJS application which will visualize real-time or historic data from your organization. Once you deploy this application to your Bluemix account run following DXL for a simple HTML viewer:

void iotDataView(string srcUrl)
{
    DB dlg = create("IoT real-time data", styleCentered)
    DBE html = htmlView(dlg, 1000, 800, srcUrl, onHTMLBeforeNavigate, onHTMLDocComplete, onHTMLError, onHTMLProgress)
    show dlg
}
iotDataView "http://(your_application_name).mybluemix.net"

This will create windows similar to the following:

Sunday, 1 November 2015

DXL and Cloudant documents

It was a very busy month and I didn't have much time to write something interesting. Today I decided to do a quick tutorial how to use HTTP perms.

You can read more on HTTP Perms in DOORS DXL Reference

Reading Cloudant documents

We know DOORS integration is not only outgoing to outside World, sometimes we need to get data into DOORS. Today I will show you a very short DXL script which will let you get data from external database. Because of ease of use and simple connectivity I decided to connect to Cloudant.

If you are not familiar with Cloudant, have a quick look at its Learning Center. In short Cloudant is a NoSQL database-as-a-service (DaaS) widely used with Bluemix Mobile and Cloud applications. But let's not jump into my future post and let me show you how can you read Cloudant documents from DOORS DXL.

As you know Cloudant uses HTTP requests and my script will focus on DXL HTTP requests.

Buffer getData(string username, password, account, db, doc)
{
  Buffer buf = create
  buf = "https://" username ":" password "@" account ".cloudant.com/" db "/" doc
 
  HttpHeader h = create
  HttpResponse resp = HttpRequest(HttpGet, tempStringOf buf, null, h)
  delete h
  delete buf

  if (!null resp && resp.isOk)
  {
    HttpBody b = resp.body
    Buffer respBuf = create
    respBuf += b.value
   
    delete resp
    return respBuf 
  }
  else {
    if (!null resp) {
      print "error getting response " resp.code""
      delete resp
    }
    else {
      print "connection error"
    }
  }

  return null
}

Buffer b = getData("", "", "ae4582eb-c21f-4c1a-92b3-8da8589eea0c-bluemix", "doors9test", "aee5f8c1be606328ddedef1a546cb7ca")

if (!null b)  {
  print tempStringOf b
  delete b
}


I set read permissions on this Database so everyone should be able to get following JSON:

{"_id":"aee5f8c1be606328ddedef1a546cb7ca","_rev":"1-f44612cc036915a9f12f1afb7ba2fc37","title":"DOORS 9 connection test","description":"You shall read this ;)"}

Writing documents back to Cloudant


If you would like to PUT something to Cloudnant database you can use the same HttpRequest but this time you need to set HttpHeader Content-Type. This is required buy HttpRequest with HttpPut verb, otherwise an error will be reported. Setting header values is done with add perm:

HttpHeader h = create
add(h, "Content-Type", "application/json")  // minimum required header
add(h, "Accept", "application/json")

setValue perm is used to set HTTP body:
HttpBody body = create
Buffer buf = create
buf = {"_id":"aee5f8c1be606328ddedef1a546cb7ca","_rev":"1-f44612cc036915a9f12f1afb7ba2fc37"}
setValue(body, buf)
HttpResponse resp = HttpRequest(HttpPut, url, body, h)
That's how you can simply put and get data from the Cloudant!

Soon I will do some more fun stuff with DXL, Bluemix and IoT but for now...

Happy Halloween!

Wednesday, 19 August 2015

C# integration

As many of you know DOORS has a COM interface. Some use it via VBS, other via Java.
Today I will show you how to make a very simple application with C#.

VBS example

Let's see very simple VBS example:

Set DOORSObj = CreateObject("DOORS.Application")

DOORSObj.runStr ("ack ""hello""")


It's so simple to call a DXL script from VBS!

C# Example

Well C# is little bit more complex than VBS. But once all steps are done it is very easy to use it.

CLR Assembly

First you need to create a .NET common language runtime (CLR) assembly. I used TlbImp.exe. This converts type definitions found in COM type library into equivalent definitions in CLR assembly.
You can find DOORS type library in DOORS bin folder:

C:\>"C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0A\Bin\NETFX 4.0 Tools\x64\TlbImp.exe" "c:\Program Files\IBM\Rational\DOORS\9.6\bin\doors.tlb"

Above created a DOORSCOMLib.dll in c:\.

Once you have this dll you can simply add it to your C# project.
  1. Select References
  2. Add Referece...
  3. Browse to the location where is your DOORSCOMLib.dll.
  4. Once imported right-click on DOORSCOMLib and see its properties
  5. Set "Embed Interop Types" to false.

Embed Interop Types set to false
Embed Interop Types set to false

Now you can use this library in your code.

DOORSCOMLib interface

DOORSCOMLib has a following interfaces:


IDoorsDXL lets you run DXL:
  • void runDXL(string)
  • void runFile(string) 
  • string restult()
IDoorsStatus enables checking DOORS session status:
  • string clientVersion
  • string clientStatus (1 user logged in, 0 DOORS started but user not logged)

Usage

Usage of DOORSCOMLib is very simple; simply declare a required interface and use it.
IDoorsStatus doorsStatus = Activator.CreateInstance(typeof(DOORSClass)) as IDoorsStatus;

IDoorsDXL doorsDXL = Activator.CreateInstance(typeof(DOORSClass)) as IDoorsDXL;

Here is a simple "Hello Word" application illustrating how to use those interfaces:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using DOORSCOMLib;

namespace HelloCOM
{
    class Program
    {
        static void Main(string[] args)
        {
            IDoorsStatus doorsStatus = null;
            doorsStatus = Activator.CreateInstance(typeof(DOORSClass)) as IDoorsStatus;
            if (doorsStatus == null) {
                Console.WriteLine("Null instance");
                return;
            }

            // the session is up, but user might not be logged in
            int trycount = 0;
            while (doorsStatus.clientState != 1 && trycount < 30)
            {
                Console.WriteLine("DOORS not logged in yet");
                System.Threading.Thread.Sleep(2000);
                ++trycount;
            }

            // we have a session
            Console.WriteLine("DOORS version " + doorsStatus.clientVersion);

            IDoorsDXL doors = Activator.CreateInstance(typeof(DOORSCOMLib.DOORSClass)) as IDoorsDXL;

            if (doors != null) {
                doors.runStr("ack \"hello from C# !\"");
            }
        }
    }
}

This should start a DOORS session (if not already started) and display a "hello from C# !" message.

Update

In Visual Studio 2010 or 2013 you can simply added doors.tlb to the references and you can skip creating CLR Assembly.

Thursday, 30 July 2015

OSLC and DOORS 9 Hierarchy

OSLC is still evolving standard. Currently there is a version 2, which even though it has major improvements over version 1, it still lacks some definitions which could help when it comes to data representation. 

When you have a look at the OSLC Resource RequirementCollection you might notice the only relation to a OSLC Requirement is oslc_rm:uses. Described as:
"A collection uses a resource - the resource is in the requirement collection"

Thus OSLC RequirementCollection could be illustrated as an array or a table of non-related requirements.


That's not really useful when you look at the size and structure of a single DOORS 9 module used in production environment. You know a module it is not a flat list with thousands of requirements, it is more like a tree with lots of branches where each Heading or Subheading is a local root.
DOORS 9 structure is more complex.

Loosing a structure when converting DOORS 9 module to OSLC Requirements Collection might not be a problem when you want simply display its unordered contents or simply find a Requirement. However the power of DOORS comes from relations, traceability and reporting which are often key factors in many standards.

How to make a tree tree

Sadly OSLC cannot help us with requirements parenting or sorting so we need to get help elsewhere. 

If you had a look a my introductory post on OSLC DXL Services you know what to do!
Indeed all you need is a DXL Service which would return the structure for you. Yeah I know there were those perms and scripts you had to write...
Well since that post on DXL services there was DOORS 9.6.1.3 release and guess what! OSLC DXL Services got their UI. Finally! I mean DXL services perms were there for quite a long time, yet there was not much use of them.

In Database Explorer File -> OSLC you can see new item "DXL Services"
new menu flow in DOORS 9

Similarly to other configuration tools in DOORS this menu is not available for everyone.

This opens DXL Services UI which is relatively simple, yet does its job.
DXL Services UI

Hierarchy DXL script

In the picture above helloWord is the script from preview post, but what is "hierarchy"? I know I didn't put description but the function name gives some hints ;)

Code listing for hierarchy.dxl can be found here. It wasn't tested thoroughly, it might be, and should be optimized - specially for URL re-usage but that's not a part of this post.

Once you add this script to your DXL services it is ready to use.

Short summary how to get use of a DXL service:
Send a HTTP PUT request to your server with specified service name

https://yourserver:8443/dwa/rm/dxl/hierarchy

setting header to:
accept: application/rdf+xml

and content initialized with DXL services arguments (remember to encode your arguments for XML):
<?xml version="1.0" encoding="UTF-8"?>
<rdf:RDF
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:doors="http://jazz.net/doors/xmlns/prod/jazz/doors/2.0/">
  <doors:Arguments>
  <doors:arguments>https%3A%2F%2Fyourserver%3A8443%2Fdwa%2Frm%2Furn%3Arational%3A%3A1-DBID-M-MODULE_ID</doors:arguments>
  </doors:Arguments>
</rdf:RDF>


DWA should return with the structured document similar to below:
<?xml version="1.0"?>
<rdf:RDF
    xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
    xmlns:doors="http://jazz.net/doors/xmlns/prod/jazz/doors/2.0/">
  <doors:DxlServiceResult rdf:about="https://yourserver:8443/dwa/rm/dxl/hierarchy">
    <doors:result rdf:parseType="Literal"><OSLCDXLService name="oslc_getHierarchy" version="1.0">
    <Module uri="https://yourserver:8443/dwa/rm/urn:rational::1-DBID-M-MODULE_ID" fullName="/Folder" viewName="" />
    <Object uri="https://yourserver:8443/dwa/rm/urn:rational::1-DBID-O-1-MODULE_ID">
        <Object uri="https://yourserver:8443/dwa/rm/urn:rational::1-DBID-O-2-MODULE_ID">
            <Object uri="https://yourserver:8443/dwa/rm/urn:rational::1-DBID-O-8-MODULE_ID">
                <Object uri="https://yourserver:8443/dwa/rm/urn:rational::1-DBID-O-14-MODULE_ID">
                    <Object uri="https://yourserver:8443/dwa/rm/urn:rational::1-DBID-O-20-MODULE_ID">
                        <Object uri="https://yourserver:8443/dwa/rm/urn:rational::1-DBID-O-24-MODULE_ID">
                            <Object uri="https://yourserver:8443/dwa/rm/urn:rational::1-DBID-O-25-MODULE_ID"/>
                            <Object uri="https://yourserver:8443/dwa/rm/urn:rational::1-DBID-O-26-MODULE_ID"/>
                            <Object uri="https://yourserver:8443/dwa/rm/urn:rational::1-DBID-O-27-MODULE_ID"/>
                        </Object>
                    </Object>
                    <Object uri="https://yourserver:8443/dwa/rm/urn:rational::1-DBID-O-15-MODULE_ID"/>
                    <Object uri="https://yourserver:8443/dwa/rm/urn:rational::1-DBID-O-19-MODULE_ID"/>
                </Object>
                <Object uri="https://yourserver:8443/dwa/rm/urn:rational::1-DBID-O-9-MODULE_ID"/>
                <Object uri="https://yourserver:8443/dwa/rm/urn:rational::1-DBID-O-13-MODULE_ID"/>
            </Object>
        </Object>
        <Object uri="https://yourserver:8443/dwa/rm/urn:rational::1-DBID-O-3-MODULE_ID"/>
        <Object uri="https://yourserver:8443/dwa/rm/urn:rational::1-DBID-O-7-MODULE_ID"/>
    </Object>
</OSLCDXLService></doors:result>
  </doors:DxlServiceResult>
</rdf:RDF>

Now depending on your needs you can parse it and create DOORS 9 like view of a Module in your application.

Friday, 5 June 2015

OSLC DXL Services

One of the primary strategies of the IBM Rational division is to continually improve upon the integration capabilities of Rational and non-Rational products.
Keystone to this strategy is Open Services for Lifecycle Collaboration (OSLC). 
However a common complaint heard from providers / consumers of OSLC functionality is that there can be certain business problems that cannot be addressed because they rely on data and/or functionality that are (a) not currently available in the implemented standard or (b) considered too specialized for inclusion in the standard and therefore not planned for future versions.
Yet all is not lost and as OSLC service discovery is extensible it is possible to add services to help close such gaps. 
An excellent example of this can be seen in the introduction of OSLC DXL Services to Rational DOORS.

What are OSLC DXL Services?

Put simply they are an extension to the Rational DOORS OSLC RM V2 interface which allows users to execute Rational DOORS DXL scripts safely and securely across HTTPS.

Yes you can run DXL scripts from DWA! Well not all DXL scripts you have of course and only those registered by Administrator as OSLC DXL service but still this gives your project new possibilities and powers.

What makes DXL Script a OSLC DXL Service?

If you looked at IBM OSLC DXL Service introduction you came across helloWorld.inc:

void getHelloString(string language)
{
    string hello = null
    if ("French" == language) {
        hello = "Bonjour le monde" 
    }
    else if ("Finnish" == language) {
        hello = "Hei maailma"
    }
    else if ("Latin" == language) {
        hello = "Ave mundi" 
    }
    else {
        hello = "Hello world" 
    }

    setDxlServiceResult hello
    print hello "\n" 
}
If you are familiar with DXL you can spot setDxlServiceResult PERM. That is exactly what will set value of hello as the result of the service.

Registering DXL Service

Currently DOORS Database Administrator has to add those services manually using DOORS Perms.

OSLCDXLService os = null
string err = null
string dxlCode = "#include <addins/services/helloWorld.inc>"
err = addOrUpdateOSLCDXLService("helloWorld", "Hello world in several languages", dxlCode, "getHelloString")
if (!null err){
      print err
} else {
      print "Installed Service\n"
}

Quick PERM reference (as it's not yet documented in DOORS DXL Reference Manual)

void setDxlServiceResult(string s)
sets s as the result for DXL Service

string addOrUpdateOSLCDXLService(string n, desc, dxlcode, entryPoint) 
registers dxlcode as the service with name n and description desc. entryPoint specifies the function in dxlcode to be run as entry point of service.

To make life easier OSLC DXL Services management will be soon available in File menu in IBM DOORS client.


Call OSLC DXL Service

Once you have your service registered you can send GET and PUT requests.
Set both the accept and content-type headers to this entry: application/rdf+xml and submit:
 
  • GET request for the DXL service description.
  • PUT request with optional parameters to invoke the service.

helloWorld service registered above could be accessed with URI like https://servername:portnumber/dwa/rm/dxl/helloWorld

As you notice before helloWorld function takes one parameter which should be supplied. You can provide parameters for DXL Services with aid of request content:


<?xml version="1.0" encoding="UTF-8"?>
<rdf:RDF
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:doors="http://jazz.net/doors/xmlns/prod/jazz/doors/2.0/">
  <doors:Arguments>
  <doors:arguments>French</doors:arguments>
  </doors:Arguments>
</rdf:RDF>

If you have more then one argument, separate those with comas ','. All arguments are passed to DXL service as strings. If you need to have other types use DXL conversions in your service.


The results


Once DXL Service will finish its operation DWA server will return response with a content in following form:

  dwaOauth.js
was used to GET only protected information from DWA. But calling OSLC DXL Services requires PUT request which should be added.

There's nothing easier:

exports.getDXLServiceResult = function(req, res, dxlService, lang, next, onError) {
    var oa = new OAuth(req.session.oa._requestUrl,
              req.session.oa._accessUrl,
              req.session.oa._consumerKey,
              req.session.oa._consumerSecret,
              req.session.oa._version,
              req.session.oa._authorize_callback,
              req.session.oa._signatureMethod);

        // Try to get protected data
        var post_body = '<?xml version="1.0" encoding="UTF-8"?><rdf:RDF    xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"    xmlns:doors="http://jazz.net/doors/xmlns/prod/jazz/doors/2.0/">'
        post_body += '<doors:Arguments><doors:arguments>' + lang +'</doors:arguments></doors:Arguments></rdf:RDF>';

        oa._headers['accept'] = "application/rdf+xml";

        var host = url.parse(location);
        console.log('requesting service ' +host.protocol+"://"+host.host+'/dwa/rm/dxl/'+dxlService + ' with param: '+lang);

        oa.put(host.protocol+"//"+host.host+'/dwa/rm/dxl/'+dxlService,
            req.session.oauth_access_token,
            req.session.oauth_access_token_secret,
            post_body, 
            'application/rdf+xml',
            function (error, data, response) {
                if (!error && next) {
                    next(req, res, data);
                }
                else if (onError) {
                    onError(error);
                }
            }
        );
}

And that's it!.

Friday, 22 May 2015

Node.js OAuth handshake with DWA

In first post I show you basic OAuth handshake between your Node.JS application and DOORS Web Access Server.

This post will be longer than other ones, here I will show you how to create application step-by-step. In next posts I will be just extending functionality of this one by adding some services or new methods.

What you'll need to build your application

If you do not have Git on your machine I can recommend Eclipse eGit, which nicely integrates Git software control manager (Git SCM) into your development environment.


Since this application is written in Node.JS you might also want to install Nodeclipse to get default formatting and language highlighting.

Any time you can also use command line tools for which you will need Node.JS and some Git client.

That's all you need to start your local server development, but we know things can quickly escalate so you might want to have a look at IBM Cloud services.

To the Clouds and beyond

If you need more services, processes, memory, storage, you might want to host your application on the Cloud. IBM Bluemix is a Next-Generation App development platform. All you need is your IBM ID and you can start developing cloud applications.

If you already have an IBM ID follow this link to begin your free trial. Otherwise you can create one using the IBM Bluemix registration page.

Once you have your IBM Bluemix account trial started a good idea would be to complete the Getting Started with IBM Bluemix tutorial.

Install dependencies

Before we can write the application we should provide Node.js dependencies. This demo application makes use of the following modules: express, oauth, request, cookie-parser and libxmljs.


These need added those to your package.json
{
  "name": "HelloDWAOAuth",
  "version": "0.0.1",
  "description": "A sample nodejs app for OAuth with DWA",
  "dependencies": {
    "express": "3.4.7",
    "jade": "^1.1.4",
                "libxmljs": "0.13.0",
                "cookie-parser":"",
                "oauth": "",
                "request":"",
                "util": "",
                "url" : ""
  },
  "engines": {
    "node": "0.10.*"
  },
  "repository": {}
}

Open command line and navigate to your project directory, and type:

> npm install

This will install all of the required dependencies based on your package.json

Note:

Sometimes you can get errors like:
The builds tools for Visual Studio 2010 (Platform Toolset = 'v100') cannot be found.install new module.
 

It might be useful to try –msvs_version=[2005, 2008, 2012] switch if default build will fail.

Writing the server application

This demo uses Express. You will need to create express instance, define routes and provide functions that might be required to execute those routes. Here is the basic code for an express server.

var express = require('express');

// setup middleware
var app = express();
app.use(app.router);
app.use(express.errorHandler());
app.use(express.static(__dirname + '/public')); //setup static public directory
app.set('view engine', 'jade');
app.set('views', __dirname + '/views'); //optional since express defaults to CWD/views

// render index page - main route
app.get('/', function(req, res){
                    res.render('index');
});

var appInfo = JSON.parse(process.env.VCAP_APPLICATION || "{}");
var services = JSON.parse(process.env.VCAP_SERVICES || "{}");
var host = (process.env.VCAP_APP_HOST || 'localhost');

// The port on the DEA for communication with the application:
var port = (process.env.VCAP_APP_PORT || 3000);
// Start server
app.listen(port, host);
console.log('App started on port ' + port);

16 lines of code is all you need for a basic HTTP server written in Node.js. Put it into a new app.js file and that will be an entry point for our application.

Certificates

If you will be working in a development environment you might need to use self-signed certificates.

Those by default will be rejected and no connection will be made. If you want to prevent self-signed certificate error add following line:
 process.env.NODE_TLS_REJECT_UNAUTHORIZED = "0";

Adding OAuth to your application

Let's add OAuth as a separate Node.js module so it won't clatter the main app.js code. Modular approach will also help you to separate OAuth module and reuse it in other applications.

So now we will have app.js and Oauth.js files.

App.js will contain main routes and application logic for our sample. OAuth 1.0 requires user to provide application key and secret used to authenticate the application with a remote server. Thus our application will have two pages, where the user will be asked to provide location of rootservices, application key and OAuth secret.

Once those are provided the user will be redirected to page providing basic access to protected resources.

In this example we are using oauth Node.js module which comes with very useful examples how to do a basic Oauth handshake.

Configuring OAuth

Let's start with gathering information required to start OAuth. Create body.jade to add a form with POST action to /set_key_and_secret:

form(name="loginform", action="/set_key_and_secret", method="post")
          | Rootservices location:
          input(type="text", name="rootservices")
          br
          | Consumer Key:
          input(type="text", name="key")
          br
          | OAuth Secret:
          input(type="password", name="secret")
          br
          input(type="submit", value="Submit")


Now we have to add POST route to our application (app.js):
app.post('/set_key_and_secret', function(req, res) {
    //if (typeof req.body.rootservices !== 'undefined') console.log(req.body.rootservices);
    if (typeof req.body !== 'undefined') {
        if (typeof req.body.rootservices !== 'undefined' && req.body.rootservices != '' &&
            typeof req.body.key !== 'undefined' && req.body.key != '' &&
            typeof req.body.secret !== 'undefined' && req.body.sercret != '')
        {
            dwaOauth.configure(req.body.rootservices, req.body.key, req.body.secret, process.env.VCAP_APP_PORT || 3000);
            console.log("got settings");
            res.redirect("/dwa_login");
        }
        else {
            res.redirect('/');
        }
    }
});
Note: request.body is not available by default in Node.js express, we need to inform our application we would like to have access to this object. In order to do this we need to add:

app.use(express.bodyParser());

This must be added before any application routes will be defined. Best place is to add it before:

app.use(app.router);

dwaOauth.js module

You could wonder what is dwaOauth.configure called in preview example:

var location=null, oauth_key=null, oauth_secret=null, port=null;
exports.configure = function(rootservices_, key_, secret_, port_) {
    location = rootservices_;
    oauth_key = key_;
    oauth_secret = secret_;
    port = port_;
}

It is a very simple function used to provide some rootservices location, application key and secret to your dwaOauth module.

dwaOauth.js module will handle OAuth 1.0 dance and will give application access to protected resources.
All we need it to get authorized OAuth access token. The following steps are required to successfully get the authorized OAuth token:
  1. Login to DWA to get JSESSIONID Cookie
  2. Request a OAuth token
  3. Authorize the OAuth token
  4. Get the access token

In '/dwa_login' route we create a OAuth object which will be used to (2) request the OAuth token and once it was correctly retrieved it will redirect user to (3) authorize it. OAuth module will handle those requests and it will also redirect user to perform (1) login to DWA in order to get JSESSIONID Cookie.

In app.js add:

//Request an OAuth Request Token, and redirects the user to authorize it
app.get('/dwa_login', dwaOauth.getOAuthURLs, dwaOauth.requestToken);


In routes/dwaOauth.js add:

exports.requestToken = function(req, res) {
    var oa = new OAuth(oauthRequestTokenUrl,
        oauthAccessTokenUrl,
        oauth_key,
        oauth_secret,
        "1.0",
        req.protocol+"://"+req.host+":"+port+ "/dwa_cb", // this should work on Bluemix and localhost
        "HMAC-SHA1");

    console.log("1. Request a OAuth Token - this will redirect to DWA login page");

    oa.getOAuthRequestToken(function(error, oauth_token, oauth_token_secret, results){
        if(error) {
            console.log('error : ' + util.inspect(error));
            res.end();
        }
        else {
            //store the tokens in the session
            req.session.oa = oa;
            req.session.oauth_token = oauth_token;
            req.session.oauth_token_secret = oauth_token_secret;

            console.log("2. Authorize the OAuth Token ");

            res.redirect(oauthUserAuthorizationUrl +"?oauth_token="+oauth_token);
        }
    });
}

Rootservices and OAuth

As you probably notice '/dwa_login' requires getOAuthURLs before it will be executed. getOAuthURLs method parses DWA rootservices so oauthRequestTokenUrl, oauthAccessTokenUrl and oauthUserAuthorizationUrl are discovered automatically. We do not need to store those in any configuration file.

// get URL from root services
exports.getOAuthURLs = function(req, res, next){
    console.log('Attemp to get OAuth URLs ' + location);
    if (location == null || oauth_key == null || oauth_secret == null) {
        return next('not configured');
    }

    request(location, function(error, response, body){
        if (!error && response.statusCode == 200) {
            var xmlDoc = libxmljs.parseXml(body);
            oauthRequestTokenUrl = xmlDoc.get('jfs:oauthRequestTokenUrl', ns.jfs).attr('resource').value();
            oauthAccessTokenUrl = xmlDoc.get('jfs:oauthAccessTokenUrl', ns.jfs).attr('resource').value();
            oauthUserAuthorizationUrl = xmlDoc.get('jfs:oauthUserAuthorizationUrl', ns.jfs).attr('resource').value();
            next();
        }
        else {
            console.log('error' + util.inspect(error));
            return next(error);
        }
    });
}
ns is defined as:
exports.ns = ns = {
  dcterms:{ 'dcterms': "http://purl.org/dc/terms/" },
  jfs: { 'jfs': 'http://jazz.net/xmlns/prod/jazz/jfs/1.0/' },
  oslc_rm:{ 'oslc_rm': 'http://open-services.net/ns/rm#' },
  oslc:   { 'oslc': 'http://open-services.net/ns/core#' },
  rdf: { 'rdf': 'http://www.w3.org/1999/02/22-rdf-syntax-ns#' },
  doors:  { 'doors': 'http://jazz.net/doors/xmlns/prod/jazz/doors/2.0/' }
So namespaces are defined and easier to use.
Above code uses request to get rootservices and libxmljs to parse it. As mentioned earlier rootservices are the only resource not protected by OAuth so we do not need to worry about setting JSESSIONID cookie or OAuth while accessing it.

Add callback route

The last step in OAuth dance is to get the access token. We will do it in callback we provided earlier while constructing OAuth object. This callback is another route defined in app.js as '/dwa_cb':

// Callback for the authorisation page
app.get('/dwa_cb', dwaOauth.callback);

and in dwaOauth.js:

exports.callback = function(req, res) {
    var oa = new OAuth(req.session.oa._requestUrl,
            req.session.oa._accessUrl,
            req.session.oa._consumerKey,
            req.session.oa._consumerSecret,
            req.session.oa._version,
            req.session.oa._authorize_callback,
            req.session.oa._signatureMethod);
    console.log("3. Get the access token");

    oa.getOAuthAccessToken(
            req.session.oauth_token,
            req.session.oauth_token_secret,
            req.param('oauth_verifier'),
            function(error, oa_access_token, oa_access_token_secret, results) {
            if(error) {
                console.log('error' + error);
            }
            else {
                req.session.oauth_access_token = oa_access_token;
                req.session.oauth_access_token_secret = oa_access_token_secret;
                //redirect to home page
                res.redirect("/");
            }
    });
}

Access protected resource (finally!)

Once we have the access token we can access protected resources. Last line in '/dwa_cb' route will automatically redirect user to a root page which as we saw earlier routes to /resource.

Let's add a resources route to app.js:

app.get('/resource', checkLogin, function(req, res) {
    res.render('resource', {"resource": ""});
});


As you can see it renders 'resource' page with resource parameter. We will use HTTP Get request to get that resource. Add following to dwaOauth.js

exports.getProtectedResource = function(req, res, resourceURL, next, onError) {
    console.log('requesting protected resource: '+resourceURL);
    var oa = new OAuth(req.session.oa._requestUrl,
          req.session.oa._accessUrl,
          req.session.oa._consumerKey,
          req.session.oa._consumerSecret,
          req.session.oa._version,
          req.session.oa._authorize_callback,
          req.session.oa._signatureMethod);

    oa._headers['OSLC-Core-Version'] = '2.0';
    oa._headers['accept'] = "application/rdf+xml";
 
    // Try to get protected data
    oa.getProtectedResource(
        resourceURL,
        "GET",
        req.session.oauth_access_token,
        req.session.oauth_access_token_secret,
        function (error, data, response) {
            if (!error && next) {
                next(req, res, data);
            }
            else if (onError) {
                onError(error);
            }
        }
    );
}
This is a general HTTP request and should work with any OAuth server (DWA, DNG, QM, etc).

Once you have this your dwaOauth.js you can use it in your app.js with following route:

function getResource(req_, res_, next) {
    if (req_.body.res_url != "") {
        dwaOauth.getProtectedResource(req_, res_, req_.body.res_url, function(req, res, data) {
            var xmlDoc = libxmljs.parseXml(data);
            var mod = xmlDoc.get('//dcterms:title', dwaOauth.ns.dcterms).text();
            next(req_, res_, "", mod);
        });    
    }
    else next(req_, res_, "", "");
}

app.post('/resource', checkLogin, function(req, res) {
    getResource(req, res, render);

});


If you provide a valid resource link, below code will attempt to read dcterms:tile of the remote resource.
Update resource.jade to include a form for Resource URL input.

form(action="/resource", method="post")
      h2 Sample 1: Get resource title
      p Please provide a URL for one of your resources (say link to module)
      p Resource URL:
       input(type="text", name="res_url")
       input(type="submit", value="Submit")
       br
       if resource == ""
        p
       else
        p Resource title: '#{resource}'
        br

You just learned know how to read data from OAuth protected server! Easy ;)

You can find listing for dwaOauth.js here.

Update:

In my original post I forgot to mention checkLogin function. Thanks to Dani for pointing that!

// main redirection method
// make sure all your routes to protected resources will go via this method 
function checkLogin(req, res, next) {
    // if session doesn't have a oauth access token redirect to login page.
    if(!req.session.oauth_access_token) {
        res.redirect("/");
        return;
    }
    next();
}

but that's not all what's needed. It's only a redirection to the default route which I need to include here as well.

//root page
app.get('/', function(req, res){    

    // check if session has oAuth token
    if(!req.session.oauth_access_token) {
            res.render("index");
        }
        else {
            res.redirect("/resource");
        }
});
index is a simple form where user provides DWA location, oAuth key and secret:
//- index.jade
body
  table
    tr
      td(style= "width:30%;" )
        img(src="/images/newapp-icon.png")
      td
        h1 Hi there!
 
        p First you will need to setup your OAuth server details:
        
        form(name="loginform", action="/set_key_and_secret", method="post")
          | Rootservices location:
          input(type="text", name="rootservices")
          br
          | Consumer Key:
          input(type="text", name="key")
          br
          | OAuth Secret:
          input(type="password", name="secret")
          br
          input(type="submit", value="Submit")

set_key_and_secret is defined earlier.