Do not give your manager a DOORS DXL Reference Manual

After reading my last post my manager came back to me with a DXL he wrote in one and a half days.

"Here is my DXL utility which can be used to perform requirements analysis on a module:" (source)

Requirements Analyzer tool UI

His selection was taken from various white-papers on the subject matter - such as that one from NASA - Automated Analysis of Requirement Specifications.

This tool creates a view in the module for each theme. It creates a compound "or" for all the keywords against either Object Text or all attributes.

Additionally only users will "manage database" can edit the themes.

You can see this UI in action in this video:


Well that's pretty cool!

Sure there is place for improvements, and it's not bullet proof, but that's not the point. Imagine what a Requirement Engineer could do in DOORS 9 if given a bit more time...

Maybe something similar to the use case I mentioned in my last post: use Watson Natural Language Classifier Service from Bluemix.

The story!

Once you have your requirements filtered you can have a list of rules your device must implement.
If you continue watching above video you will see how I created a rule from a very quick UI I wrote in DXL (source).

UI for Rule Creation in IoT Real-Time Insights

Code short and simple but shows you how to get schema information so you can:

• view condition items
• assign rule to schema 

If rule is successfully created its ID is stored in ruleId string attribute.

Of course, instead of extra attribute with value it could store an external link to:
https://iotrti-prod.mam.ibmserviceengage.com/api/v2/rule/{ruleId}

It would support 'follow' action, but user name and password which are apiKey and apiToken from VCAPS are not easy to remember ;)

As a exercise you can extend this example to enable action selection. In this way you will create a complete rule with action which can trigger alerts.

Traceability is Power

If you are working with DOORS 9 I'm pretty sure you are familiar with concept of Suspect links and changed objects. 

OOTB DOORS 9 let's you check information about in-links or out-links to open modules or to all modules.

Well that's great but with OSLC links, or even without links (if you select to stay with extra attribute in last chapter)?

With DXL you can easily do it, just retrieve your rule from IoT RTI and compare to what you have. See the last part of the video to see it working.

Suspect Linked Rule

As promised sources for

• rule2.dxl
• rule3.dxl

Conclusion

If you do not like challenges do not give your DXL Reference Manual to your Manager, he might come back with some great ideas!

2015 Blog Summary

It would be too easy to dismiss previous posts on "DOORS 9 and IoT" as disposable fun with no real value.

Yes they are bite size examples - deliberately so when you think about it - but at the high level they set out to prove a number of very serious points.

Those points are:

1. There are no barriers preventing you from using DOORS 9 with BlueMix and IoT services today.
2. There is no requirement for complicated, over-engineered solutions or add-ons. Forget that. You already have the most important piece - DOORS 9 itself.
3. You can jump right in and immediately start to explore and take advantage of these new and exciting technologies.

So we thought it would be helpful if we took a step back and tried things from a different angle.

This time the aim is to walk through an end-to-end scenario. In doing so we hope to demonstrate how best practice RM in DOORS 9 can be used alongside IoT to sustain ongoing efficiency, improvement and success.

Scenario for a Story

Consider a company that makes a device.

Now it isn't particularly important what this device does: it could be large or small, shiny or matte, there may be one of them or there may be thousands, it could sit in your home or it could move around on water, roads or the air.

However one thing for sure is that it is complicated, takes time to build and involves both hardware and software.

Now the company feels very strongly about this device. It is the market leader. It makes them money. Of course they are going to be emotional about it.

The company isn't stupid either. They realise that over time this device will need to change in order to continue market dominance.

Change may come for any number of reasons such as simple aesthetics, available of new materials or increased regulation.

Change may also occur because life has a frustrating habit of throwing up problems.

In other words the company's device is built to exacting standards but it can still fail.

To help deal with this failure the device includes sensing, monitoring and communication capabilities. In real-time there is a continual flow of information back to the company and naturally the company wants to act fast in diagnosing and fixing any problem.

This historical data can also be used when considering a new variant of the device. The company wants to identify small changes that can be made incrementally and hopefully at low cost.

Phew... that took a while to set the scene. Might have been better to use a GIF to present a "Star Wars" crawl, or a Mad Men sales pitch.

So let's gets back to DOORS and IoT.

How DOORS 9 fits into this story?

We may have already mentioned this but the company isn't stupid and this means their requirements are written and managed in DOORS 9, so let's start there...

There has been a lot of work over the years into what makes a good requirement.

For example if you google "nasa requirement quality analysis" you will find excellent whitepapers on natural language processing. See Automated Analysis of Requirement Specifications by Wilson, Rosenberg and Hyatt.

This essentially boils down to scaffolding the requirements process with tools that maximise the conciseness, consistency and overall quality of the statements used to describe what a thing should do.

There are more involved utilities available, but using DXL it is a relatively straight-forward exercise to parse your module looking for keywords and/or symbols. A filter can be applied to focus the attention on those that are ambiguous or likely to cause confusion.

You could even consider using a service such as Watson Natural Language Classifier.

On November 1st I've shown how to use DXL HttpRequest to connect to Clouding, so one shall know how to use it with DXL, no matter layout, attribute or utility.
So what's stopping our fictional company from feeding Watson NLC with their DOORS 9 data using DXL or NodeJS and OSLC with aid of a very first example I presented? With HttpRequest this company could use Watson NLC to classify requirements as according to:

• imperatives
• continuances
• directives
• weak phrases
• options

With this said there is no reason why the same utilities cannot be used to identify requirements that are essentially "rules" in their own right. This act alone helps system engineers focus the attention on the rules that will need added to Workbench or Real Time Insights.

In terms of our device this means that it would have a thing dedicated to capturing and reporting a specific behaviour or variable.

When RTI detects that a boundary condition is breached an alert is made.

The rule in RTI that controls this behaviour should be able to be extracted from a good requirement.

XXX must not exceed a maximum of YYY units.

So not only can we use DOORS 9 to increase the confidence in our requirements – but we can then use those requirements as a basis for development of things and the subsequent reporting capabilities.

Did I forget to mention DOORS 9 can easily create a rule (or action, alert, whatever you need and the API lets) in Real Time Insights or IoT Workbench (currently an experimental service)?
Well, you know the drill use your DXL HttpRequest to POST some data to a URL.
URL you found in this blog, in a post not only telling you how it fits into IoT World but also telling you where is the IoT Real Time Insights HTTP(S) API documentation, good luck finding that with any search engine (well apart from link to this post).

A thing in the wild

So the requirements are filtered and rules identified, implemented and introduced via things in the real world. Sooner or later something bad happens.

A message is sent to RTI from the thing.

We can easily report on this information from DOORS 9 - and contextualise it in terms of the requirements that perhaps could be a problem. Requirements are linked through to design, test case and source code - so impact analysis of change can be investigated and more importantly executed.

The Layout DXL columns posted earlier show this particular scenario playing out in DOORS 9.

Closing the lifecycle circle

But remember there was another use case.

The company feels it is time to consider a new variant of the product but they want to ensure maximum benefit from minimum change.

We’ve seen how errors can be handled but what about other intangibles. This is where IoT can really begin to drive benefit from the unstructured data.

What if the device was a car that was intended to get mpg at mph. Accelerate to mph in ss.

What if it never achieved that? The information to hand allows you to cross-check weight of passengers, softness of brakes, engine performance to determine areas of weakness.

Is this climate based – were environment factors to blame? Is the car configured wrongly OOTB for the geographical area?

Conclusions

Hopefully my 2015 posts will have raised your awareness and interest in IoT.

It represents fantastic levels of potential and my aim was show that in a few relatively small steps you can begin to unlock that potential now.

At this moment you have everything to need in order to take advantage of DOORS 9 and the GB/TB of data it sits on top of.

Any further advice I could give is best summed up by IBM Fellow John Cohn when he stresses "The importance of play":

DOORS 9 direct IoT support - build-in challenge #1

Monday morning is a good time for reflection and quick chat with coworkers before throwing yourself into work cycle.
In this way I had quick chat session with my manager yesterday. That ended up with challenge#1 (DOORS and IoT) and later on I had another quick chat my friend from DNG team, that ended up with challenge #2 (DOORS 9 itself).

Since challenge #1 is about IoT let's leave details of the other one for another post- but I like the idea so I will definitely post something in that subject.

Challenge #1

It was simply; "DXL triggers and IoT."

Well in my last post I used external tool written in NodeJS to publish DOORS 9 TRS feed to IoTF broker.

You need a full DOORS Web Access stack with some extra settings (as TRS is not enabled by default) to be able to get TRS feed which you will consume with your external application before posting that to IoT Foundation broker.

DWA Stack used in last post

There are no events in TRS so the last example was querying for changes every 10 seconds. That's far from real-time in IoT terms.

DOORS 9 Triggers

Have a look at DXL Reference Manual if you are not familiar with concept of triggers in DOORS 9.

Triggers are as close to real-time events as it can be in RM tool. Yeah but DOOR 9 doesn't have MQTT client.

Does it have to have it?

Well NO!

Looking at Internet of Things documentation, you can find topic on HTTP(S) Connectivity for Devices. If you read my previous post you should be familiar with HttpRequest. So your trigger DXL have to do is send a HTTP POST request to
<target server: org_id.internetofthings.ibmcloud.com>/api/v0002/device/types/{DeviceType}/devices/{DeviceID}/events/{eventID}

Looking at the URL format you can guess some event values will be set automatically:

{

"device_type": {DeviceType},

"device_id": {DeviceID},

"evt_type": {eventID},

"timestamp":

{

"$date": 1450194956425

}

All you need to add is a evt itself.

So your entire trigger DXL will look like

Module m = current Module
Buffer msg = create
msg = ""
msg += "\"id\": \"" getResourceURL m"\", "
msg += "\"content\": \"" name m "\", "
msg += "\"group\": \"Open\""
msg += "}"

string ioturl = "https://org_id.internetofthings.ibmcloud.com/api/v0002/device/types/DeviceType/devices/DeviceID/events/trigger_update"

HttpHeader h = create
string auth = ""
toBase64_("use-token-auth:YOUR_TOKEN", auth)
auth = auth[0:length(auth) -2]
add(h, "Authorization", "Basic "auth)
add(h, "Content-Type", "application/json")

HttpBody b = create
setValue(b, msg)

HttpResponse resp = HttpRequest(HttpPost, ioturl, b, h)
delete b

if (!null resp && resp.isOk)
{
// no one really needs to see it
  print "got it " (!null resp ? resp.code "" : "null") "\n"
  delete resp
}

delete h
delete msg

That's all you need to do publish events in real-time to IoTF!

Conclusion

Again HTTP Request proved to be a very powerful and useful perm. Using it the initial stack was reduced to very simple form:

Final stack in DOORS to IoTF communication

DOORS 9 has a potential in it, you just need to know how to use it.

DOORS reporting to IoT

So far we had an example how to use DOORS to read IoT historian data. Now it's time to publish something back.

There's no problem sending some useless data, the problem is to make sense out if it. So let's think on some 'real World' usage.

"Would you like to know when your most favorite module was modified?" 

Why not!

This could look like this:

If you read my previous posts you know we have a Tracked Resource Set (TRS) support in DOORS Web Access.
When enabled (as it's not by default) TRS "tells" us about all Modification, Creation and Deletion changes in DOORS Database.

So we need to modify a TRS reader to its new purpose.

I decided I won't need a front end for my TRS translation service so I rewrote the app.js:

/* jshint node:true */
var trs = require('./trs')
var superagent = require('superagent');
var iotf = require("ibmiotf");

//this is used to hide self-signed certificate errors
process.env.NODE_TLS_REJECT_UNAUTHORIZED = "0";

// IoTF setup
var deviceClientConfig = {
  org: 'quickstart',
  type: 'mytype',
  id: '001122334455',
  'auth-method':'token',
  'auth-token': 'secret'
};

var deviceClient = new iotf.IotfDevice(deviceClientConfig );
deviceClient.connect();
deviceClient.on("connect", function () {
 console.log("Connected to IoTF");
});

var user1 = null;
function updateTRS() {
 if (user1 == null) {
  user1 = superagent.agent();
  user1
    .post('https://DOORS_SERVER:8443/dwa/j_acegi_security_check')
    .type('form')
    .send({ j_username: 'DOORS_USER', j_password: 'PASSWORD'})
    .end(function(err, res) {
    })
 }
 else{
  trs.TRSUpdate(user1, deviceClient, function(err) {
    // just ping I'm alive
    console.log(". " +err); // one might want to check if err is defined
  });
 }
}

//query DWA for TRS changes
setInterval(updateTRS, 10*1000); //every 10 seconds
updateTRS()

Simple and quick. Now changes to trs.js which as you can see now takes additional deviceClient parameter.

Changes are really obvious:

1 . First publishing method

function publishTRS(deviceClient, data) {
   //publishing event using the user-defined quality of service
   var myQosLevel=2

   deviceClient.publish("trs_update","json",'{"d" : { "trs" : '+JSON.stringify(data)+' }}', myQosLevel);
   console.log("send data ->" + util.inspect(data));
}

2. Modified function parameters.
Note changes are no longer added to array. Instead those are send directly to IoTF

< function parseChangeLogs(clResults, client, last) {
---
> function parseChangeLogs(clResults, last) {
>     var changes = [];
97c88
< publishTRS(client, {start:dt, id:o.object, content:object, group:grp});
---
> changes.push({start:dt, id:o.object, content:object, group:grp});
108c99
< last(err);
---
> last(changes);
169c159
< exports.TRSUpdate = function(usr, client, last) {
---
> exports.TRSUpdate = function(usr, last) {
176c166
< parseChangeLogs(results, client, last);
---
> parseChangeLogs(results, last);

Now your TRS example is ready to talk to IoTF! Time to prepare IoTF application to use this feed.

Internet of Things Foundation

Go to you Bluemix dashboard and add new IoTF starter application. This will create for you a simple Node-Red application. Remember to add some credentials so only you will have access to your dashboard.

Once ready navigate to you IoTF dashboard


Here we will add a new device type to you IoT world.

New IoTF Device Type

Select Devices, Device Types and Create Type (on the bottom of the page)

The wizard will guide you to provide

1. Name and description
   
2. Define template
   
3. Define template
   

Save the type and proceed to add new Device

IoTF Device

Select Devices, Create Device (again on the bottom of the page)
You will have similar wizard. If you select Type defined earlier some fields will be populated.

1. Device ID
   
2. Security
   
3. Summary
   
   Please note credentials are NOT recoverable!

Finally Node-Red end

Once your device is ready, you can work on your Node Red consumer. 

Here I will just indicate how it might look like. I'm not going to write your business logic ;) I'm just showing an example what you can do.

So your full example would might look like:

Mine is just showing it works so you can try and do what you want.

I had to configure my IoT input:

Prepare my "business logic"

And observe the output if warning is set to true:

Conclusion

In those really simple steps we made a DOORS Web Access Tracked Resource Set a s Thing!

We just 'thingified' DOORS! 

IoT data in JavaScript

Introduction

In my previous post Displaying real-time IoT data in IBM DOORS, I displayed IoT data in the DOORS Web Access hover over. This involved an investigation into how I would access IBM Foundation IoT data from JavaScript. My initial thoughts were this would be very easy, I will just use the Paho Js utility, though it didn't end up as straight forward as anticipated. Therefore the aim of this post is to make the process a little easier and highlight the gotchas I encountered.

DOORS 9 IoT Report

Let's think about following user story:
"As a Requirement Engineer I would like to know how many errors my devices report"

In DOORS 9 module one can visualize this as:

So we would like to get a number of errors of given type reported by each of devices.

Device Error Reports in Sample App

First I need a device which can send some error codes. The IoT Starter Application mentioned in one of previous post publishes three kind of messages

• touchmove - user interaction with screen
• accel - orientation of the device and its position
• text - a text provided by user

I extended that application to be able to send error codes.

 

Pressing "Send Error" button and selecting error code sends a MQTT message to IoTF broker.

Typical error message looks like;

{

device_id: "doors9"

evt_type: "error"

timestamp:

{

$date: 1449059192155

}

evt: 

{

errorCode: "20"

}

}

For a compiled version of this application follow this link.
Now I could start updating my module.

Design for Analytics

Requirements in my DOORS module have a "ErrorCode" integer attribute which links a requirement to a error code reported by my device.

Additionally I'm using DOORS module level attributes to store values I do not need hardcoded in my DXL. Those are:

• Authorization (string)- so I do not need to calculate Base64 on each call
• DeviceType 
  (string) - which of my devices this module describes
• Organization (string) - my Bluemix organisation name

With all information in place I can write some simple layout DXL (which can be convert to attribute DXL later on to improve performance)

Layout DXL 

I want a specific type of event from all devices of a given type from my organization. So I need to use a historic query for a device type:

buf = "https://internetofthings.ibmcloud.com/api/v0001/historian/" org"/" deviceType "?evt_type=" eventId

This will return a JSON string with a list of events. If you do not have a JSON parser ready you can try to parse this data with Regular Expressions. Please remember this is a very simple example and in real world one shall not attempt to parse JSON with regular expressions.

My main worker code looks like:

if (!null obj) {
  int ival = obj."ErrorCode"
  if (ival == 0) halt
 
  string val = obj."ErrorCode"
  Module m = module obj
  string auth = m."Authorization"
  string dtype = m."DeviceType"
  string org = m."Organization"
 
  if (!null m && auth != "" && org != "" && dtype != "")
  {
    Buffer b = getData(auth, org, dtype, "error")

    if (!null b)  {
      string s = stringOf b
      Regexp re = regexp2 "\"device_id\":\"([^\"]*)\"[^}]+.[^}]+{\"errorCode\":\"([^\"]*)\""
      int i = 0
      string device = "", code =""
      //  temporary skip to hold names of devices which reported
      Skip erSkp = createString
      int allErrors = 0
      int numDevices = 0
      while (!null s && re s && i<100) {  // i is just a guard, we know there is no more then a 100 results in one page
        device = s[match 1]
        code = s[match 2]
        int ireported = 0
        // if code matches attribute value
        if (code == val)
        {
          allErrors++ // icrease number of errors
         
          if (!find(erSkp, device, ireported)) {
            put(erSkp, device, 1)
            numDevices++
          }
          else {
            ireported++
            put(erSkp, device, ireported, true)
          }
        }
        s = s[end 0 +1:]
        i++
      } // while
      // clean up
      delete b
     
      // report
      if (allErrors != 0) {
        for ireported in erSkp do {
          device = (string key erSkp)
          displayRich "Device with Id {\\b "device "} reported an issue " (ireported == 1 ? "once" : ireported" times")
        }
      }
     
      delete erSkp
    } // null b
  } // module setup
} // !null obj

You can find full DXL here.

Conclusion

Above layout DXL works fine when there're not so many devices. Once there will be more of them we no longer want to see how many times each device reported given issue. Thus the DXL could be rewritten to show something like:

As you saw in this example I'm using layout DXL but I think for a better understanding and feedback, one should consider writing a utility DXL.
Maybe that utility could provide its own UI for easier navigation?

In example above there's no need to send a HttpRequest for each object... It is enough to make one call per 100 (max page) events returned by query and write a little more complex Skip management. That however would require to make one top level Skip, but I'm sure you all know how to do it.