Monday 14 September 2015

TRS in DOORS Next Generation

DOORS Next Generation has two TRS specifications implemented TRS and TRS 2.
With very small changes to the my previous posts you can read TRS1 feed from DNG.

Well if you remember my first post you will notice that my dwaOauth.js works just fine with DNG. So that little module gives you not only access to DWA protected resources but you can reuse it with DNG or other CLM servers.
Remember there were small changes done in order to make it working with most recent Node.js. You can read about those changes in my OSLC TRS listener.

DNG setup

You will need to prepare your DNG to provide TRS. Have a look at Register applications as TRS providers for the Lifecycle Query Engine in IBM Knowledge Center.

Usually DNG is ready to provide TRS feed, all you need is to assign a license to user who would be used to read it.

Also you will need to check permissions on /lqe/web/admin/permissions

Now yow are ready to rad TRS from DOORS Next Generation.

Changes in trs.js

There are very little changes in function getChangelogFromDWA
> var previous = '/rm/trs';
var previous = '/dwa/rm/trs/changelogs'

and
> if (previous == '/rm/trs') {
if (previous == '/dwa/rm/trs/changelog') {

So we just changes the starting point for triple feed.

You probably will notice the difference in TRS changelog id in DNG feed. It ses uuid and it looks something like:
urn:uuid:cdaf9dae-ad2d-4434-87a7-aea2488182d6

Thus we need to modify parseChangeLogs function parser:

async.whilst(
  function () { return rest != trs.nil && order > iLastOrder},
  // get ordered list or changes
  function (next) {
      var o = store.find(rest, trs.first, null)[0];
      if (typeof o !='undefined') {
        
        var timearr = o.object.split(':');
        if (timearr[0] == 'urn') {
            // update order
            order = parseInt(n3.Util.getLiteralValue( store.find(o.object, trs.order, null)[0].object) );
            idx++;
            //
            //  DO SOMETHING USEFUL HERE
            //
        }
        
          rest = store.find(rest, trs.rest, null)[0].object;
      } else {
          rest = trs.nil;
      }
      next();
  },
  function (err) {
    console.log("read " + idx + " changelog items");
      last(changes);
  });

DWA has change id has a useful format so one doesn't need further requests to gets some basic details of the change occurred but it is easy enough to write some callback functions getting changes from DNG. Of course one should not relay on format of the id as it is not standardized, defined nor promised not to change. 

Conclusion

Well TRS is TRS ;) with little changes to my inital TRS posts you can consume DWA, DNG or even Bugzilla TRS feed. In near future I will explain how to modify this parser for TRS 2.0.
Stay tuned!

Tuesday 8 September 2015

TRS sample usage

If you read my last two posts you should be able to parse DWA TRS data.

Here I want to show two possible user cases using TRS data.

Below examples are using visjs.org to display graphs.

Change "Counter"

So simply show the amount of each type of changes per minute (day, week)

 

Data input

Each point has following format:
{
  "x": dt, //date and time of the event
  "group":grp, // one of [module,project,object][Creation,Deletion,Modification]
  "y": i // number of group operations per time interval (minute)
}

Browser code

Very simple Graph2d initialization and usage.
    <script>
        var groups = new vis.DataSet([
            {id:'objectModification', content:'Objects Modified', value:0},
            /* ....  */
            ]);
        var items = new vis.DataSet([]);
        var container = document.getElementById('visualization');
        var timeline = new vis.Graph2d(container);
     
        timeline.setItems(items);
        timeline.setGroups(groups);
        timeline.setOptions({
            drawPoints: {style:'circle'},
            interpolation: false,
            defaultGroup: 'ungrouped',
            legend: true
          });
 
        var socket = io();
        socket.on('trsevent', function(msg){
            items.add(msg);
        });
    </script>

Server code

Below code goes trough changes from latest to last parsed and calculates data for each minute which is present TRS feed.

Parser assumes following format of the change id:
urn:rational::1-dbid-[M|P|O-\d+]-00000525:hostname:port:YYYY-MM-DD:HH:mm:ss.nnn:xx
and it is not parsing other ids (like users:rational::...)

trs.js updated parser:
function parseChangeLogs(clResults, last) {
  var store = clResults[1];
  var lastID = clResults[0] || '';
  
  // check for store
  if (typeof store !== 'undefined') {
    var changesHead = store.find(store.find(null,
                        trs.changes,
                        null)[0].object,
                   trs.first,
                   null)[0];
    if (typeof changesHead !== 'undefined') {
      if (changesHead.object != clResults[0]) {
        
        // remember head
        cache.putCache('trs_lastChangeLog', changesHead.object);
        
        // try to get last trs.order based on lastId
        var lastOrder = (lastID != '') ? store.find(lastID, trs.order, null)[0].object : '"0"^^http://www.w3.org/2001/XMLSchema#integer';
        var iLastOrder = parseInt(n3.Util.getLiteralValue(lastOrder));

        // process changes from the newest to last parsed (or last in list)
        var rest = changesHead.subject;
        var changes = [];

        var order = Number.MAX_VALUE;
        async.whilst(
            function () { return rest != trs.nil && order > iLastOrder},
            // get ordered list or changes
            function (next) {
              var o = store.find(rest, trs.first, null)[0];
              if (typeof o !='undefined') {
                
                // o [_b:xx, trs.first, urn:rational::1-dbid-M-00000525:hostname:8443:2015-09-03:14:45:58.971:39]
                var timearr = o.object.split(':');
                if (timearr[0] == 'urn') {
                  var d = timearr[6].split('-');
                  var dt = new Date(d[0], d[1]-1, d[2], timearr[7], timearr[8], timearr[9]);
                  dt.setSeconds(0);
                  
                  // update order
                  order = parseInt(n3.Util.getLiteralValue( store.find(o.object, trs.order, null)[0].object) );
                  
                  // timearr[3] |= 1-dbid-M-00000525
                  //        |= 1-dbid-O-9-00000525    
                  var tmp = timearr[3].split('-');
                  var target = (tmp[2] == "M") ? 'module' : ((tmp[2] == "P") ? 'project' : 'object');
                  
                  if (target != '') {
                    // type -> group
                    var type = store.find(o.object, trs.type, null)[0].object;
                    var grp = target + type.split('#')[1];
                    
                    // push
                    var bDone = false;
                    for (var idx in changes) {
                      if (changes[idx].x.getTime() == dt.getTime() &&
                        changes[idx].x.getDate() == dt.getDate() && 
                        changes[idx].group == grp)
                      {
                        changes[idx].y++;
                        console.log('=>'+grp);
                        bDone = true;
                      }
                    }
                    if (!bDone) {
                      changes.push({x:dt, group:grp, y:1});
                    }
                  }
                }
                
                rest = store.find(rest, trs.rest, null)[0].object;
              } else {
                rest = trs.nil;
              }
              next();
            },
            function (err) {
              last(changes);
            });
      }
    }
  }
}

As you can see all data processing is done on server side, browser just displays those points. Same approach is used in second example

What, when, how

A timeline graph showing when item (project, module or object) was either created, modified or deleted:


Above is an iframe so you could try Timeline. Data behind this iframe is a static data used to show possible usage.

Browser code

index.ejs is similar to previous one but this time is using a Timeline graph:
<script>
        var groups = new vis.DataSet([
            {id:'Modification', value:0},
            {id:'Creation', value:1},
            {id:'Deletion', value:2}
            ]);
        var items = new vis.DataSet([]);
        var container = document.getElementById('visualization');
        var timeline = new vis.Timeline(container);

        timeline.setItems(items);
        timeline.setGroups(groups);
        timeline.setOptions({
            groupOrder: function (a, b) {
              return a.value - b.value;
            },
            editable: false,
            type: 'point'
          });

        var socket = io();
        socket.on('trsevent', function(msg){
            items.add(msg);
        });
</script>

Server code

Below is just a working part of parser:
async.whilst(
  function () { return rest != trs.nil && order > iLastOrder},
  // get ordered list or changes
  function (next) {
      var o = store.find(rest, trs.first, null)[0];
      if (typeof o !='undefined') {
        
        // o [_b:xx, trs.first, urn:rational::1-dbid-M-00000525:hostname:8443:2015-09-03:14:45:58.971:39]
        var timearr = o.object.split(':');
        if (timearr[0] == 'urn') {
          var d = timearr[6].split('-');
          var dt = new Date(d[0], d[1]-1, d[2], timearr[7], timearr[8], timearr[9]);
          
          // update order
          order = parseInt(n3.Util.getLiteralValue( store.find(o.object, trs.order, null)[0].object) );
          
          // timearr[3] |= 1-dbid-M-00000525
          //         |= 1-dbid-O-9-00000525    
          var tmp = timearr[3].split('-');
          var object = '';
          
          if (tmp[2] == "M") {
            var obj = store.find(o.object, trs.changed, null)[0].object;
            if (obj.indexOf('view') == -1) {
              object += 'Module ' + tmp[tmp.length-1];
            }
          } else if (tmp[2] == "P") {
            // Object
            object = 'Project ' + tmp[tmp.length-1];
          } else {
            // Object
            object = 'Module ' + tmp[tmp.length-1] + " object " + tmp[3];
          }
          
          if (object != '') {
            // type -> group
            var type = store.find(o.object, trs.type, null)[0].object;
            var grp = type.split('#')[1];
            
            // push
                  changes.push({start:dt, id:o.object, content:object, group:grp});
          }
        }
        
          rest = store.find(rest, trs.rest, null)[0].object;
      } else {
          rest = trs.nil;
      }
      next();
  },
  function (err) {
      last(changes);
  }
);

Same assumptions for change id.

Data passed to Timeline looks like:
{
  "start": time of event,
  "id": id of event (urn:rational...)
  "content": what to show
  "group": modification, deletion or creation
}

Conclusions

TRS gives us ordered list of changes to the resources in working set. Those are ordered from newest to the latest change. One can simply show them as a list of changes, to generate some kind of report, or a graph.

One might want to request some additional information using DXL services and show those changes in browser (see below), store them, etc. Possibilities are endless.

Showing a change made to the object with aid of DXL service and Timeline item templates: