Categories
Geen categorie

Update

En toen was het ineens februari 2021. De tijd vliegt. Te snel om een persoonlijke blog bij te houden met periodieke mentale boeren. Ergo, ik schrijf nog steeds. Veel en vaak. Boeken en software. En doe nog veel meer dingen. Druk baasje van nature maar zonder een testosteron-kanon te zijn (denk ik dan). Anyway, er komt vanzelf wel weer een volgende post. Of niet. En anders is er altijd www.i2rs.nl nog, of Twitter. Ciao!

Categories
Geen categorie

Een visie op de toekomst van overheidssoftware

Zie bijgaande pdf (open_doorkijk_visiedocument_v1.1) voor mijn visie op de ontwikkeling van algemeen nut beogende software.

Categories
Geen categorie

My talk for Tecnoteca’s CMDBuild day 2016

You can see it here: https://www.youtube.com/watch?v=vY_LZQ_8udU

Categories
Geen categorie

Communicating with REST to CMDBuild

Intro

Hi. Today I like to talk about a nice open source tool called CMDBuild. CMDBuild is an excellent open source project that you can use at the heart of your IT structure. It offers you a web-interface that allows you to define objects and their relations. Once done you can use CMDbuild to enter information for these objects.

Example: you define an object “server” and and object “IP-address”. You define a 1-to-n relation from “server” to “IP-address”. Once done you can add servers to a list of server-objects and add IP-objects to server objects.

This is all fine and dandy but why would you like to do this? Well, after setting up the way you organize your infrastructure (eg: you could define on top of the server-objects “application”-objects to which servers belong, or locations where servers are located, etc.) you can use this information to hook up other systems.

As an example: knowing the IP-address of a server and it’s application you can instruct the monitoring application Zabbix to apply a certain template to automatically setup monitoring for that server.

Now the next challenge is how to get information out of CMDBuild. For this the application offers a SOAP and a REST interface.

For IT-newbies: This is the preferred way to communicate with an application. Yes, you can write a slick SQL Select statement to get the info out of the database but who guarantees you that after the next update the database model isn’t seriously altered?

There is only one challenge: there is a lot of documentation for the application (user manual, administration manual, web services manual) and once you get it working it’s all downhill from there. The catch is to get a small example running using only the current documentation.

Side note: allthough I have no ties to the CMDBuild organization I urge all readers who are using CMDBuild professionally to also buy themselves a support contract with the CMDBuild programmers firm Tecnoteca . Why? Well, if you place such a thing in the core of your infrastructure would you accept the risk that they go belly up because everybody uses the software thinking it’s a free ride? No. For a small amount of money you buy yourself peace of mind that you’ve done the right thing. And the fact that it provides you with excellent support is an added bonus.

Techno-speak

After a lot of trial and error on my part I got an example curl script from Tecnoteca that succesfully talks to the REST interface. It looks like this:

curl -v -X POST 'http://yourcmdbuildurl.nl/cmdbuild/services/rest/v1/sessions' -H "Content-Type: application/json" --data-binary '{"username": "admin", "password": "yourpassword"}' | json_pp

After checking that this script works and it gives you some meaningfull json output I got back to work.

When using the REST interface of CMDBuild you have first have to authenticate to it and get a token that identifies your session in subsequent REST calls. So the authentication token is important to have. Without it, you’re stuck.

First challenge was to make the same functionality work in Python. In the end this is what I wrote:

#!/usr/bin/python

import requests, json, pprint
import os,sys
from pprint import pprint
from prettytable import PrettyTable
from requests.auth import HTTPDigestAuth
import logging


## The next lines enable debugging at httplib level (requests->urllib3->http.client)
## You will see the REQUEST, including HEADERS and DATA, and RESPONSE with HEADERS but without DATA.
## The only thing missing will be the response.body which is not logged.
#try:
#    import http.client as http_client
#except ImportError:
#    # Python 2
#    import httplib as http_client
#http_client.HTTPConnection.debuglevel = 1
#
## You must initialize logging, otherwise you'll not see debug output.
#logging.basicConfig()
#logging.getLogger().setLevel(logging.DEBUG)
#requests_log = logging.getLogger("requests.packages.urllib3")
#requests_log.setLevel(logging.DEBUG)
#requests_log.propagate = True

print "***************************************************************"
print "*** Login and get authentication token "
print "***************************************************************"

cmdbuild_url = "http://server:8080/cmdbuild/services/rest/v2/sessions/"
data = {'username': 'soap', 'password': 'secret'}
headers = {'Content-type': 'application/json', 'Accept': '*/*'}
r = requests.post(cmdbuild_url, data=json.dumps(data), headers=headers)

print r.json()
r1=r.json()
sessionid=r1["data"]["_id"]
print "***************************************************************"
print " Authentication token is : " + sessionid
print "***************************************************************"


print "***************************************************************"
print "*** Session info"
print "***************************************************************"
#curl -v --trace-ascii - -X GET "http://server:8080/cmdbuild/services/rest/v1/sessions/${id}" -H "Content-Type: application/json"

cmdbuild_url = "http://server:8080/cmdbuild/services/rest/v2/sessions/"+sessionid
headers = {'Content-type': 'application/json', 'Accept': '*/*', 'CMDBuild-Authorization': sessionid }
r = requests.get(cmdbuild_url, data=json.dumps(data), headers=headers)
print r.json()
pprint(r.json())

There is a lot more to say about this, but basically, when you made this work you can read the manual, get the WSDL, find the REST endpoints you can call and download the WADL that also lists the available endpoints for your version of the software.

Advanced stuff

Now that you’ve got your hands dirty on some easy REST stuff let’s look at something more complex. The CMDBuild system is filled with objects called “classes”. Let’s first get a list of classes:

print "***************************************************************"
print "*** Classes "
print "***************************************************************"
#curl -v --trace-ascii - -X GET 'http://server:8080/cmdbuild/services/rest/v1/classes' -H "Content-Type: application
/json" -H "CMDBuild-Authorization: ${id}"

cmdbuild_url = "http://server:8080/cmdbuild/services/rest/v2/classes"
headers = {'Content-type': 'application/json', 'Accept': '*/*', 'CMDBuild-Authorization': sessionid }
r = requests.get(cmdbuild_url, data=json.dumps(data), headers=headers)
print "There are " + str(r.json()["meta"]["total"]) + " results"
#print r.json()
pprint(r.json())

Now that we have our list of classes in the system we can loop through them and get the individual details of every class:

for value in r.json()["data"]:
 print "\nTrying to get cards for : " + value["_id"] 
 id=value["_id"]
 #for id in value["_id"]:
 #pprint(id)
 print "Getting id: " + id

 print "***************************************************************"
 print "*** Class '"+id+"'"
 print "***************************************************************"
 #Asset 
 cmdbuild_url = "http://server:8080/cmdbuild/services/rest/v2/classes/" + id 
 headers = {'Content-type': 'application/json', 'Accept': '*/*', 'CMDBuild-Authorization': sessionid }
 r = requests.get(cmdbuild_url, data=json.dumps(data), headers=headers)
 #print "There are " + str(r.json()["meta"]["total"]) + " results for class " + id + "?"
 #print r.json()
 pprint(r.json())

 print "***************************************************************"
 print "*** Class '"+id+"' attributes"
 print "***************************************************************"
 # GET .../classes/Asset/attributes 
 cmdbuild_url = "http://server:8080/cmdbuild/services/rest/v2/classes/" + id + "/attributes"
 headers = {'Content-type': 'application/json', 'Accept': '*/*', 'CMDBuild-Authorization': sessionid }
 r = requests.get(cmdbuild_url, data=json.dumps(data), headers=headers)
 print "There are " + str(r.json()["meta"]["total"]) + " results for class " + id + " attributes "
 #print r.json()
 pprint(r.json())

 print "***************************************************************"
 print "*** Class '"+id+"' cards"
 print "***************************************************************"
 #GET .../classes/Asset/cards
 cmdbuild_url = "http://server:8080/cmdbuild/services/rest/v2/classes/" + id + "/cards"
 headers = {'Content-type': 'application/json', 'Accept': '*/*', 'CMDBuild-Authorization': sessionid }
 r = requests.get(cmdbuild_url, data=json.dumps(data), headers=headers)
 print "There are " + str(r.json()["meta"]["total"]) + " results for class " + id + " cards "
 #print r.json()
 pprint(r.json())

 

Happy programming!

Update 1:CMDBuild builders Tecnoteca tweet about this posting:

Update 2: Follow-up testimonial publised on the website of the CMDBuild builders Tecnoteca: article

Update 3: I got permission from Deltares to publish the sourcecode to a small CMDBuild python library module.

Categories
English LibrePlan

Libreplan, State of the Union 2015

In Libreplan,  State of the Union 2015, I want to talk about what happened this year and what my plans are for the next.

C Jeroen BatenOne year as past since I took over as project leader for the open source LibrePlan project from the excellent people of Igalia. Of course that is not my complete involvement with the project. I started as the Dutch language translator some 3 years ago. This year a lot has happened and a lot didn’t happen.

I would like to share with you my adventures with the LibrePlan project. I would like to tell you what happened this year, what didn’t happen and what will happen in the, hopefully near, future.

What did happen?

  • I wrote “LibrePlan, the missing manual” in Q1 of 2014. So far over 75 copies were sold and this makes this my most successful self-publishing book to date!
  • I answered every email send to info@libreplan.com and helped people using the application and discussed their wishes with them. This amounts to 294 separate questions that I answered. 29 of these were if LibrePlan could be installed on Windows (yes, that is possible but you have to figure it out all by yourself).
  • I regularly cleaned the Q and A site where spammers keep trying to sell crap.
  • I discussed wishes with over 50 individual companies.
  • I traveled to France and did 5 training sessions there for one organisation. Great stuff!
  • I contacted several Java programmers to be able to response to customer requests should they occur.

What did not happen?

  • A new LibrePlan release.

You might ask yourself why? Now this is were the story becomes a little personal but I will tell you anyway. But if you are used to corporate marketing bla bla only I suggest you skip the next part and go straight to the “What will happen?” section as you are probably only interested in solid results. 🙂

I want to shjeroen-in-de-voortuinare with you the reasons why there has not been a release yet. One thing is that I lost my day job 1st of January and I have been busy trying to find a new one. Being 50 years of age in a time of economic slum turned out not to bode well for this. Not even the fact that I am a volunteer fire fighter in my community makes any difference :-). I am not in the habit of torching places of work so, combined with the relatively optimistic economic growth forecasts I expect to find a job soon.

20140610_104532Another reason for the absence of a new release is that I have a very lovely stepdaughter. She loves to take a computer apart just for fun! Upon closer inspection of the picture you might see that she sits in a wheelchair. She was born 17 years ago with spina bifida and recently underwent here 43th surgery. Needless to say there is an extremely high level of care taking involved. Did I mention that we have a total of 5 kids to care for, two of which with disabilities? If you can read Dutch we have a website were we inform all members of the family about health issues and progress. Last June it became clear that keeping here living at home was no longer a reasonable option. So from October onwards she started living somewhere else and she visits us regulary. These recent events have had a tremendous impact on all our lives. It will take some time before the deep level of fatigue will subside but both my wife and I are also very optimistic in this regard.

What will happen?

Now this is an excellent question! What will happen? And when? Well, I found out the hard way that communicating solid deadlines is a sure thing to make people unhappy. What I can do is share with you my plans. You may even call it a roadmap. Here it is:

  • Make a new release, most likely 1.4.1.
  • Set up a stable build environment, most likely on openSuSE Build Service, where I can easily build packages for several distributions at once.
  • Make a new release, most likely 1.5.0 where margins are introduced.
  • Merge the existing Risk/Issue log code and finish it.
  • Add email functionality to the program to inform users of changes.
  • Add a pipeline page to show, well, err, the pipeline of projects.
  • Add documents functionality to be able to add documents to a project.
  • Add a GOTIK report. I know, GOTIK is a Dutch abbreviation meaning Money, Organisation, Time, Information and Quality. So it will probably be a MOTIQ report.

As you can see there are enough plans. Will they all happen? I don’t know, but I am sure as hell gonna try! When will they happen? As soon as I can. That is my promise to you.

Kind regards,

Jeroen Baten

P.S. Normally I do not accept replies to posts because I think there are enough other places on the Internet where people can vent. But this time I am curious about your thoughts. So tell me.