Intro
Hi. Today I like to talk about a nice open source tool called CMDBuild. CMDBuild is an excellent open source project that you can use at the heart of your IT structure. It offers you a web-interface that allows you to define objects and their relations. Once done you can use CMDbuild to enter information for these objects.
Example: you define an object “server” and and object “IP-address”. You define a 1-to-n relation from “server” to “IP-address”. Once done you can add servers to a list of server-objects and add IP-objects to server objects.
This is all fine and dandy but why would you like to do this? Well, after setting up the way you organize your infrastructure (eg: you could define on top of the server-objects “application”-objects to which servers belong, or locations where servers are located, etc.) you can use this information to hook up other systems.
As an example: knowing the IP-address of a server and it’s application you can instruct the monitoring application Zabbix to apply a certain template to automatically setup monitoring for that server.
Now the next challenge is how to get information out of CMDBuild. For this the application offers a SOAP and a REST interface.
For IT-newbies: This is the preferred way to communicate with an application. Yes, you can write a slick SQL Select statement to get the info out of the database but who guarantees you that after the next update the database model isn’t seriously altered?
There is only one challenge: there is a lot of documentation for the application (user manual, administration manual, web services manual) and once you get it working it’s all downhill from there. The catch is to get a small example running using only the current documentation.
Side note: allthough I have no ties to the CMDBuild organization I urge all readers who are using CMDBuild professionally to also buy themselves a support contract with the CMDBuild programmers firm Tecnoteca . Why? Well, if you place such a thing in the core of your infrastructure would you accept the risk that they go belly up because everybody uses the software thinking it’s a free ride? No. For a small amount of money you buy yourself peace of mind that you’ve done the right thing. And the fact that it provides you with excellent support is an added bonus.
Techno-speak
After a lot of trial and error on my part I got an example curl script from Tecnoteca that succesfully talks to the REST interface. It looks like this:
curl -v -X POST 'http://yourcmdbuildurl.nl/cmdbuild/services/rest/v1/sessions' -H "Content-Type: application/json" --data-binary '{"username": "admin", "password": "yourpassword"}' | json_pp
After checking that this script works and it gives you some meaningfull json output I got back to work.
When using the REST interface of CMDBuild you have first have to authenticate to it and get a token that identifies your session in subsequent REST calls. So the authentication token is important to have. Without it, you’re stuck.
First challenge was to make the same functionality work in Python. In the end this is what I wrote:
#!/usr/bin/python
import requests, json, pprint
import os,sys
from pprint import pprint
from prettytable import PrettyTable
from requests.auth import HTTPDigestAuth
import logging
## The next lines enable debugging at httplib level (requests->urllib3->http.client)
## You will see the REQUEST, including HEADERS and DATA, and RESPONSE with HEADERS but without DATA.
## The only thing missing will be the response.body which is not logged.
#try:
# import http.client as http_client
#except ImportError:
# # Python 2
# import httplib as http_client
#http_client.HTTPConnection.debuglevel = 1
#
## You must initialize logging, otherwise you'll not see debug output.
#logging.basicConfig()
#logging.getLogger().setLevel(logging.DEBUG)
#requests_log = logging.getLogger("requests.packages.urllib3")
#requests_log.setLevel(logging.DEBUG)
#requests_log.propagate = True
print "***************************************************************"
print "*** Login and get authentication token "
print "***************************************************************"
cmdbuild_url = "http://server:8080/cmdbuild/services/rest/v2/sessions/"
data = {'username': 'soap', 'password': 'secret'}
headers = {'Content-type': 'application/json', 'Accept': '*/*'}
r = requests.post(cmdbuild_url, data=json.dumps(data), headers=headers)
print r.json()
r1=r.json()
sessionid=r1["data"]["_id"]
print "***************************************************************"
print " Authentication token is : " + sessionid
print "***************************************************************"
print "***************************************************************"
print "*** Session info"
print "***************************************************************"
#curl -v --trace-ascii - -X GET "http://server:8080/cmdbuild/services/rest/v1/sessions/${id}" -H "Content-Type: application/json"
cmdbuild_url = "http://server:8080/cmdbuild/services/rest/v2/sessions/"+sessionid
headers = {'Content-type': 'application/json', 'Accept': '*/*', 'CMDBuild-Authorization': sessionid }
r = requests.get(cmdbuild_url, data=json.dumps(data), headers=headers)
print r.json()
pprint(r.json())
There is a lot more to say about this, but basically, when you made this work you can read the manual, get the WSDL, find the REST endpoints you can call and download the WADL that also lists the available endpoints for your version of the software.
Advanced stuff
Now that you’ve got your hands dirty on some easy REST stuff let’s look at something more complex. The CMDBuild system is filled with objects called “classes”. Let’s first get a list of classes:
print "***************************************************************"
print "*** Classes "
print "***************************************************************"
#curl -v --trace-ascii - -X GET 'http://server:8080/cmdbuild/services/rest/v1/classes' -H "Content-Type: application
/json" -H "CMDBuild-Authorization: ${id}"
cmdbuild_url = "http://server:8080/cmdbuild/services/rest/v2/classes"
headers = {'Content-type': 'application/json', 'Accept': '*/*', 'CMDBuild-Authorization': sessionid }
r = requests.get(cmdbuild_url, data=json.dumps(data), headers=headers)
print "There are " + str(r.json()["meta"]["total"]) + " results"
#print r.json()
pprint(r.json())
Now that we have our list of classes in the system we can loop through them and get the individual details of every class:
for value in r.json()["data"]:
print "\nTrying to get cards for : " + value["_id"]
id=value["_id"]
#for id in value["_id"]:
#pprint(id)
print "Getting id: " + id
print "***************************************************************"
print "*** Class '"+id+"'"
print "***************************************************************"
#Asset
cmdbuild_url = "http://server:8080/cmdbuild/services/rest/v2/classes/" + id
headers = {'Content-type': 'application/json', 'Accept': '*/*', 'CMDBuild-Authorization': sessionid }
r = requests.get(cmdbuild_url, data=json.dumps(data), headers=headers)
#print "There are " + str(r.json()["meta"]["total"]) + " results for class " + id + "?"
#print r.json()
pprint(r.json())
print "***************************************************************"
print "*** Class '"+id+"' attributes"
print "***************************************************************"
# GET .../classes/Asset/attributes
cmdbuild_url = "http://server:8080/cmdbuild/services/rest/v2/classes/" + id + "/attributes"
headers = {'Content-type': 'application/json', 'Accept': '*/*', 'CMDBuild-Authorization': sessionid }
r = requests.get(cmdbuild_url, data=json.dumps(data), headers=headers)
print "There are " + str(r.json()["meta"]["total"]) + " results for class " + id + " attributes "
#print r.json()
pprint(r.json())
print "***************************************************************"
print "*** Class '"+id+"' cards"
print "***************************************************************"
#GET .../classes/Asset/cards
cmdbuild_url = "http://server:8080/cmdbuild/services/rest/v2/classes/" + id + "/cards"
headers = {'Content-type': 'application/json', 'Accept': '*/*', 'CMDBuild-Authorization': sessionid }
r = requests.get(cmdbuild_url, data=json.dumps(data), headers=headers)
print "There are " + str(r.json()["meta"]["total"]) + " results for class " + id + " cards "
#print r.json()
pprint(r.json())
Happy programming!
Update 1:CMDBuild builders Tecnoteca tweet about this posting:
Update 2: Follow-up testimonial publised on the website of the CMDBuild builders Tecnoteca: article
Update 3: I got permission from Deltares to publish the sourcecode to a small CMDBuild python library module.