Zie bijgaande pdf (open_doorkijk_visiedocument_v1.1) voor mijn visie op de ontwikkeling van algemeen nut beogende software.
Categorie: Geen categorie
You can see it here: https://www.youtube.com/watch?v=vY_LZQ_8udU
Categorieën
Communicating with REST to CMDBuild
Intro
Hi. Today I like to talk about a nice open source tool called CMDBuild. CMDBuild is an excellent open source project that you can use at the heart of your IT structure. It offers you a web-interface that allows you to define objects and their relations. Once done you can use CMDbuild to enter information for these objects.
Example: you define an object “server” and and object “IP-address”. You define a 1-to-n relation from “server” to “IP-address”. Once done you can add servers to a list of server-objects and add IP-objects to server objects.
This is all fine and dandy but why would you like to do this? Well, after setting up the way you organize your infrastructure (eg: you could define on top of the server-objects “application”-objects to which servers belong, or locations where servers are located, etc.) you can use this information to hook up other systems.
As an example: knowing the IP-address of a server and it’s application you can instruct the monitoring application Zabbix to apply a certain template to automatically setup monitoring for that server.
Now the next challenge is how to get information out of CMDBuild. For this the application offers a SOAP and a REST interface.
For IT-newbies: This is the preferred way to communicate with an application. Yes, you can write a slick SQL Select statement to get the info out of the database but who guarantees you that after the next update the database model isn’t seriously altered?
There is only one challenge: there is a lot of documentation for the application (user manual, administration manual, web services manual) and once you get it working it’s all downhill from there. The catch is to get a small example running using only the current documentation.
Side note: allthough I have no ties to the CMDBuild organization I urge all readers who are using CMDBuild professionally to also buy themselves a support contract with the CMDBuild programmers firm Tecnoteca . Why? Well, if you place such a thing in the core of your infrastructure would you accept the risk that they go belly up because everybody uses the software thinking it’s a free ride? No. For a small amount of money you buy yourself peace of mind that you’ve done the right thing. And the fact that it provides you with excellent support is an added bonus.
Techno-speak
After a lot of trial and error on my part I got an example curl script from Tecnoteca that succesfully talks to the REST interface. It looks like this:
curl -v -X POST 'http://yourcmdbuildurl.nl/cmdbuild/services/rest/v1/sessions' -H "Content-Type: application/json" --data-binary '{"username": "admin", "password": "yourpassword"}' | json_pp
After checking that this script works and it gives you some meaningfull json output I got back to work.
When using the REST interface of CMDBuild you have first have to authenticate to it and get a token that identifies your session in subsequent REST calls. So the authentication token is important to have. Without it, you’re stuck.
First challenge was to make the same functionality work in Python. In the end this is what I wrote:
#!/usr/bin/python
import requests, json, pprint
import os,sys
from pprint import pprint
from prettytable import PrettyTable
from requests.auth import HTTPDigestAuth
import logging
## The next lines enable debugging at httplib level (requests->urllib3->http.client)
## You will see the REQUEST, including HEADERS and DATA, and RESPONSE with HEADERS but without DATA.
## The only thing missing will be the response.body which is not logged.
#try:
# import http.client as http_client
#except ImportError:
# # Python 2
# import httplib as http_client
#http_client.HTTPConnection.debuglevel = 1
#
## You must initialize logging, otherwise you'll not see debug output.
#logging.basicConfig()
#logging.getLogger().setLevel(logging.DEBUG)
#requests_log = logging.getLogger("requests.packages.urllib3")
#requests_log.setLevel(logging.DEBUG)
#requests_log.propagate = True
print "***************************************************************"
print "*** Login and get authentication token "
print "***************************************************************"
cmdbuild_url = "http://server:8080/cmdbuild/services/rest/v2/sessions/"
data = {'username': 'soap', 'password': 'secret'}
headers = {'Content-type': 'application/json', 'Accept': '*/*'}
r = requests.post(cmdbuild_url, data=json.dumps(data), headers=headers)
print r.json()
r1=r.json()
sessionid=r1["data"]["_id"]
print "***************************************************************"
print " Authentication token is : " + sessionid
print "***************************************************************"
print "***************************************************************"
print "*** Session info"
print "***************************************************************"
#curl -v --trace-ascii - -X GET "http://server:8080/cmdbuild/services/rest/v1/sessions/${id}" -H "Content-Type: application/json"
cmdbuild_url = "http://server:8080/cmdbuild/services/rest/v2/sessions/"+sessionid
headers = {'Content-type': 'application/json', 'Accept': '*/*', 'CMDBuild-Authorization': sessionid }
r = requests.get(cmdbuild_url, data=json.dumps(data), headers=headers)
print r.json()
pprint(r.json())
There is a lot more to say about this, but basically, when you made this work you can read the manual, get the WSDL, find the REST endpoints you can call and download the WADL that also lists the available endpoints for your version of the software.
Advanced stuff
Now that you’ve got your hands dirty on some easy REST stuff let’s look at something more complex. The CMDBuild system is filled with objects called “classes”. Let’s first get a list of classes:
print "***************************************************************"
print "*** Classes "
print "***************************************************************"
#curl -v --trace-ascii - -X GET 'http://server:8080/cmdbuild/services/rest/v1/classes' -H "Content-Type: application
/json" -H "CMDBuild-Authorization: ${id}"
cmdbuild_url = "http://server:8080/cmdbuild/services/rest/v2/classes"
headers = {'Content-type': 'application/json', 'Accept': '*/*', 'CMDBuild-Authorization': sessionid }
r = requests.get(cmdbuild_url, data=json.dumps(data), headers=headers)
print "There are " + str(r.json()["meta"]["total"]) + " results"
#print r.json()
pprint(r.json())
Now that we have our list of classes in the system we can loop through them and get the individual details of every class:
for value in r.json()["data"]:
print "\nTrying to get cards for : " + value["_id"]
id=value["_id"]
#for id in value["_id"]:
#pprint(id)
print "Getting id: " + id
print "***************************************************************"
print "*** Class '"+id+"'"
print "***************************************************************"
#Asset
cmdbuild_url = "http://server:8080/cmdbuild/services/rest/v2/classes/" + id
headers = {'Content-type': 'application/json', 'Accept': '*/*', 'CMDBuild-Authorization': sessionid }
r = requests.get(cmdbuild_url, data=json.dumps(data), headers=headers)
#print "There are " + str(r.json()["meta"]["total"]) + " results for class " + id + "?"
#print r.json()
pprint(r.json())
print "***************************************************************"
print "*** Class '"+id+"' attributes"
print "***************************************************************"
# GET .../classes/Asset/attributes
cmdbuild_url = "http://server:8080/cmdbuild/services/rest/v2/classes/" + id + "/attributes"
headers = {'Content-type': 'application/json', 'Accept': '*/*', 'CMDBuild-Authorization': sessionid }
r = requests.get(cmdbuild_url, data=json.dumps(data), headers=headers)
print "There are " + str(r.json()["meta"]["total"]) + " results for class " + id + " attributes "
#print r.json()
pprint(r.json())
print "***************************************************************"
print "*** Class '"+id+"' cards"
print "***************************************************************"
#GET .../classes/Asset/cards
cmdbuild_url = "http://server:8080/cmdbuild/services/rest/v2/classes/" + id + "/cards"
headers = {'Content-type': 'application/json', 'Accept': '*/*', 'CMDBuild-Authorization': sessionid }
r = requests.get(cmdbuild_url, data=json.dumps(data), headers=headers)
print "There are " + str(r.json()["meta"]["total"]) + " results for class " + id + " cards "
#print r.json()
pprint(r.json())
Happy programming!
Update 1:CMDBuild builders Tecnoteca tweet about this posting:
Update 2: Follow-up testimonial publised on the website of the CMDBuild builders Tecnoteca: article
Update 3: I got permission from Deltares to publish the sourcecode to a small CMDBuild python library module.
So, I like Zabbix. Sue me 🙂
But upgrading is a whole different ballgame. Zabbix on Ubuntu 12.04 is version 1.8.1 and Zabbix on Ubuntu 14.04 is version 2.2.2. And Zabbix only does an automatic database upgrade from version 2.0 onwards. So, basically, you are on your own and Google does not seem to help either.
So, let me tell you what I did and maybe it works for you too. If not, I am sorry.
In the end I had to checkout the Zabbix sources to find a database patchfile.
So I did:
svn co svn://svn.zabbix.com/trunk
And I found the following directory: trunk/upgrades/dbpatches/2.0
And it has a “upgrade” script. Don’t bother, it does not work as is. But there is also a “patch” subdirectory with a shitload of sql scripts. They turn out to be just the ticket. So I listed them al in a file, put a mysql command in from of them and, for safety, a “read” between every line and run the script.
So, my logging output in /var/log/zabbix-server/zabbix_server.log changed from:
6033:20140701:133019.785 WEB monitoring: YES 6033:20140701:133019.785 VMware monitoring: YES 6033:20140701:133019.785 Jabber notifications: YES 6033:20140701:133019.785 Ez Texting notifications: YES 6033:20140701:133019.785 ODBC: YES 6033:20140701:133019.785 SSH2 support: YES 6033:20140701:133019.785 IPv6 support: YES 6033:20140701:133019.785 ****************************** 6033:20140701:133019.785 using configuration file: /etc/zabbix/zabbix_server.conf 6033:20140701:133019.789 Cannot upgrade database: the database must correspond to version 2.0 or later. Exiting ...
to:
9446:20140701:151423.638 Starting Zabbix Server. Zabbix 2.2.2 (revision 42525). 9446:20140701:151423.638 ****** Enabled features ****** 9446:20140701:151423.638 SNMP monitoring: YES 9446:20140701:151423.638 IPMI monitoring: YES 9446:20140701:151423.638 WEB monitoring: YES 9446:20140701:151423.638 VMware monitoring: YES 9446:20140701:151423.638 Jabber notifications: YES 9446:20140701:151423.638 Ez Texting notifications: YES 9446:20140701:151423.638 ODBC: YES 9446:20140701:151423.638 SSH2 support: YES 9446:20140701:151423.638 IPv6 support: YES 9446:20140701:151423.638 ****************************** 9446:20140701:151423.638 using configuration file: /etc/zabbix/zabbix_server.conf 9446:20140701:151423.797 current database version (mandatory/optional): 02010000/02010000 9446:20140701:151423.797 required mandatory version: 02020000 9446:20140701:151423.797 starting automatic database upgrade 9446:20140701:151423.831 completed 0% of database upgrade 9446:20140701:151423.863 completed 1% of database upgrade 9446:20140701:151423.955 completed 2% of database upgrade 9446:20140701:151424.055 completed 3% of database upgrade 9446:20140701:151424.371 completed 4% of database upgrade 9446:20140701:151424.916 completed 5% of database upgrade 9446:20140701:151425.232 completed 6% of database upgrade 9446:20140701:151425.701 completed 7% of database upgrade
etc.
So, before you begin, please make a backup of your database first!
The script I used in the end was this one:
mysql -h mysqlhost -p -D zabbix -pmysqlpassword < acknowledges.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < actions.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < alerts.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < applications.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < auditlog_details.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < auditlog.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < autoreg_host.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < conditions.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < config.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < dchecks.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < dhosts.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < drules.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < dservices.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < escalations.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < events.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < expressions.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < functions.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < globalmacro.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < globalvars.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < graph_discovery.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < graphs_items.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < graphs.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < graph_theme.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < groups.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < help_items.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < history_log.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < history.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < history_str.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < history_str_sync.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < history_sync.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < history_text.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < history_uint.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < history_uint_sync.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < host_inventory.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < hostmacro.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < hosts_groups.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < hosts_profiles_ext.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < hosts_profiles.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < hosts.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < hosts_templates.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < housekeeper.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < httpstepitem.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < httpstep.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < httptestitem.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < httptest.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < icon_mapping.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < icon_map.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < ids.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < images.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < interface.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < item_discovery.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < items_applications.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < items.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < maintenances_groups.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < maintenances_hosts.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < maintenances.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < maintenances_windows.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < mappings.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < media.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < media_type.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < node_cksum.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < nodes.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < opcommand_grp.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < opcommand_hst.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < opconditions.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < operations.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < opgroup.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < opmediatypes.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < opmessage_grp.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < opmessage.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < opmessage_usr.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < optemplate.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < profiles.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < proxy_autoreg_host.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < proxy_dhistory.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < proxy_history.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < regexps.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < rights.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < screens_items.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < screens.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < scripts.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < service_alarms.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < services_links.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < services.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < services_times.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < sessions.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < slideshows.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < slides.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < sysmap_element_url.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < sysmaps_elements.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < sysmaps_links.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < sysmaps_link_triggers.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < sysmaps.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < sysmap_url.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < timeperiods.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < trends.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < trends_uint.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < trigger_depends.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < trigger_discovery.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < triggers.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < user_history.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < users_groups.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < users.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < usrgrp.sql read dummy mysql -h mysqlhost -p -D zabbix -pmysqlpassword < valuemaps.sql
After the update you are (almost) ready to go. When you get the url to /zabbix working in Apache he hits you with a configurator with lot of questions because he wants a /etc/zabbix/zabbix.conf.php file. Simply answer the questions and install the config file and you are good to go.
Enjoy!
Categorieën
WordPress monitoring with Zabbix
In want to be able to see if WordPress needs to be updated on a specific webhost.
I have several WordPress sites and it would be nice to see which ones need to be updated.
First, I place a small file in the root directory of every WordPress instance that I call “zabbix-check.php”. Warning: choose your own filename! You don’t want hackers to scriptsearch for hackable WordPress instances!
Anyway, after this short disclaimer we continue with placing the following php code in said file:
<?PHP
# script to test remotely if wordpress update is needed
# tested on WordPress 3.6
# 1.0 J.Baten september 17, 2013
# License: GPLv2
ini_set('display_errors',1);
ini_set('display_startup_errors',1);
error_reporting(-1);
require('wp-load.php');
require('wp-admin/includes/update.php');
print "Currently: ".$wp_version;
$updates = get_core_updates();
if ( !isset($updates[0]->response) || 'latest' == $updates[0]->response ) {
echo ', no update needed';
} else {
echo ', update needed';
}
?>
All I need to do now is add a scenario to Zabbix that checks for the text “no update needed”. It also shows the version number if you like.
I hope this script is usefull to you.