There are basically two different types of Digi XBee modules available, the series 1 and the series 2 (which includes 2.5). From what I read the series 1′s are much easier to work with, however I bought the latest series 2.5 ZB version not the series 1. I thought I’d share how I managed to get these little modules talking to each other.

The Zigbee protocol requires at least one unit to act as a network Coordinator but out-of-the-box they both came configured as Zigbee Endpoints. It took a bit of head scratching to figure out how to make one Xbee a Coordinator. Turns out you have to download some software from Digi called X-CTU and use it to upload a suitable Coordinator firmware to one of the units. The X-CTU web page says that 64-bit operating systems are not supported but it works fine on my Windows 7 64-bit laptop. X-CTU is actually a really useful piece of software and makes it very easy to work with the XBees.

After I uploaded a new Coordinator firmware to one of the modules they just worked. I currently have the Coordinator hooked up to a Windows PC using the Sparkfun USB Explorer and the endpoint sitting in a (awesome) Ciseco Xbee shield on the Arduino. I have a small program running on the Arduino that continually generates messages and prints them to the serial port. The messages magically appear in X-CTU on the PC.

My plan now is to build a Zigbee/WiFi bridge using an Arduino, an XBee and a WiFi shield. The bridge will connect to the MQTT broker and publish received data to different topics depending on the XBee sending the message. iDoorbell v2 will use just an XBee (no Arduino required) to send a message through the bridge onto the MQTT broker where a small script will subscribe to the appropriate topic and react by sending a “there’s someone at the door” message through Prowl. Very cool!

http://bravo.local/homework/home

Where homework is the sub URI configured to serve my application and home is the name of one of the controllers. The Apache/Passenger configuration for this set up is documented here.

This approach has worked fine until I used the Paperclip gem in my latest Rails app. I discovered that the attachment links that were being generated started with a slash but did not include the application’s sub URI. To work around this issue I needed some way to append the relevant sub uri to the Paperclip URI. I hate hard coding stuff like this and I wanted something that would work in different environments. I added a

<%= debug request.inspect %>

to one of my views and found that there is a method request#script_name that returns the complete protocol, port, host name and the sub URI. Problem solved! Not quite. I like to keep my views nice and tidy so I construct that URL in a helper. In fact, after watching Ryan Bates’ Rails Casts, I have become a big fan of the decorator gem Draper. Thanks to an email from Jeff Casimir, the author of Draper, I learned that I could access the request object directly from the Draper helper method. Here’s an example:

  def link
    # doc.doc is the Paperclip object
    url = h.request.script_name + doc.doc.url 
    h.link_to doc.description, url
  end

Today I stumbled upon a podcast interview on MQ Telemetry Transport, or MQTT, with Andy Piper from IBM. MQTT is an M2M/Internet of things connectivity protocol developed at IBM. Much of the podcast talks about how this technology is being used in home DIY sensor projects. MQTT is based on a “Publish and Subscribe” architecture and requires a Broker to handle the messaging. Although IBM have a few different MQTT Broker implementations, there is an open source implementation of the broker server called Mosquitto  that looks worth experimenting with.

If that wasn’t enough to wet your appetite, Andy Stanford-Clark (also from IBM) gave a presentation called “The house that Twitters” which explains how he has used MQTT to create a number of interesting home sensor projects.

I have started a project using my Arduino that notifies my iPhone whenever a switch in a separate circuit is closed. The plan is to connect this to our home doorbell so as well as the usual ding-dong we also get an iPhone alert. Software-wise this works great using a Wifly shield to enable the Arduino to talk the the Prowl notification service on the Internet over http. It's kind of a wifi version of the project described here http://blog.makezine.com/archive/2010/12/snail-mail-push-alerts.html.

Now the problem I have is how to properly connect the Arduino to the doorbell circuit. The doorbell circuit is powered by a transformer that deliver 5V AC at 1mA. Note AC not DC. I tried measuring the voltage across the transformer using my meter but it read 13.8 so I'm not sure why that is. I want to detect a connection in the AC circuit and notify an Arduino digital input pin. I have been trying to use a H11AA1X optocoupler transistor to keep the two circuits separate. The H11AA1X is designed for AC input as it contains two leds, one for each AC current direction. I connected the H11AA1X in series with the doorbell circuit along with a small 470ohm resistor (to protect the leds in the optocoupler). I also have a couple of leds in the circuit so that I can see when something happens. I have tested the prototype away from the doorbell circuit using a separate (admittedly DC) power supply and simulated the doorbell switch and it works fine. But when I connect it to the doorbell circuit I can not get it to respond to the doorbell switch and the bell itself does not ding-dong.

I now have a working prototype, although I am still a bit worried about the current the H11AA1X is handling.I realised that my original "test" led on the AC side of the circuit was probably a bad idea and removed that. I created a simple circuit with just the H11AA1X and the smallest resistor I had in series with the doorbell circuit. I put 5v across the collector and emitter with an led and a resistor across the DC side of the H11AA1X. When I pressed the doorbell the DC led lit up but the dinger did not dong. I obviously had too much resistance on the AC side to trigger the solenoid. It occurred to me that the bell solenoid is already providing sufficient resistance and that I could probably run the H11AA1X without an additional resistor. So I tried that and it worked. My test meter can't do AC current so I have no idea what current is actually flowing through the H11AA1X. I did find a web site that described the transformer as 1amp, so fingers crossed it will be OK. Here's my working circuit diagram.

I also got it to work by building a different circuit with  the H11AA1X and a resistor in parallel with the bell solenoid. Unfortunately that would entail extra wires and the Arduino to be installed in our hall way which would not pass the wife-acceptance-test. The bell transformer is in our garage so  well out of sight but it does only leave the possibility of connecting my iDoorbell Arduino in series.

Inspiration came from this excellent blog post: How to compile your own Nginx and Passenger

Based on that blog post I just downloaded the source from nginx then wrote myself this little shell script.
— build.sh —#!/bin/bash
# Load RVM Functions into shell
[[ -s "/usr/local/lib/rvm" ]] && . "/usr/local/lib/rvm"
[[ -r $rvm_path/scripts/completion ]] && . $rvm_path/scripts/completion
#
rvm use ree
rvm current
#
echo passenger gem is installed in:
passenger-config --root
echo
export PASSENGER_NGINX_DIR=`passenger-config --root`/ext/nginx
echo PASSENGER_NGINX_DIR=$PASSENGER_NGINX_DIR
#
NGINX_SRC_DIR=`pwd`/nginx-1.0.8/
echo Building $NGINX_SRC_DIR
#
cd $NGINX_SRC_DIR
./configure \
--prefix=/usr/local \
--sbin-path=/usr/sbin \
--conf-path=/etc/nginx/nginx.conf \
--error-log-path=/var/log/nginx/error.log \
--http-log-path=/var/log/nginx/access.log \
--with-http_ssl_module \
--with-http_gzip_static_module \
--add-module=$PASSENGER_NGINX_DIR
make
make install
#
echo All done. Just edit /etc/nginx/nginx.conf to ensure it is
echo picking up correct version of Passenger


Next time all I need to do is edit this script.

I set up my semantic model in Oracle as per the manual, having previously used SEM_APIS.CREATE_SEM_NETWORK to add RDF support to my database.

SQL> create table address_book_rdf_data(id number, triple sdo_rdf_triple_s);

SQL> execute sem_apis.create_sem_model('address_book', 'address_book_rdf_data', 'triple');

Next I tried to load model I created earlier into Oracle using the Oracle Java Bulk Loader. First problem was the Oracle RDF bulk loader only supports data files in N-Triple format. I created my model in Notation-3 (N3) format as I find it the easiest format to actually read. A quick google brought up this useful site for converting between RDF formats.

http://www.mindswap.org/2002/rdfconvert/

I created a small script to invoke the bulk loader.

cp=${ORACLE_HOME}/md/jlib/sdordf.jar:${ORACLE_HOME}/jdbc/lib/ojdbc5.jar
db='-Ddb.user=ab -Ddb.password=ab -Ddb.host=localhost -Ddb.port=1521 -Ddb.sid=orcl'
java $db -classpath $cp oracle.spatial.rdf.client.BatchLoader contacts.nt ADDRESS_BOOK_RDF_DATA RDF_USERS address_book

I found that if the loader failed for any reason, such as invalid username, it leaves behind a temporary table that prevents it running subsequently. To clear the temporary table you have to log in as a priviledged user (system in my case) and run this piece of SQL.

SQL> drop table mdsys.RDF_LINK$_TEMPBM$;

Eventually I got the bulk loader to partially run before it stopped with the following error:

$ sh load.sh
Note: You can use JVM properties -Ddb.user= -Ddb.password=
-Ddb.host= -Ddb.sid= -Ddb.port=
to configure database connection.
JVM property -Dappend=true can be used to specify APPEND mode. By default, this option is false.
JVM property -Dexchange=true can be used to allow table partition exchange which is faster and less space consuming.
This option is less robust though. By default, it is false

Connecting to jdbc:oracle:thin:@localhost:1521:orcl as user ab
Insert mode
Load triples into one column and populate another ID column
Batch loading started...
Subject: <http://freakent.co.uk/addressbook/phone-numbers/447xxxxxxxx0>
Property: <http://freakent.co.uk/addressbook/phone-numbers/number>
Object: "+447xxxxxxxx0"

Loading up to rows = 69
java.sql.SQLException: ORA-55315: batch load attempt failed: SQLERRM=ORA-14098: index mismatch for tables in ALTER TABLE EXCHANGE PARTITION ORA-06512: at “MDSYS.SDO_RDF_INTERNAL”, line 5929
ORA-06512: at “MDSYS.MD”, line 1723
ORA-06512: at “MDSYS.MDERR”, line 17
ORA-06512: at “MDSYS.SDO_RDF_INTERNAL”, line 5950
ORA-06512: at “MDSYS.SDO_RDF_INTERNAL”, line 5960
ORA-06512: at “MDSYS.RDF_APIS”, line 1283
ORA-06512: at line 1

at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:439)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:395)
at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:802)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:436)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:186)
at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:521)
at oracle.jdbc.driver.T4CCallableStatement.doOall8(T4CCallableStatement.java:202)
at oracle.jdbc.driver.T4CCallableStatement.executeForRows(T4CCallableStatement.java:1005)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1307)
at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3449)
at oracle.jdbc.driver.OraclePreparedStatement.execute(OraclePreparedStatement.java:3550)
at oracle.jdbc.driver.OracleCallableStatement.execute(OracleCallableStatement.java:4693)
at oracle.jdbc.driver.OraclePreparedStatementWrapper.execute(OraclePreparedStatementWrapper.java:1086)
at oracle.spatial.rdf.client.NTripleConverter.loadNTriple(NTripleConverter.java:648)
at oracle.spatial.rdf.client.BatchLoader.main(BatchLoader.java:302)

I have put a question on the OTN discussion forum, if I get a response I’ll update this post.

Update:  This is the response I received.

Hi, That loader (oracle.spatial.rdf.client.NTripleConverter) is a bit old. Please use the loader provided in Jena Adapter. You can refer to Example 7-3 of the following document. You may want to use addInBulk. http://download.oracle.com/docs/cd/E18283_01/appdev.112/e11828/sem_jena.htm#CBBCIJAD
Thanks, Zhe

So I followed that advice and it worked a treat!

One more thing…

Whilst doing this work with the Oracle database, I discovered a nice little trick in SQLPlus on Linux. On Windows it is very nice to be able to use your keyboard arrow keys to go back through your SQLPlus command history and make edits. On Linux this does not work by default. I found a few blogs that suggested using rlwrap to invoke sqlplus. I quickly installed rlwrap using yum and set up an alias in my .bashrc, it works perfectly.

alias sqlplus='rlwrap sqlplus'


My operating system of choice for this project was CentOS 5 (32 bit), running as a VirtualBox guest. CentOS is a derivative of Redhat and is the only truly open Enterprise Linux distribution. I manage my Ruby environment using RVM, using it to install both Ruby 1.9.2 and JRuby. I also installed OpenJDK 1.6 for my Java runtime environment.

To get started I took a clone of my application from my Git repository and made sure it ran against the SQLite locally using Ruby 1.9.2 and WEBrick. The only issue I had was getting SQLite 3.6.2 (which is the version required by the sqlite3-ruby gem) installed in CentOS. I temporarily added the ATrpms repository to my Yum config (see below) to get it in the end.

Set up repo file /etc/yum.repos.d/ATrpms.repo

[atrpms]
name=Fedora Core $releasever - $basearch - ATrpms
baseurl=http://dl.atrpms.net/el$releasever-$basearch/atrpms/testing
#baseurl=http://dl.atrpms.net/el$releasever-$basearch/atrpms/stable
gpgkey=http://ATrpms.net/RPM-GPG-KEY.atrpms
gpgcheck=1


$ sudo yum update


Next I ran the same application configuration under JRuby, just by using RVM to switch Ruby environments. I had to add the appropriate JDBC SQLite adapter gem to my bundler Gemfile. I discovered a nifty bundler trick using the “platforms” statement which let me configure gems based on my Ruby runtime environment.

Set up Gemfile In Rails app root directory

platforms :jruby do
gem 'jruby-openssl'
gem 'ffi'
gem 'ffi-ncurses'
gem 'activerecord-jdbcsqlite3-adapter', :group => [:development, :test]
end

Now that I had the app running under JRuby and was confident that my JRuby environment was properly configured, it was time to get it running against the Oracle database. My CentOS server has a regular install of Oracle 11g Enterprise Edition. You can download it from http://otn.oracle.com for evaluation purposes. You could also use the XE edition if you want the slightly less featured free version of Oracle. Either way, just download and follow the install instructions provided. The 11g R2 Enterprise Edition install was one of the simplest Oracle installs I have ever experienced.

With the database installed, I created a new Rails database environment in my application’s config/database.yml file. This way I could switch database environments just by changing the RAILS_ENV environment variable.

oradev:
adapter: jdbc
driver: oracle.jdbc.OracleDriver
url: jdbc:oracle:thin:@localhost:1521:orcl
username: expenses
password: XXXXXXXXXXXX


After creating the necessary database user and running the migrations I was able to run my application against the Oracle database under JRuby. The only issue I discovered was my use of the MySQL/SQLite keyword IFNULL in one piece of SQL. I was planning to create a simple IFNULL database function under Oracle (using NVL) to emulate the the MySQL/SQlite function (a wise man once told me it’s easy to make a more powerful system look like a less powerful one), but a little bit of Googling turned up the COALESCE function. I’ve been using Oracle for over 20 years but had never seen this. I was able to do a straight switch of the IFNULL function with COALESCE. Aren’t standards great.

Final step was to get the application running under the Oracle Weblogic Java EE server. I chose the Weblogic Development distribution to download and install from OTN. To install you basically unpack a zip file in the correct directory and run a few configuration scripts. All the info required was in the Weblogic README file. Once the Weblogic server was installed, I created a new Weblogic domain for my JRuby application using $WEBLOGIC_HOME /wlserver/common/bin/config.sh. Just fill out the form to create a new domain. Choose a suitable folder and name for your domain (I called mine “jrubydomain”) and be careful to get the right path to your Java environment. Once the domain is created you can start the server using the script automatically created in your domain’s root directory:

$ ./startWebLogic.sh

You can connect to the domain’s administration interface using your web browser and the URL http://localhost:7001/console (assuming you didn’t change any of the defaults). Use the username and password you defined during domain creation to log in.

To turn my Rails app into a Java EE app I used the excellent Gem, Warbler. Warbler is a set of Rake tasks that create a Java EE WAR file of your Rails app that can be deployed to a standard Java EE server. From your application’s root directory you type warble and a few seconds later you have a complete WAR file containing your app and all the dependent Gems. To deploy the WAR file you can go into the Weblogic domain administration interface and click the deployments link in the menu bar on the left. Then click install and follow the steps to upload your warbler created WAR file. Pay attention to the Context Root setting as this is needed to run your application, mine is set to /Expenses. To run and test my application I just pointed another browser window at http://localhost:7001/Expenses.

At this point I received the famous “Rails Application Failed To Start” page. Looking at the Weblogic server output I could see a problem with some class dependencies involving the Joda date/time library. I have to give credit and thanks to agardiner for working out the solution to this problem. (Full description here https://github.com/nicksieger/jruby-rack/issues/24)

“The issue occurs because WebLogic 10.3 includes an old version of Joda Time (version 1.2.1), which predates the addition of the withYear method on DateTime. This version is referenced in the manifest of the weblogic.jar file, and thus is higher in the classloader hierarchy than anything your web app references or includes, including JRuby.”

Fortunately he also proposed a simple solution that worked perfectly. Using a WEB-INF/weblogic.xml file it is possible to configure Weblogic to “give higher priority to any Java classes found in your WAR file, thereby ensuring that JRuby gets to use the bundled version of Joda Time.”

I created the following weblogic.xml file in my Rails application’s config directory.
<weblogic-web-app xmlns="http://www.bea.com/ns/weblogic/weblogic-web-app">
<container-descriptor>
<!--
The following entry is necessary to prevent Weblogics old version of the Joda time JAR (1.2.1) from taking precendence
over the much more recent version (1.6.0) included in JRuby.
Without this setting, Rails 3 will fail to start, due to the
missing withYear method on org.joda.time.DateTime.
-->
<prefer-web-inf-classes>true</prefer-web-inf-classes>
</container-descriptor>
</weblogic-web-app>

To have Warbler include this weblogic.xml file correctly in your WAR file you need to create a warbler.rb config file in your application’s config directory. The easiest way to do this is let Warbler create a default file for you by typing:
$ warble config

Edit the warble.rb and add the following line:
...
# Files for WEB-INF directory (next to web.xml). This contains
# web.xml by default. If there is an .erb-File it will be processed
# with webxml-config. You may want to exclude this file via
# config.excludes.
# config.webinf_files += FileList["jboss-web.xml"]
config.webinf_files += FileList["config/weblogic.xml"]
...

Now run the warble command again to recreate your WAR file. This time it should include the new weblogic.xml file. Use the domain’s administration interface again to re-deploy the WAR file. At this point my application started up fine and I was able to successfully test it.

MySQL

$ sudo apt-get install mysql-server libmysqlclient-dev
...
$ gem install mysql
...

SQLite

$ sudo apt-get install sqlite3 libsqlite3-dev
...
$ gem install sqlite3-ruby
...

This is the error I received when trying to start SQL*Plus:
$ sqlplus / as sysdba
sqlplus: error while loading shared libraries: /oracle/product/11.2.0/dbhome/lib/libclntsh.so.11.1:cannot restore segment prot after reloc: Permission denied

Solution A

After a bit of Googling I found two solutions. The first was :

(run as root)
# /usr/sbin/getenforce
Enforcing
# /usr/sbin/setenforce 0
# /usr/sbin/getenforce
Permissive
Only problem with this method was that once the machine was rebooted, the settings were lost as SQL*Plus produced the same error.

Solution B

A bit more Googling gave me this alternative solution:

Open two terminal windows, run one as root and one as oracle.

(as root)
# tail -f /var/log/audit/audit.log | tee oracle.log

(as oracle)
$ sqlplus / as sysdba
sqlplus: error while loading shared libraries: /oracle/product/11.2.0/dbhome/lib/libclntsh.so.11.1: cannot restore segment prot after reloc: Permission denied
(as root)

crtl-c (to abort the tail)
#audit2allow -M oracle < oracle.log
******************** IMPORTANT ***********************
To make this policy package active, execute:
semodule -i oracle.pp
# /usr/sbin/semodule -i oracle.pp

(as oracle),p>
$ sqlplus / as sysdba
SQL*Plus: Release 11.2.0.1.0 Production on Mon Nov 8 18:05:32 2010
Copyright (c) 1982, 2009, Oracle.  All rights reserved.
Connected to an idle instance.
SQL>

Afterwards I shut everything down and rebooted. This time it all worked without further problems.