Category Archives: Web

SmartMirror project part 2: Software

Development, IT, Web, , , , , , , , , , , , ,

In the first part of this series I looked into how to assemble a two-way mirror so that an Android phone behind it can be used as an information hub. Of course this is only one part of the story, cause we also need some software to display all this useful information on our Android device. Now, obviously there is already such software available, however I decided to write my own version from scratch for the following reasons:

  • to learn more about Android application development
  • it had to support Android Gingerbread (API level 10)
  • have all the modules important to me here in the UK
  • I like coding …

I will not fully describe all the code in this post, you can find the complete git repository here, but instead only pick the parts I think need some more detailed explanation. So lets start …

All about modules

I chose the common software pattern of  separating the final presentation of the data from the task of gathering it. All the data generation is done in modules which are independent of the UI. Most of those modules are using a remote web service for the information retrieval and therefore need to run asynchronous to the application main thread. By using OkHttp asynchronously for the remote access this is basically provided for free. A module is based on the abstract class Module which acts as connection to the UI. Every module will call postData when new data is available and postNoData when there is no data available or the module wants to clear any previous data it had sent. Obviously there are also some simple modules like the ClockModule which don’t use any web service, but also don’t need the complex asynchronies behaviour cause they executing rather quickly. Each module defines it own update rate cause this really depends on the nature of the data itself. Train time updates are rather frequent in contrast to the sunrise/sunset times which only change twice a day. Speaking about those two modules, let’s have a look which modules are available at the time of this writing:

  1. Time/Date: ClockModule
  2. Sunrise/Sunset: SunTimesModule
  3. Weather information from the UK Met Office: ForecastMetOfficeModule
  4. Travel time to work by car: TimeToWorkModule
  5. Next tube trains from a specific station: TflModule
  6. Next national rail trains from a specific station: NationalRailModule
  7. Upcoming calendar events for the next few days (API level >= 23): CalendarModule

Actually there is an eighth module called the TrainJourneyModule. This is a meta module combining the output of the tube and national rail train times modules into one. It sorts them by departure time so both types are shown interleaved.

For the data query part, Met Office, Tfl and Google Maps are using JSON. This is pretty standard nowadays and Java on Android has built-in support for parsing it. I was looking into more advanced JSON parsing libraries like gson and jackson but decided for the really simple task of parsing one static response it is not worth it to pull in those big dependencies. Maybe I will change that it the future, but for now the present code works just fine.

Looking at JSON data in Firefox is much more comfortable by using a JSON document viewer add-on like JSONView.

A completely different beast is the data query task for the National Rail module. National Rail uses the SOAP protocol and compared to JSON the authentication is already complicated. All web services use some type of API access token which is unique to every developer. This exists to prevent misuse of the public API, but in some cases also to limit the number of times an API can be accessed per day. To successfully prove to the National Rail web service that we are who we are we need to add an authentication token to the SOAP envelope header:

Namespace soap = Namespace.getNamespace("soap", "http://schemas.xmlsoap.org/soap/envelope/");
Namespace ldb = Namespace.getNamespace("ldb", "http://thalesgroup.com/RTTI/2014-02-20/ldb/");
Namespace typ = Namespace.getNamespace("typ", "http://thalesgroup.com/RTTI/2010-11-01/ldb/commontypes");
Element env = new Element("Envelope", soap);
/* Header */
env.addContent(new Element("Header", soap).
        addContent(new Element("AccessToken", typ).
            addContent(new Element("TokenValue", typ).
                addContent(BuildConfig.NATIONALRAIL_API_KEY))));

BuildConfig.NATIONALRAIL_API_KEY contains the API key. The rest of the body contains the actual request:

Element dep_request = new Element("GetDepartureBoardRequest", ldb).
    addContent(new Element("crs", ldb).
            addContent(BuildConfig.NATIONALRAIL_CRS)).
    addContent(new Element("numRows", ldb).
            addContent("5"));
...

Now, you may think this is simple enough, but I can assure you I needed quite some time to figure this out, cause the server on the other side is pretty strict on how this structure should look like. And as you can see SOAP uses XML which makes the request creation and the response parsing even harder. Luckily jdom2 exists which greatly simplifies this task. Here you can see the complete xml request:

<?xml version="1.0" encoding="UTF-8"?>
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
  <soap:Header>
   <typ:AccessToken xmlns:typ="http://thalesgroup.com/RTTI/2010-11-01/ldb/commontypes">
     <typ:TokenValue>6ef19def-6037-44f4-a2ac-06449273214f</typ:TokenValue>
   </typ:AccessToken>
  </soap:Header>
  <soap:Body>
    <ldb:GetDepartureBoardRequest xmlns:ldb="http://thalesgroup.com/RTTI/2014-02-20/ldb/">
      <ldb:crs>CHX</ldb:crs>
      <ldb:numRows>5</ldb:numRows>
    </ldb:GetDepartureBoardRequest>
  </soap:Body>
</soap:Envelope>

For more information on the various formats used by the different providers, have a look to their developer portals:

User interface

The user interface is quite simplistic. There is no way to interact with the application because it is behind a mirror. The only function the application has is to display the output of our modules in a nice and readable way. All the text and icons are white on black background. This ensures for best contrast behind the two-way mirror and makes it possible to see the information even on normal daylight. I tried to make all state changes animated, see for example AnimationSwitcher for the details. One interesting challenge is the code for switching to fullscreen for the different supported API levels. A nice feature of Android is that you can use new functionality of newer Android versions in your application and check at runtime if this new functionality is available. In your build.gradle you define the minimum supported Android version (minSdkVersion) and also the version your application is written against (targetSdkVersion). At runtime you check Build.VERSION.SDK_INT against the Android version when a specific feature was introduced. Only if this is equal or greater you can use this code. Now if you are below the required Android version you either need to fallback to other code providing similar functionality or your application can’t provide support for this particular feature (see e.g. CalendarView).  Main.java contains four different functions for switching into fullscreen. In API level 13 and below the only way to do this was to set the FLAG_FULLSCREEN for the main window. Starting with API level 14, setSystemUiVisibility was introduced which allows a more fine grade configuration of the behaviour of the Android system when your application is in the foreground. Over the following Android versions more and more flags have been introduced.

Configuration

MirrorHub doesn’t come with any runtime configuration. Remember the user is not able to make any changes when the phone/tablet is attached to the mirror. All the configuration is done at compile time by setting the various options in the gradle.properties file. You can use gradle.properties.example as the start for your own experiments. It contains things like the location for the weather forecast or the tube/train station you like to have departure information displayed on your mirror. But most importantly it contains the API keys for the various web services in use as outlined here.

I also added a runDebug/runRelease task to gradle which allows you to start the application without any physical interaction with the device. Together with an app like WiFi ADB you can install and run the application remotely while the device is attached to the mirror. However, please note that WiFi ADB needs a rooted device.

Final thoughts

I think the app has a lot of nice feature already. It runs on my mirror 24/7 and works just fine. I still have some more ideas for modules, like a caldav module to display calendar events from a caldav server, because the Android version that I use has no direct calendar support. I also played around with the proximity sensor to detect some hand waving in front of the mirror. It works fine with just the device, but unfortunately not when the phone is behind the mirror. The code is still there, so if you have some ideas how to make something like this work let me know in the comments.

Finally here are some screenshots while MirrorHub is running:

For some images together with the mirror see the end of the first post.

Why an ending slash on a URL matters (or not)

IT, Web, , , , , , ,

Recently I read a report from Google about the status of their own Web sites in terms of Search Engine Optimization (SEO). Uhh, Google, the search engine giant, is looking how the own sites are behaving when a Web crawler is stumble over them! To be honest, it’s fully clear to me that in such a big company, like Google, not all is perfect in the sense of cooperative appearance. There are different projects, different teams, with different strengths and weaknesses, different priorities and of course different managers. To summaries the report: In some areas Google does a good job, but in most it doesn’t.

How to optimize

Although SEO even lead to a business case for some companies, I believe it’s not so hard to do and in my humble opinion the most important points, a Web site owner should take care of, are the following 6:

  1. Usage of Web standards like XHTML 1.0 Transitional and making sure the Web site conforms to them.
  2. Usage of the <title> tag.
  3. Usage of meta tags like description and keywords.
  4. Usage of the header tags <h1>, <h2>, and so one.
  5. Adding canonical URL information to every site of the Web site, if a specific site is reachable from more than one URL.
  6. Writing good content.

Although 6. is quite obviously, it’s the most important point and often people forget about it and wonder why there is no traffic on there Web site at all.

1.-5. are technical aspects and if a Web site owner is using e.g. WordPress the Web site should be in good shape, already. Of course this depends a little bit on the used theme and the plug-ins the user has installed. For item 1. I always propose to make a bug report if a theme or a plug-in doesn’t confirm to them, as I have done for the theme used in this blog. Conformity could be easily tested with the W3C Valitator.

Why one should use HTML tags like the title, the meta and the heading tag is also easy to understand. A Web crawler isn’t a human, so he can’t distinguish between structural information because e.g. the font size is different. Helping him by semantic marking some of the text with the available tags of HTML is therefor a good idea. For the same reason using HTML tables to layout a page is a bad idea. Although this was standard in Netscape 4.0 times it isn’t necessary anymore these days.

Slash or no slash

Item 5. is about giving the Web crawler a clear idea about the structure of your Web site in a whole. Comparing 64k-tec.de/test and 64k-tec.de/test/ as a human doesn’t seem to be very different. From a technical point of view, it is. Considering that the Web itself is grown up in a UNIX environment, the former points to a filename and the later to a directory. This means for a Web crawler two different sites are targeted. The easiest way to fix this, is to decide about the nomenclature globally used. Either use the one form or the other. WordPress uses the “ending with a slash” variant (most of the time). By the way, this is also important for 64k-tec.de/test/index.html and other variants. Another way is to tell the Web crawler the canonical address even if the site is served by another URL. This could be done by adding the link rel tag canonical to the header of the page. On my homepage this looks as follow:

As you see, even on the top-level domain a slash is added. WordPress does this automatically for you since version 2.3. On older versions plug-ins for this task are available. The canonical tag is a good way to make clear which address is the base URL of a specific page. On the other side I see some potential for improvements. I have found two places in my blog where the base address isn’t targeted right. The first one is the tag for the site index relationship. It’s noted as follow, on my blog:

The second one is the link of the logo presented on top of every page on the blog. It use the following link:

64k

As you see the ending slash is missing, both times. It is not really a problem, cause the page itself use the canonical tag. The second mistake is clearly a failure of the theme. It’s not fully clear to me if the first wrong target is a failure of the theme or WordPress itself. It also happens with the default WordPress theme (version checked is 2.9.2).

Conclusion

Creating a Web site which is easily understandable by a Web crawler isn’t any magic. Of course you could make a pure science out of it. There are tons of plug-ins for WordPress available. On the other side considering some simple rules will help a lot. Reading Google’s hints or using Google webmaster tools might help, too. Even for Web crawlers from other companies.

On a last note, here is a nice article about how to effectively keep users out of your blog ;).

A simple WordPress plugin

IT, Web, , , , , ,

As you may noticed I use WordPress for my blog. This is an unbelievable great piece of software. It has uncountable possibilities for enhancements and as soon as you begin to explore these, you start adding some of them to your own blog. One of the plugins I added was WPtouch. This plugin wraps your blog into a theme for mobile devices if they are detected. This works for the iPhone, the Android platform or even Palms webOS. The theme is styled in a way it makes the content more readable on such (screen) limited devices and moves the menus to some extra place.

Why writing a plugin if there are so many

On our web server we are using Power PHLogger for a simple user statistic tracking. Unfortunately this software isn’t maintained anymore, but it does it job very well. To use it you have to embed some JavaScript code and a blind image to every website you want to track. My first try was to add this code to the theme I use. This worked ok, but I soon realized that this has to be done on every update of the theme. Additionally this doesn’t work with plugins like WPtouch, as there the whole theme is replaced. I searched for a plugin which takes this job, but I didn’t find one. So what, I’m a programmer, time for writing it myself.

Creating the necessary bits of code

Plugins in WordPress are located at the wp-content/plugins directory below the WordPress installation path. You have to choose a name, I used phlogger, and create a directory with that name. The plugin itself is written in PHP and named like the directory with the php extension. As the title says, it’s a simple plugin with one public method only, as shown in the following code.

<?php
/*
Plugin Name: phlogger
Plugin URI:
Description: Adds the power phlogger code to the footer.
Version: 0.1
Author: Christian Pötzsch
Author URI: http://www.64k-tec.de
*/

add_filter('wp_footer', 'phlogger_footer');

function phlogger_footer()
{
 $l = '<script type="text/'."n".'mce:1'."n";

 echo $l;
}
?>

The method just outputs the code for the phlogger tracking. WordPress executes this method because of the add_filter call. wp_footer is the filter keyword for content which should be added to the footer of a webpage, just before the </body> tag. The comment on the top of the file isn’t only there for cosmetic reasons. It’s parsed by WordPress and shown in the Plugin admin page of your blog settings. So it have to be there.

Of course I could have been adding methods for setting the phlogger link or the user name, but for my case this wasn’t necessary. For more information on writing WordPress plugins have a look at the good documentation.

Conclusion

Starting to write WordPress plugins is easy. It makes even sense for so simple tasks like this, because it offers the possibility to inject code (and functionality) without changing WordPress or the theme itself.