nosewheelie

nosewheelie is moving

2006-07-03T20:46:00.000+10:00

After lots of contemplating, I've decided to move my blog over to my main site. I'll be keeping this blog around for posterity (also the Blogger import on Wordpress seems to be broken), but all future posts will be made at the all new nosewheelie, a new feed is also available.

The 7 (f)laws of the Semantic Web

2006-06-13T01:14:00.000+10:00

When it comes to the Semantic Web, you might call me a disillusioned advocate. I’ve been dipping in and out of the technologies for the last 5 years or so, but am increasingly frustrated by the lack of any visible progress.

Source: The 7 (f)laws of the Semantic Web.

Not on my watch!

2006-06-09T13:53:00.000+10:00

Our resident IntelliJ plugin guru Rob has been at it again. For a while we've been dreaming up ways to automate a build breakage notification, so that it's really in your face. Rob's solution?

Will wonders never cease?

Why do they hate us?

2006-06-07T21:17:00.000+10:00

The hypocrisy of the US administration astounds me constantly (and they're not alone unfortunately).

For a textbook example of why we are hated, consider Gaza and the West Bank. There, a brutal Israeli/U.S.-led cutoff in aid has been imposed on the Palestinians for voting the wrong way in a free election.

...

Query: who, besides al-Qaeda and recruiters of suicide bombers, can conceivably benefit from persecuting the Palestinian people like this? Does President Bush or Condi Rice think the Palestinians will respect an America that did this to their children, after we urged this election, called for Hamas to participate, and preached our devotion to democracy?

...

The White House says we don’t negotiate with terrorists. But when we had to, we did. FDR and Truman summited with Stalin at Yalta and Potsdam. Nixon met with Mao in Beijing. Kissinger negotiated with the Viet Cong and North Vietnamese at Paris. Bush I allied with Assad in the Gulf War. Clinton had Arafat to the White House too many times to count.

Source: The Persecution of the Palestinians.

Ruby early return considered harmful (for idiots like me)

2006-06-05T13:17:00.000+10:00

I've been playing with some Rails code at home to support some of the work we do at the day job. I'm building a search tool that detects the kind of input and performs searches based on this type. I've been working with Ruby's regular expression support, and test driving a class to find UUIDs. Here's what I started with (please don't comment that it could be simpler, I was starting simple... :).

class Uuid
  REGEX = /[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}/

  def Uuid.isUuid(fragment)
    if fragment =~ REGEX
      return true
    end
    return false
  end
end

Thinking that this could be simplified be removing the return keyword, I removed it!

class Uuid
  REGEX = /[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}/

  def Uuid.isUuid(fragment)
    if fragment =~ REGEX
      true
    end
    false
  end
end

This caused all sorts of funny problems, which in hindsight are painfully obvious, but at the time are just a pain. Removing the return causes the if to essentially fall-through, always returning false.

Discussing this at work, perhaps I've hit up against a potential maintenance issue with Ruby. I'm hoping this isn't the case (you can do similar stupid things in most languages), but some of the things you can do with Ruby make me a little nervous. I'm holding my tongue for a while (I did the same thing with XP and am hooked) to see where things go, I'm sure it'll all be fine...

Managing Software Developers

2006-05-31T12:42:00.000+10:00

If you're an engineer at a company where becoming a manager is considered a promotion, then you only have three choices: become a manager yourself, or leave, or resign yourself to being a second-class employee. It should be obvious — you can work through the math using three sock puppets — that this is an arrangement that pushes a company inexorably towards mediocrity. The best engineers either leave the company or try their hand at management, often with doubly disastrous consequences: they simultaneously lose the company a great engineer and gain them an awful manager.

Source: (Not) Managing Software Developers (Via Anarchaia).

On project estimation

2006-05-11T08:04:00.000+10:00

Tom writes about How Long is a Piece of String?, which addresses one of my main frustrations with software development; estimates and how project managers' and stakeholders deal with them.

So, how long is a piece of string? ... It's a pretty silly question, isn't it? But would you believe that questions like that are asked all the time regarding IT projects? And what is even more amazing is that people actually give answers to such vague questions and then they make important business decisions based on them.

Far too often I see people wondering around in disillusionment as estimates are not hit. This can sometimes be exaggerated on agile projects as management get more feedback than they're used to, and some panic.

Immediate feedback is a key element to exceptional performance

2006-05-10T20:16:00.000+10:00

Matt tells us of the importance of instant feedback, and hightlights A Star Is Made. Very timely considering the time our build takes...

How to make programming hard for yourself

2006-05-06T13:56:00.000+10:00

Reg Braithwaite post about How to make programming hard for yourself. He makes several interesting comments that ring true with me. I've worked with several people who meet the "people that consider my hard problems to be recreational diversions" comment, and though they are phenomenally smart and can indeed tackle the hardest of the hard problems, (I feel that) they find it hard to produce code that is of a high quality consistently. Then, there are those people, who are naturally smart enough to tackle the hard problems yet are also able to produce good quality code on a day-to-day basis. There would only be about 5-10% of developers who fall into this bucket, and getting the chance to work with them is mind blowing. There is nothing better for your career development than to be thrown in the deep end with a bunch of people that know more about everything than you do. Indeed Dave Hoover's Be The Worst apprenticeship pattern embodies this idea. I'm currently working on a project with several people that fall into this category. There's nothing like a dose of reality to keep an ego in check!

On a similar note, Damien Katz shows us the Signs You're a Crappy Programmer (and don't know it). I don't agree with all of these, the 20 lines of code one especially. We spoke about this at work today, and while they're all "magic" numbers, they really force you to think hard about your design. I heard a podcast by David Heinemeier Hansson once that talked about how being under constraints you may not like forces you to be creative (he was referring to Rails). I think length is one of those occasions.

Mac Backup Software Harmful

2006-04-29T07:46:00.000+10:00

Paradoxically, copying a file and being sure that all information has been copied is not easy under Mac OS X. I analyzed a variety of file copying engines, most of them command-line tools, and demonstrated how they fare in preserving file metadata.

Source: Mac Backup Software Harmful

This is quite interesting as I've never felt the need to move from Carbon Copy Cloner to something else, as I've successfully managed upgrades using it from Jaguar to Panther & Panther to Tiger.

Subversion

2006-04-24T21:05:00.000+10:00

Andrew asked me to take a look at some JRDF work he's been working on. As I've been slack lately, I needed to check out the code to take a look, and he's recently moved it over to Subversion. I've dealt with svn before from within IntelliJ, but I usually like to check out my projects from the command line, to get a feel for what's going on. So my running commentary for all today is information on getting svn going for CVS wannabes.

Gotta love the blame/praise command!

Samurai

2006-04-11T12:14:00.000+10:00

If anyone ever needs to analyse Java thread dumps, Samurai is the Beez Neez. No more squinting at less.

Cannot Delete

2006-04-10T13:22:00.000+10:00

The ever helpful error message, from a colleague's interactions with a remote server.

SPARQL-a-go-go

2006-04-07T08:03:00.000+10:00

Via Dave: The DAWG (the WG I once sat on) has just released three SPARQL specifications as W3C Candidate Recommendations! This is great news for the semweb community, even if some of the decisions made are not ones I agree with. The potential upside of semweb is huge, hopefully it can be another one of those overnight success stories, 10 years in the making.

Firing squad

2006-03-29T08:20:00.000+10:00

I think we should take the “Web Services” label into the jailyard, strap on a blindfold, give it a last cigarette, and shoot it. It doesn’t mean much any more, and to the extent that it does, it’s misleading: WS-* doesn’t have much of the Web about it. I don’t have a proposal for a new name for the WS-* style; sorry, but I just don’t care.

From Tim Bray, Styles: Beyond WS and REST.

The Wrong Questions

2006-03-24T20:54:00.000+10:00

Walking around the office this last week, I've begun to notice some things I've forgotten about large bureaucracies built around IT projects. Everyone has great answers, but they're answering the wrong questions.

A couple of examples. We're collocated within a division who manage the development and deployment of large projects within the company. Walking through the office last week I saw a large poster of the "Rational Unified Process - Process Made Practical". Now I've never developed under any Rational process (or tools for that matter), but the idea of something that calls itself the "Unified Process" (complete with caps) struck fear in my heart.

In a previous role, we had the experience of receiving several functional requirements specifications that were not suitable to develop from. They missed several key requirements, did not accurately describe the current system and contained lots of diagrams that were nothing more that pretty pictures. One of these specs was a 30-odd page Word file that when distilled contained a page and a half of requirements in bullet point form.

Now, from a business point of view I'm sure these specs were of value, but for their intended purpose they were of little. The problem was, the documents were produced within a process that did not take into account their value to development, only on their adherence to documentation standards. In one case, the requirements had not been gathered from the customers, and the touch points with the function and capabilities of the current system had not been checked with those who knew it best. The document was derived in complete isolation, from a higher level document alone.

Now I don't think that anyone truly believes that we know how to develop software. Those of us who practice XP would like to think that agile is the way and the light. I believe it's a great step forward - and a step that practices self review and the infamous continuous improvement - but it may not be the best way to develop all software.

All in all I have the sinking feeling that there are lots of people are all busy doing a wonderful job that contributes little to the effective delivery of a software project. There are lots of great answers, but the questions being asked are not the ones that should be.

You know you have problems when...

2006-03-14T21:17:00.000+10:00

Your classes are over 5000 lines long.
You compare your class to three JDK classes.
You incorporate transaction management, path management, file management, connection & URL semantics in a single class.
Your class-level javadoc is 100 lines long.
You use a version of commons httpclient that has bugs parsing URLs with ports & is incompatible with the latest release.

Also known as WebdavResource. Why this sort of code continues to be written is bizarre.

Update. As James has pointed out in the comments, editing your blog over a modem when you cannot spell is not good. I've never had a good experience with slide, and none of my recent experiences is any better.

The C guru

2006-02-23T10:17:00.000+10:00

We have the pleasure of inheriting an application written in C that wraps a GDS API. I've been in the process of upgrading the API to the latest version and ran into a few stumbling blocks. In typical fasion, I'm posting the resolutions them here so I can find them again.

ldd - Lists shared library dependencies
nm - Lists symbols from object files
objdump - Displays information from object files. This is like nm on steroids, and apparently also lists symbols for file that have had their symbols stripped.

BTW, the title of this post does not alude to me, but rather a label I've been throwing around at work for kicks, only it hasn't stuck to someone yet!

Continuous Integration

2006-02-02T14:54:00.000+10:00

Was having lunch today with a friend of mine and one of his colleagues when the topic of my current work came up. I told him that I was working on a new feature of our product and had spent the last chunk of time integrating it with the existing codebase. Greg commented that if I was performing continuous integration I wouldn't have run into this problem. My reply was that we were and I went on to explain why I was having problems.

On the walk back to the office however, I thought some more about the question and it struck me that I'd misinterpreted it. On reflection I think that perhaps there are two ways that we could have completed this work and continuous integration means more than just good tests running in a continuous build environment.

Separate Implementation Method

This is the approach we took. Basically, we came up with a high-level approach, decided where we were going to hook it in, then built up the code to match the existing functionality (using a bottom-up TDD approach). We then (this is the situation I'm in now) tried to hook it into the exising code base, where I discovered we did not currently have access to required details.

Integrate Always Method

Using this approach we would break the system up front by integrating our new code directly into the existing code base. The more code we had written, the closer the system would be to completion. We wouldn't need to perform after the fact integration and we'd more easily know when we where finished.

Summary

In hindsight, we could have taken option 2 and saved ourselves the hassle. One problem with it is as we are working on a production system, we cannot break current functionality as maintenance releases are frequent and business often changes priority by pulling us off and on "sub-projects" according to their need. For us, this approach requires access to a separate source branch (which we have) in order not to break the system. To me this highlights some lessons I've obviously forgotten...

jMock ordering

2006-01-24T11:17:00.000+10:00

In case anyone (i.e. me next week) needs to specify explicit method ordering in jMock, here it is. This code specifies the order of a chain of method calls (it came from testing a wire protocol implementation - don't ask...), rather than just expecting all calls to come after a single call. In particular note the ordering of the id() and after() calls, once again IntelliJ saves the day.

Mock mock = mock(RedstripeSocketTransport.class);
mock.expects(once()).method(METHOD_OPEN).id("CallOpen");
mock.expects(once()).method(METHOD_WRITE).with(eq(seededRequest))
    .after("CallOpen").will(returnValue("foo")).id("CallWrite");  
mock.expects(once()).method(METHOD_READ).after("CallWrite")
    .will(returnValue(seededResponse)).id("CallRead");
mock.expects(once()).method(METHOD_CLOSE).after("CallRead");
return (RedstripeSocketTransport) mock.proxy();

Note that that write() does not actually return a value in the production code, it's in the post to help me remember ;) Apologies to Andrew if he's posted this already, I thought he had, but google does not know about it...

Vlad has made a good suggestion in the comments, that statements have the method name given to them as ID by default. I'd seen this as an error message but wasn't sure what it meant.

Closures for Java

2006-01-23T12:36:00.000+10:00

Danny's been talking about Closures in Java. We did a similar thing in JRDF for testing.

AssertThrows.assertThrows(RuntimeException.class, new AssertThrows.Block() {
    public void execute() throws Throwable {
      somethingThatIExpectWillThrowAnException();
    }
});

All in all the syntax is ugly and nowhere near as neat as Ruby's, however such constructs can significantly cut back on duplication.

Up yours

2006-01-19T08:44:00.000+10:00

Now this is priceless: Mac OSX. Mac OSX Up Yours T-Shirt.

Exception messages

2005-12-21T09:49:00.000+10:00

I've been spending a lot of time working on legacy code lately and apart from everything else, the thing that sticks out is good exception handling. Doing this has firmed my belief in handling exceptions as close to the source as possible (or alternatively where you have the most context) and wrapping them in RuntimeExceptions (or subclasses of). It also highlights the importance of placing decent messages into exceptions. The usual "Cannot do X" is next to useless. In our bookings engine you have all sorts of good context, booking numbers, flight details, site codes, etc. that allow you to track down problems in code that a stack trace doesn't always allow.

What prompted this post was the above image from TheServerSide.com with a wonderful message.

Software Quality

2005-12-15T14:00:00.000+10:00

I've been meaning to write about the cost of writing software for a while (unfortunately it's sitting as a draft), but the recent Tokyo Stock Exchange fiasco highlights the true cost of not making the effort to write software correctly. When will people (i.e. the management with the money bags) learn that the cost of maintenance and also mistakes are orders of magnitude higher than the costs of doing it properly the first time (and doing it once). Creating quality software is not rocket science, there are well known methods for doing this. What I find amazing is how many organisations baulk at paying slightly more initially, but will happily pay bucket loads down the track.

IntelliJ and Eclipse

2005-11-30T08:30:00.000+10:00

Well I thought I'd never stoop so low as to enter into this flame war on my blog, but Patrick's Eclipse Live Templates post is worth keeping, so I may as well document my feelings also...

As everyone knows, Eclipse is the best IDE out there, it's the only choice right? Very wrong. Sure, Eclipse has the market share, has more plugins, and widepsread industry support, but is it really a better IDE? My take on it is no. Yes I can use it, but it feels like the difference between riding a hardtail vs a dually. Sure they both get you from A to B, but one is much nicer to ride and doesn't hurt your ass as much. Another analogy I like to quote is that Eclipse feels like using Word, it tries hard, but in the end you end up fighting the editor just to get work done.

I've been using Eclipse a lot lately (by a lot I mean a couple of days solid) as we're merging between branches and IntelliJ does not support this (as far as I can tell anyway). I've also had a lot of people ask me for my opinion on IntelliJ as we've got some spare licences floating around and most of the guys are using Eclipse.

Eclipse does have some good things going for it.

Better CVS support. Eclipse seems to have re-thought the entire revision control experience, making it easy to work with. This does have it's downsides, like Eclipse only developers who don't understand CVS or Subversion fundamentals (and cannot operate outside of Eclipse) or with CVS gurus who cannot understand Eclipse's naming of operations and its workflow.
Better use of screen real estate. The tree in the Package Explorer is a good example of this. Eclipse seems to use much less horizontal and vertical space in the tree view, allowing more classes to fit in a given screen size.
Eclipse looks nicer (by default) to me, perhaps owing to its SWT heritage vs IntelliJ's Swing. Its window management is a bit more advanced also.
Eclipse's debugger is apparently better, I say apparently as I haven't noticed it myself. It supports threaded apps in a more logical way, by sticking to a single thread and not jumping threads when breakpoints are hit in these threads.

So what are the bad things?

Eclipse requires you to save all the time. This gets very annoying when you're used to IntelliJ. Say for example you make a change to a test and you want to run it, why wouldn't I want to save the test file. This stuff should just happen without the need to prompt the user. File management should be taken care of. IntelliJ does a good job of this and allows it to be configurable (the save timing that is), to the point where you forget about it altogether. If you want an older version, it's in CVS (cause you're doing tight spinning when writing new code) or in the local history.
When comparing files (select two projects, right click, Compare With -> Each Other), copying all changes from one file to another (called "Copy All Non-Conflicting Changes From Right to Left"), Eclipse does not warn you that you need to save the file (I'd been using a different form of merging/diff - the Compare With -> Another Branch or Version... - where this change is automatically saved). So I'd been happily chugging along all morning, when Eclipse crashed, losing a few hours of work. On the plus side, when you do hit save, it quickly shows you the "file has changed" symbols, saves the file, then removes them. Very helpful.
Eclipse has serious usability issues in graphical diff (Java Source Compare) screen. 1) When merging differences, Eclipse continues to highlight changes, even though they've been applied. Whereas IntelliJ will remove the coloured background and border from changes that you've already applied. This provides an easy visual clue that you've complete a merge. Also, the little buttons for merging are too small, have icons that look the same (probably a result of them being too small, you can see the differences if you look closely) and are in the wrong position. IntelliJ gets this right, by placing the the icons (a cross for delete or directional arrow for merge) at the point of difference, allowing you to clearly see the affect clicking will have.
Like Word, Eclipse wants to own your files and wants you to make all your changes through it (I assume so it can keep its caches up to date). So if I change a file on the file system (say to back out a change because Eclipse won't undo), Eclipse continuously reports that the file is out of sync with the filesystem. Why on earth it doesn't just sync it and be done with it is beyond me. Now I have to a) remember where I am in my merging, then b) get out of the view, c) re-sync the project and d) re-compare the projects and e) go back to where I was. Again, a little thing that really gets in the way of productivity.
Eclipse's two ways of comparing do not give the same result. CVS differences (select a project, right click -> Compare With -> Another Branch or Version...) show less files than a straight file system difference (select two projects - representing different branches - right click, Compare With -> Each Other).

By now I think everyone knows my view on the matter, but my short summary is that while Eclipse is fully featured, everything is done in such a half-arsed way that I really don't know how people can use it on a day-to-day basis. I think I'd prefer to go back to using a straight text editor than be forced to work around its inadequacies. And why is IntelliJ better? It just feels right. That's the best way I can put it. Like any tool it takes a while to get used to it, its keyboard shortcuts, etc., but once you learn these you'll never willingly go back. It seems to know what you want to do and will suggest things to improve the way you work all the time. Everyday I find a new thing to like about it. Its smarts are not the Word breed of smarts however. You always have control and it does not bend you into its way of working. It's a tool that works with you rather than against you.

Thursday Links, Part II

2005-11-24T08:34:00.000+10:00

Evaluation: moving from Java to Ruby on Rails for the CenterNet rewrite;
Greg Luck's Architectural Priniciples;
The always timely No Silver Bullet: Essence and Accidents of Software Engineering;
Application Integration and Agility (1): Understanding Agility.

Thursday Links

2005-11-17T14:29:00.000+10:00

Some quick links for Thursday

Making the Date;
Dilbert on Agile Development;
Sustainable Software Development,being worked on to help organisations improve the way they build software, using Agile methods;
vim Regular Expressions (via Dave);
How to Safe Sleep (Hibernate) Your Mac
FindBugs Part 1 & Part 2.

Another reason not to install Windows

2005-11-11T07:46:00.000+10:00

As if the world needed more reasons... Here is a blog entry about Sony's latest DRM tactics, which has been followed up by the BBC and USA Today. There are followup posts here and here.

Free Smalltalk books

2005-11-10T14:19:00.000+10:00

In case anyone (like me) wants to get an idea of what smalltalk is about: Stef's Free Online Smalltalk Books

SPARQL support

2005-09-14T12:47:00.000+10:00

I thought that I'd write a bit more about the initial end-to-end support of SPARQL in JRDF. While it's no where near complete, it does parse simple queries and apply them to a graph, giving real results. It's taken a while to get it to this point as I've been concentrating on test-driving the functionality, partly as an exercise in test-driving (can it be done with a SableCC parser) and partly to ensure better quality code that's well structured and loosely coupled. This has drawn out a very nice architecture for the connection layer (connection, parser, executor) that means that it's easy to test and easy to swap out implementations when we come to supporting new languages, which is probably never, but the testability makes it all worthwhile even without this feature. So here's a rough sketch of the current architecture in HTML glory.

org.jrdf.connection.JrdfConnection - Top level way to send textual queries to a JRDF graph.
- org.jrdf.query.QueryBuilder - Turns textual queries into org.jrdf.query.Query objects.
  - org.jrdf.sparql.parser.QueryParser - Adapter to compiler (i.e. SableCC) specific class.
    - org.jrdf.sparql.analysis.SparqlAnalyser - Parses the queries into domain objects (e.g. Triple, Query, etc.).
      - org.jrdf.sparql.builder.TripleBuilder - Part of a set of builders that take SableCC org.jrdf.sparql.parser.node.Nodes and turn them into corresponding domain objects, such as ObjectNodes, Triple, etc. As more of SPARQL is implemented, these will be fleshed out more, and probably governed by a higher level class (perhaps called a herder ;).
- org.jrdf.query.JrdfQueryExecutor - Takes org.jrdf.query.Query objects and executes then against a graph. This is the class that hooks into a query layer, a very bad one at present!
  - Query layer - currently does a select all and filters using an iterator based on the constraints in the query.
  - org.jrdf.graph.Graph - The JRDF implementation of an RDF graph.

So why didn't I just implement it in Kowari? Well, initially I did, however it proved to be too difficult for a number of reasons. Not all of these were my initial drivers, but it's proved to be a good decision in hindsight.

Kowari has a convoluted build system, which while fine for managing a large team of developers, was not agile enough to enable me to develop on the train on my small PowerBook. Kowari's build is broken into components, which is great for pluging, not not so great when you need to build everything from scratch and the dependencies are not always checked. This is one of those issues about being "owned" by the build rather than the other way around. Large Ant-based build fines need constant maintenance and refactoring. On a project as large as Kowari, this could almost a full time job. The same issue also manifested itself in it being quite hard to build Kowari in IntelliJ. It was doable, but required a round-a-bout solution that wasn't optimal.
Slow turn-around on tests. The full test suite (excluding load/performance) in Kowari takes a long time to run (much longer than the full test suite in JRDF) which meant that I'd be spending my entire train ride (and battery) waiting for tests to complete. Most of this was based on the build system, which because of its componentised nature, has lots of dependancies between components, which aren't always checked properly, resulting in recompilations and rebuilding of JARs when they've just been done. I could have taken the time to fix it (I believe Mark was working on this before Tucana ceased operations), however my PowerBook just isn't grunty enough to handle that kind of workload.
The unit tests are not unit tests. Most of the tests in Kowari are integration tests, that is, while they may test only a single class, they test that class with the roles of its dependent class filled by real concrete instances, rather than mocks. So the tests not only test the class, but also every other class hanging off the class under test. Consequently, the tests take a long time to run.
The Kowari code is not test-driven. Even though at Tucana we did our best to ensure we had tests to cover everything (and we did a pretty decent job), these tests are not unit tests and we were not test driving the code, we were testing before at best. I wanted to test-drive the code for this, and unfortunately this was harder to do with Kowari than with a fresh(er) codebase.
The code is tightly coupled. This was the main reason I stopped work in the Kowari codebase and moved to JRDF. It's all over the codebase (see for example recent comments on the Kowari Developers list about this), but the kicker for me was that the code to execute a query was embedded in the parser - ItqlInterpreter (Paul also commented on this a while back). Seeing as I wrote a lot of this, I only have myself to blame, but the job of pulling the query execution code out of the parser was too big a job. I wanted to do this, so that I could pull out the common code between SPARQL and iTQL into a set of shared classes and have the language specific bits being the parsers only. All the parsers should need to do is generate Query objects for sending to the query layer, something else should take care of the execution.
Andrew and I have been bitten by the agile, quality, TDD and refactoring bug, and having our own small project to play with makes this much easier. It also allows us to enforce higher standards on the codebase as there's only two of us that need to agree at the moment.
Andrew had some ideas about building a query layer, learning from what was done in Kowari
We're interested in seeing if we can use some of the DI principles in the code, now that we've decoupled the classes and have clear responsibilities. I don't think you'll see a Spring requirement just yet, but you'll see a lot of comments saying things like // FIXME TJA: Set builder using IoC, which will make the job of plugging in different implementations of things like org.jrdf.query.JrdfQueryExecutor really easy. It should also make JRDF a lot more embedable than it currently is.

So what's next?

Completion of the SPARQL grammar as it stands in the candidate recommendation (this may be dependent on the next point).
Implementation of a better query layer. The current iterator-based approach will not scale as it reads the entire graph into memory and then filters out results that match the constraints. Andrew has already started work on this, and I think that it'll be a lot simpler than the model used in Kowari. Kowari has more features also, such as resolvers, that JRDF does not need (nor perhaps want), so we can afford to make it simpler, and learn from what was done in Kowari.
More long term, and perhaps beyond the scope of JRDF, we could Learn from what guys like Obbie Fernandez are doing in the Ruby arena, especially some of the thought about a Rails-like framework. I'm not convinced as yet, but semweb still needs a killer app (yes, we all know it's WonderGui).
Andrew wants to release a 0.4 release soon, so perhaps we'll have a SPARQL parser and a stable API in that release. At the moment we've been a bit eager with the refactoring, which is fine up to a point, but we'll need to decide on a stable API as the release nears.

My plan for Kowari is to complete the SPARQL implementation in JRDF and hopefully just update the JRDF version in Kowari, thus getting SPARQL support for free. Of course this assumes there will be none of the normal problems such an upgrade entails. The downside of this is that Kowari is heavily optimised for speed, something that I haven't spent any time on in the work I've been doing. I'm not overly worried by this as the architecture lends itself to plugging in different implementations and really all I've done is create a parser that turns query strings into Query objects. This parsing does not constitute a significant amount of time in overall query execution. We're lucky in this regard as I fully expect that someone with more skill in this area will be able to implement an efficient query executor that plugs directly into Kowari's session or some other mid-level API. In fact, because of the de-coupling of the components in JRDF, this is very easy to do. Some easy options are:

Create a translator that translates org.jrdf.query.Query instances into org.kowari.query.Query instances. These Querys can then be sent directly a org.kowari.server.Session for execution.
Create an instance of org.jrdf.query.JrdfQueryExecutor that takes org.jrdf.query.Querys and executes them directly against a Kowari API.
Then there's the other option of plugging in the JRDF query layer into Kowari's layer. I'm not sure that this is either the right way to go or the most efficient, it smells like a hack to me.

I hope none of this comes off as criticism for Kowari as this wasn't my intention. It's certainly the best triplestore there is, and the project is moving ahead at great guns now. It's basically a reflection that Kowari is too heavyweight to do what we want to do and that both projects have different goals. Kowari is effectively a scalable fully transactional database, while JRDF is a Java RDF API. The two can (and should) happily coexist, learning off the work each of us do. Fortunately, this is easy, as we all used to work together (bar the NG guys) and know each other well. Oh, it helps that Paul is a committer on JRDF and Andrew and I are on Kowari...

SPARQL support in JRDF

2005-09-12T08:28:00.000+10:00

The SPARQL support in JRDF is now working end-to-end. It's very incomplete, but basic SELECT * WHERE { ?s ?p ?o } type queries are working and being applied to the underlying graph. This means that we now have the full stack that allows you to grab a connection, throw a query at it and then expect a result (Answer) back. Here is the goods running in IntelliJ.

SableCC, SPARQL and AST/CSTs

2005-09-08T08:49:00.000+10:00

In my spare time (i.e. the train ride to work) I've been working on JRDF, specifically implementing SPARQL support. As I was one of the people who did the original iTQL support in Kowari, I pulled most of the inspiration for this from the SableCC implementation Simon and I did in Kowari. Hence SableCC was a natural choice. However, after talking with Andy S about his ARQ work and being generally lazy about keeping up to date with the SPARQL spec, I almost decided to switch to using JavaCC.

Thankfully Andrew talked me out of it, and JRDF now has simple SPARQL support (SELECT * WHERE { ?s ?p ?o }) and a rudimentary query layer. And just when I thought that I was the only person in the world who uses SableCC, along comes Nat Pryce with a nice little tutorial on the latest version, Concrete to Abstract Syntax Transformations with SableCC.

Mountain biking and semweb

2005-09-07T11:21:00.000+10:00

Now this is cool! Mountain biking and Semantic Web.

In the past two weeks I hacked a little Google Maps application called #B4maps which shows some tracks driven lately. Tracks and the Images used to annotate some waypoints are stored in some semi-good XML file… ... So what we got? An application to show and edit tracks on a map: #B4maps, a webservice to represent this tracks using RDF, a nice query to figure out the waypoints...

On unit testing and "rm -rf src/java"

2005-09-02T20:14:00.000+10:00

I've just gone through the task of recovering the source files I've been working on for JRDF after an inadvertent rm -rf src/java. As you would expect this is not a pleasant experience... Two things however, made my life easier.

Firstly, using IntelliJ I was able reverse the change as easily as going back in my local history. So this kept me going for a few days, until I needed to commit my changes back to CVS. Here's where the trouble started. CVS gave me the always helpful "XXX is in the way, move it aside" warning and promptly gave me big fat Cs down the screen. Figuring the easiest way to deal with it was to in fact move the files aside, I did so and began a manual merge. Into the ring comes IntelliJ again with its very clean manual comparison tool.

Secondly, the high quality of the unit tests Andrew and I been writing lately (more on that later) ensured that once I'd completed the merge, I could be sure that I'd not lost any functionality. Lucky for me, I've lately taken to storing my test source in a separate source tree (test/java) from my production source (src/java), so my fat fingered delete left the tests intact.

Brisbane MTB Links

2005-08-11T20:07:00.000+10:00

Mountain Bike Web Magazine
MTB Dirt, has good riding maps.

Ruby Links

2005-08-10T12:46:00.000+10:00

So I don't forget them...

BUFD

2005-08-10T12:40:00.000+10:00

Big Up Front Design, been seeing too much of this recently. Naresh Jain has some comments that are quite interesting.

Dependency Injection

2005-08-09T15:12:00.000+10:00

Further to my post about doing the simplest thing (which turned into an Ioc/DI rant), Jim Weirich has a nice post about DI in dynamically typed languages, i.e. Ruby. The post starts off with a nice little intro to inversion, and moves onto examples in Ruby.

Quick links for today

2005-08-09T12:11:00.000+10:00

This is neat, Google Maps can work off lat/long, so once you find your coords, you can plug them in: my house.
A couple of useful tools for doing conversions between different representations of the same thing, e.g. going from web service generated objects to domain. Morph and Lorentz (from TSS). I've been exploring a similar idea in JRDF, where I map between SableCC generated parser objects into my local representation of the same thing. I think generics will make my life easier here, but I'm yet to find the right syntax to allow me to do this.
POJO Application Frameworks: Spring Vs. EJB 3.0, enough said.
Sticky tests

The Middle East for Geeks

2005-07-25T09:30:00.000+10:00

From Tim comes The War on Terror. This one is probably for those with a sense of humour and a left leaning.

XINS

2005-07-22T09:52:00.000+10:00

XINS looks like an interesting technology for simplifying web services, well, if its marketing lives up to the hype. They've just released a XINS Primer that takes you through building a web service using XINS. And for once, someone built a tutorial on a Unix platform ;)

Generalizing Specialists: Improving Your IT Career Skills

2005-07-22T09:51:00.000+10:00

I believe that to become an agile software developer that you need to move away from being a narrowly focused specialist to become more along the lines of what I like to call a generalizing specialist. A generalizing specialist is someone with one or more technical specialties who actively seeks to gain new skills in both their existing specialties as well as in other areas, including both technical and domain areas. When you get your first job as an IT professional it is often in the role of a junior programmer or junior DBA. You will initially focus on becoming good at that role, and if you’re lucky your organization will send you on training courses to pick up advanced skills in your specialty. Once you’re adept at that specialty, or even when you’ve just reached the point of being comfortable at it, it is time to expand your horizons and learn new skills in different aspects of the software lifecycle and in your relevant business domain. When you do this you evolve from being a specialist to being a generalizing specialist. Generalizing specialists are often referred to as craftspeople or even "renaissance developers".

Source: Generalizing Specialists: Improving Your IT Career Skills

40 Things That Only Happen In Movies

2005-07-21T15:29:00.000+10:00

This is golden! 40 Things That Only Happen In Movies. My favourites:

Should you wish to pass yourself off as a German officer, it will not be necessary to learn to speak German. Simply speaking English with a German accent will do.
If you are heavily outnumbered in a fight involving martial arts, your opponents will wait patiently to attack you one by one by dancing around you in a threatening manner until you have defeated their predecessor.
Rather than wasting bullets, megalomaniacs prefer to kill their enemies with complicated devices incorporating fuses, pulleys, deadly gases, lasers and man-eating sharks.

Tripletest Report

2005-07-20T12:37:00.000+10:00

Dave Wood reports on the results of the Tripletest Report, a report comparing Kowari and Sesame. This confirms Tucana's former internal performance testing. An interesting note is that they only tested up to 20 million triples, back in the day we were regularly testing up to 500 million during load and performance tests. Plans for the new backing store will increase this even further, if it's ever implemented...

What does "the simplest thing" mean?

2005-07-19T14:18:00.000+10:00

Agile advocates often tout the adage do "the simplest thing". Indeed I've heard this phrase used as both justification for and criticism of agile methodologies (XP in particular). However, many agilers and non-agilers alike often have trouble knowing just what is the simplest thing. I often use the phrase within a given context, meaning the simplest thing given a number of healthy practices. Generally these include test driving, loose coupling, use of patterns) and other sometimes nebulous practices. Which practices are important comes mostly from experience, and I often find it hard to articulate just what practices are important.

To help, let's look at doing the simplest thing for a simple example, linking to an external API, in this case an XPath processor. The example comes from a recent project, actually, a lot of my recent rants come from this project. This project linked to an XPath API directly in over 300 places and when we needed to replace the implementation with a faster and less memory intensive processor, we were forced to change all of these lines of code, increasing the chances of new problems and regression.

Simplest Simplest

In order to hook into this API, I simply embed a call to the API directly in my code.

class ElementFinder {
    void findElement() {
        Element element = XPathAPI.findSingleElement("...");
    }
}

I'd say that this is the absolute simplest thing that you can do to link to this API. It's quick and does what is desired, but what are the downsides?

Tightly coupled to implementation - If I was to use the same code in 300 classes, and then find I needed to change APIs (say as a result of performance testing), I'm stuck with changing 300 lines of code.
Untestable - I can't test that my code does the right thing as I'm calling directly into the API I want to use. For example I cannot drop a mock in during unit testing.

Uncoupled Simplest

The next step in de-coupling our example is to remove the direct call, and hide it behind an API of our own.

class BetterXPathApi {
    Element findSingleNode(String xpath) {
        return XPathAPI.findSingleNode(xpath);
    }
}

class ElementFinder {
    void findElement() {
        Element element = BetterXPathApi.findSingleElement("...");
    }
}

The beauty of this is that whenever we want to call the external API we call through our own version, so if we want to changed the backing implementation, we just need to change one line. But what are the downsides?

Tightly coupled to implementation - Even though we've pulled out the call to the external API we cannot change the call without changing code and I can't swap out implementations easily.
Untestable - We still cannot test that our code does the right thing as we can't (easily) set a mock in place of our BetterXPath code.

Interface Simplest

To make this more adaptable, we can hide our implementation class behind an interface.

interface BetterXPathApi {
    Element findSingleNode(String xpath);
}

class XercesXPathApi implements BetterXPathApi {
    Element findSingleNode(String xpath) {
        return XPathAPI.findSingleNode(xpath);
    }
}

class ElementFinder {
    void findElement() {
        BetterXPathApi xpathApi = new XercesXPathApi();
        Element element = xpathApi.findSingleElement("...");
    }
}

This version is a lot better than the last, the implementation is abstracted behind an interface allowing us to swap implementations easily. But what are the downsides?

Untestable - We still can't drop in a mock in place of our implementation to test that we're doing the right thing.

IoC/Dependancy Injection Simplest

To make this even more adaptable and easily testable, we can allow implementations to be dropped in using IoC or dependancy injection (we'll use the constructor form to make it easier).

interface BetterXPathApi {
    Element findSingleNode(String xpath);
}

class XercesXPathApi implements BetterXPathApi {
    Element findSingleNode(String xpath) {
        return XPathAPI.findSingleNode(xpath);
    }
}

class ElementFinder {
    BetterXPathApi xpathApi;
    ElementFinder(BetterXPathApi xpathApi) {
        this.xpathApi = xpathApi;
    }
    void findElement() {
        Element element = xpathApi.findSingleElement("...");
    }
}

What we've done is decouple our construction of the implementation with its use, delegating the construction to some other entity. So while this may seem more complex, it allows us to drop in implementations to suit our needs for example during unit testing or at runtime using an IoC container.

So now we've gone from one class to three and increased the lines of code from five to seventeen. But we've loosened the coupling between our code and the external API we're linking against, improved its testability and clarified its intent. So while this may not be the simplest simplest code, it's worthwhile taking a little extra time to build in some things which increase the health of the code.

JavaDoc Problems

2005-07-19T14:16:00.000+10:00

Seems like the 1.5 version of JavaDoc won't generate documentation for classes that have package local (default) scope and are not referenced by source included in the JavaDoc command (i.e. they're only referenced in test code). Bug or feature?

Ping Pong Development

2005-07-15T09:42:00.000+10:00

Back in my Suncorp days (sounds like the far past doesn't it ;), we did "XP with the notches turned up to 11." One of the really good practices we used was when pairing, we'd have one developer writing the test, and the other implementing the production code. This works well if the code writer tries to be as painful as possible by writing the absolute simplest code, thus forcing the test writer to improve the tests. And the cycle goes on...

So it seems that this technique actually has a name, it's called "ping-pong development". Mike Spille discusses it in Action Hippo Rangers! as well as some other things I've noticed with 180 degree turns. While I don't see the sense in going to the extreme he cites, the ping-pong nature of paired development works really really well. As for the 180 degree turns... well, I've seen both some good and some harm come from it.

JRDF 0.3.4 Released

2005-07-13T10:49:00.000+10:00

Andrew has released JRDF 0.3.4. This release includes the updates I've been working on to integrate an SPARQL parser (and also some cleaning up of the build process). The current plan is to create a parser based on a dummy query layer in JRDF. Someone will obviously have to write a real layer, or, I've been thinking of plugging into something like ARQ (the new Jena-derived query engine). This should also enable easy integration into Kowari, where I'd originally planned to write the SPARQL parser.

There is no best practices...

2005-07-11T12:46:00.000+10:00

James Blach writes about there being no best practice:

Dear Reader, I would like you to give up, henceforth, the idea of "best practice." Thank you. I want to stamp out "best practice" for several reasons...

I think in general when people say best practice they mean a practice that has been known to work within a given context, well I know I do. So it's technically true that there is no canonical "best" practices.

Patterns of Software Craftsmanship

2005-07-05T20:07:00.000+10:00

Patterns of Software Craftsmanship looks like a promising site for those interested in staying in coding, something I'm not having that much luck with lately. I've worked with a few people like this recently, and in general they seem to be very comfortable staying in a coding role, some actively refusing roles and promotions.

[httpRange-14] Resolved

2005-06-28T20:19:00.000+10:00

Well this clears it up: [httpRange-14] Resolved! Seriously, this is common sense, let's hope it sticks.

Kowari: A Platform for Semantic Web Storage and Analysis

2005-06-28T20:14:00.000+10:00

Paul, Dave and my Kowari paper has finally been published (officially that is).

Large-scale Semantic Web applications require large-scale storage of Resource Description Framework (RDF) information and a means to analyze that information via the Web Ontology Language (OWL) in near real time. The Kowari Metastore was designed as a purpose-built RDF database to fulfill this requirement. Kowari provides a scalable, transaction-safe storage infrastructure for RDF statements and an expressive query language for their analysis, with or without the use of a subset of the RDF Schema and/or OWL languages. OWL Lite plus the full cardinality constraints from OWL Full are currently supported via the interactive Tucana Query Language (iTQL) or the Simple Ontology Framework API (SOFA). The native quad-store indexing scheme used in Kowari has been shown to scale to hundreds of millions of RDF statements on a single machine. Kowari is an Open Source project licensed under the Mozilla Public License, version 1.1.

This is an overview paper, intended to give a good overview of the workings of Kowari without going into too much detail. This is good timing, as Andrew and I have been working on JRDF updates recently, including adding SPARQL support.

Flame on!!!

Keith's Law of System Updates

2005-06-28T20:09:00.000+10:00

Well, wouldn't you know it, our performance report was almost finished and I decided to run a system update: $ sudo yum update. Of course, I should have listened to Keith's advice on the matter.

Keith's Law of System Updates: Don't be the first to update your system when new software becomes available.

So I ended up hosing my system 3 hours before we were due in a meeting with the business. Luckily, there's a few spare machines at work, so all was not lost. I'm now thinking that the move to a more supported (by our sysadmins as well as RedHat) distribution of Linux won't be that bad after all.

The old way of building software

2005-06-15T11:48:00.000+10:00

In the not so distant past, developers built software without tests, without refactoring and consequently without hope. The company I currently work for has inherited one such piece of software, which we're now tasked with developing a plan of attack to fix it. I can't begin to explain the number of bad decisions that were made in developing this software, they range from architectural, to code, to build and deployment. From where I sit, most of the problems stem from a lack of process.

The last project I worked on used the notion of "going dark". This is where you spent a considerable length of time going off on your own, without using the test, code, refactor cycle (with regular commits). I've seen this (and used it myself) for most of the time that I've been developing software, and had previously thought it the norm. This style of development lends itself to isolated development, lack of knowledge transfer, integration hassles, non-testability and poor code. As this is the norm in the software industry, it's not surprising that it isn't questioned and other methods are not considered. I certainly wasn't aware of another way until very recently.

However an alternative does exist, and it's not that hard to achieve. This alternative involves short development cycles, doing a small amount of tested fully refactored work, and regular commits. There's no magic involved, and it lends itself to fully tested code, constant integration, knowledge transfer and robust code that is easy to update as requirements change (which they always do). Of course if you throw pairing in on top, you get even more of the benefits, however judging by recent experiences this is something that managers and most developers are not ready for, even if it produces excellent results.

Apple Intel Summary Posts

2005-06-10T10:50:00.000+10:00

Once again, Ars Technica provides a great summary of the Apple to Intel news.

A Guide For The Un-Initated To Buying Guinness In An Irish Pub

2005-06-10T10:15:00.000+10:00

Choose your pub carefully. A pint of Guinness does not appreciate loud music, loud people or bright flashing lights.
Ask politely for a pint of Guinness. Depending on the pub, it is possible to catch the barmans eye and mouth the word "pint", he will translate this accurately.

The barman will fill the glass between 70% and 80% capacity. It will then be put to the side for a few moments to allow it "to settle". Once the brownish liquid has almost turned to a solid black the barman will then fill the rest of the glass. NB: do not under any circumstances take the glass before it is filled. Some virgins seem to think that the settling stage is the final stage and walk away with an unfinished pint. At this point we Irish DO understand the predicament, but I assure you it causes endless mirth as well.

Once you have received your pint, find a comfortable stool or seat, gaze with awe into the deep blackness, raise the pint to your mouth and take a large mouthful. Be firm.

A good pint can distinguished by a number of methods. A smooth, slightly off- white head is one, another is the residue left on the inside of the glass. These, surpise surprise, are known as rings. As long as they are there you know your're okay. A science of rings is developing - the instance that comes to mind is determining a persons nationality by the number of rings (a ring is dependent on a swig of Guinness each swig leaving it's own ring). An Irishman will have in the region of 5-6 rings (we pace ourselves), an Englishman will have 8-10 rings, an American will have 17-20 (they sip) and an Australian won't have any at all as they tend to knock it back in one go!

As you near the end of your pint, it is the custom to order another one. It is a well known fact that a bird does not fly on one wing.

Link: http://www.ivo.se/guinness/beginner.html.

Monty Python Semweb Jokes

2005-06-03T13:41:00.000+10:00

I'm always up for some Monty Python take offs, Dave Wood mentions a couple. Semantic Knight and The Dead Spec Sketch.

Quick links

2005-06-03T08:17:00.000+10:00

Publish and Subscribe Using C++ and the Observer Pattern: Separation of concerns is increasingly on the programmer's radar. Given the growing range of data access products and platforms, it is now essential to separate data producers from data consumers. The observer pattern provides a simple but powerful model for achievin
The Corporate Developer: Been seeing a lot of this lately where I currently work...
Web Services Invocation Framework (WSIF): Looks like an interesting way to expose services.

Small tidbits

2005-05-31T13:29:00.000+10:00

Trolltech Secures $6.7 Million in Series B Investment Led by Index Ventures

2005-05-30T18:59:00.000+10:00

This is great news: Trolltech Secures $6.7 Million in Series B Investment Led by Index Ventures.

Trolltech®, provider of leading technologies for Linux and cross-platform software development, today announced the completion of a $6.7 million Series B round of financing led by Index Ventures. Existing investors Teknoinvest and Northzone Ventures also participated in this round. The funding will be used to fuel Trolltech's global expansion as it capitalizes on the growing markets for embedded Linux and cross-platform desktop applications.

Not least they have an office in Brisbane, where I've thought of applying for work in the past. Scott Collins has additional information.

vi Reference Mug

2005-05-29T20:46:00.000+10:00

Finally, these appear to have come back on the market. PI had a couple of these years ago, but both were dropped and broken.

Backups with Burn Folders

2005-05-29T13:38:00.000+10:00

I have a two step backup strategy for my Mac, 1) I create a complete bootable clone of my hard disk to an external FireWire disk using Carbon Copy Cloner and 2) I create smaller backups of important data to DVDs. I've used a bunch of different methods to create the backups and burn them, including the built-in command line tools to manipulate disk images. However, I've never found any technique that I really like, or is quick enough for me to create these regularly and automatically (my core data is usually around 10Gbm so a lot of manual deletion is usually in order).

Tiger however includes the new Burn Folders, which are essentially a folder full of symlinks to stuff you want to burn. Fill it up, check its size (Command-I) and burn!

New kinds of databases

2005-05-25T12:41:00.000+10:00

The ACM is hosting an article about forms of databases other than SQL: Beyond Relational Databases. Tucana dealt with this issue for a few years, so the point that "relational" databases may not be the only answer is not new, howeve this article gives it good treatment.

Web Services are easy

2005-05-25T08:41:00.000+10:00

Oh dear... After my recent experiences with a auto-generated web services from MS-land, this article really worries me. What worries me the most, is that people will think creating web services are easy, and then harass poor developers when they don't build a service in a few minutes.

To repeat, for anyone who hasn't heard, web services are not hard to create, what is hard is getting them right, removing the squidginess and making them integrate. They're not a panacea for integration woes...

Semantics in action

2005-05-18T12:04:00.000+10:00

Dave Wood is in action in his new role at the MIND lab. Read the interview. Quite a high level overview of the semantic web, but quite good.

Code Formatting

2005-05-16T13:39:00.000+10:00

Some links to keep in mind while reading the code standard of my new workplace.

Dictionary.app

2005-05-15T21:18:00.000+10:00

It appears that the dictionary used in applications (within text fields, etc.) for spell checking is not the same as the "dictionary" used by Dictionary(.app). The word "preemptively" is flagged as being incorrect, the suggestion is "pre-emptively". However, a quick search in Dictionary shows:

preemptive
adjective
serving or intended to preempt or forestall something, esp. to prevent attack by disabling the enemy : preemptive action | a preemptive strike.
• relating to the purchase of goods or shares by one person or party before the opportunity is offered to others : preemptive rights.
• Bridge denoting a bid, typically an opening bid, intended to be so high that it prevents or interferes with effective bidding by the opponents.
USAGE See usage at peremptory.

(I like the example of a preemptive strike, quite topical considering the news coming out of the US on Iraq.)

Also, the Command-Control-D shortcut for invoking Dictionary doesn't like hyphenated words. And another thing, double clicking on a word in the full dictionary (i.e. the app not the popup) takes you to the entry for that word, very nice. Seems like it's using WebCore for the rendering, a copy and paste to get the definition above gave me some words in lowercase, even though they're in uppercase in Dictionary, my guess is CSS to the rescue.

Axis 1.2 RC1

2005-05-11T11:25:00.000+10:00

OK, this release of Axis is worse than we previously thought. We're in the process of changing the name of an operation, and thought the easiest way to manage the change was to bring in another operation (being the new name), switch the tests over to the new operation, then remvoe the old operation. Simple.

Turns out Axis doesn't like two operations on the same port (my teminology may be wrong) with the same parameters. It generates code just fine, but ignores the SOAPAction header, selecting the first operation defined in the WSDL, instead of the one specified by the action header.

Nice one.

Update: Turns out that the I cannot reproduce this again on a different machine. We've changed the WSDL subtely (namespaces and element names), again which scares me.

A better WSDL

2005-05-10T20:36:00.000+10:00

The project I'm currently working on is using Web Services as the integration point between itself an the outside world. After using it on a few previous projects, I once had no problems with WS, but have now come to look at it in a new light. There's a good adage that we've adopted, "if you have a problem and you pick XML as a solution, you now have two problems". I think the same sentiments can be levelled at WSDL.

We've found the technology we're using (Castor/Axis/XSD/etc.) to be "squidgy", such that it sometimes works and sometimes doesn't, failing silently sometimes, working other times when it should fail (for example allowing an operation to be invoked with no message, when the WSDL clearly states otherwise). We've taken steps to sure it up (such as hooking payload validation into the backend, but we're constantly butting heads with it, so much so, that we've developed a mini-methodology for working with changes to the WS interface (no such method required for Java integration points, they have their own niceties :).

I have no problem with WS in theory, sending XML across HTTP seems like a great way (albeit heavyweight) to do integration, what I do have a problem with is the current state of the standards and the tools (thankfully most are OSS so they can be improved readily).

Yes, we could have used a REST-based Web Service, but unfortunately bigger organisational issues prevented us from doing so. Still, it's not really the transport mechanism that is broken, much comes down to the way you declare your service, typically done using WSDL.

So it didn't really strike me as a surprise when I found that Tim Bray is also replacing WSDL with something better. He poses two options, his own SMEX-D and another, NSDL. Now, I'm yet to read either, but they both sound quite promising.

Update: XINS - XML Interface for Network Services seems to be another candidate in this simplification effort.

FileVault

2005-05-10T20:30:00.000+10:00

Tim Bray is telling us that FileVault slows down your system 4x on writes... Hmm, not good, I'd just decided to give it a go on the new Tiger install on my PowerBook. After confirming the comments that I'd heard about IntelliJ being faster on 1.5 - I'm obviously not getting an apple-for-apple comparison - it seems it would be faster still if FileVault were turned off. Most of the observed slowness is in compilation before running tests, which writes quite a few files to the file system.

Test before vs. test driven

2005-05-04T20:51:00.001+10:00

I've been seeing a lot of articles lately about "test driving" code when developing software. This has appeared most lately in an article on ADC, but also - as I've been looking for work recently - during interviews. Companies I've worked for have always written tests before they write code (or at worst, just after). This is often passed off as test driven code. However, it isn't. Test driving means writing a very simple test first, that forces you to write the simplest code that will pass the test.

So for example, suppose you want to create a class called Xxx. To do this, first create a class called XxxUnitTest. Add a method called testCreate():

public void testCreate() {
  assertNotNull(new Xxx());
}

What this does, is force you to write the implementation class. You can follow the same simple pattern for driving out member methods, the test doesn't need to stay the same, it can change dynamically.

public void testCreate() {
  Xxx x = new Xxx();
  assertEquals(“bar”, x.foo());
}

We've now driven out the method foo(). Repeating this process, we can drive out all the implementation, safe in the knowledge that we have pretty good test coverage.

If you always write the simplest code to pass the test, sometimes you end up needing to triangulate to drive out the desired behaviour. So for example consider the following:

public void testFoo() {
  assertEquals(“bar”, new Xxx(“bar”).foo());
}

This code implies that the constructor will store the string passed in a member field, then return it when getFoo() is called. However the simplest code that will pass this test is something like:

class Xxx {
  Xxx(String bar) {}
  getFoo() {
   return “bar”;
  }
}

This code passes the test, however, it doesn't express the intent implied in the unit test. So now, you need to include a new test that checks a different argument.

public void testFoo() {
  assertEquals(“bar”, new Xxx(“bar”).foo());
  assertEquals(“baz”, new Xxx(“baz”).foo());
}

So the simplest code that will pass the test will store the parameter passed into the constructor, and return it when getFoo() is called. Of course, this test lends itself nicely to a little refactoring to reduce the duplication in the test...

If we were to test before we wrote this class, we would potentially build up a large chunk of test code, that will contain a lot of compile errors, meaning the code is a long way from being complete at any point in time. We may also decide, once we write the code, that the test is not correct, as we learn something by writing the code that we didn't know beforehand. By keeping the tests and the code in tight sync, you minimise the amount of time spent in the wilderness, and ensure good test coverage.

This approach lends itself nicely to pairing. The pair can work off each other, one writes the failing test, the other does the simplest thing to make it pass, and the cycles continues. This shares the keyboard around, and introduces some good natured competition, usually producing better code than the traditional pairing approach of driver and navigator.

Up and racing

2005-05-03T21:30:00.000+10:00

Well, after several months of being offline, due to the international move, my blog is now back. I shall be retrofitting all the old content again (for posterity) sometime soon, with more to come. Still not sure about this hosted blog thing, but seems to be all the rage...

Screamfeeder Videos

2004-11-24T20:48:00.000+10:00

Continuing the Screamfeeder posts of late, I just found a collection of videos. Very nice indeed!

Biblical Literalists

2004-11-22T21:04:00.000+10:00

Only about a third of Americans believe that Charles Darwin's theory of evolution is a scientific theory that has been well supported by the evidence, while just as many say that it is just one of many theories and has not been supported by the evidence. The rest say they don't know enough to say. Forty-five percent of Americans also believe that God created human beings pretty much in their present form about 10,000 years ago. A third of Americans are biblical literalists who believe that the Bible is the actual word of God and is to be taken literally, word for word.

Source: Third of Americans Say Evidence Has Supported Darwin's Evolution Theory.

Introducing Screamfeeder

2004-11-16T20:44:00.000+10:00

The new Screamfeeder album "INTRODUCING: SCREAMFEEDER | Singles and More 1992 - 2004" just arrived as a birthday gift from Janelle's parents. It's great to get a handwritten note from the band to accompany it. Now this doesn't surprise me as they are a Brissie band! Screamfeeder are an interesting band, they've been a favourite ever since I saw them with Pete at a Big Day Out on the Gold Coast in the mid nineties. I also have some bazaar connections to them. A good friend of mine Kent tells me he used to hang out with them in the early nineties (along with Powderfinger et al.) and my buddy Keith is good friends with Tim's (singer/guitarist) brother. It's very refreshing to know that you can support a "local" band (not that local to me anymore) from half a world away. Actually, listening to the songs makes me quite homesick...

Kowari Paper for WWW2005

2004-11-10T21:03:00.000+10:00

David Wood, Paul Gearon and I have just completed a paper on Kowari entitled "Kowari: A Platform for Semantic Web Storage and Analysis" for the 14th International World Wide Web Conference (WWW2005) next year in Japan.

Large-scale Semantic Web applications require large-scale storage of Resource Description Framework (RDF) information and a means to analyze that information via the Web Ontology Language (OWL) in near real time. The Kowari Metastore was designed as a purpose-built RDF database to fulfill this requirement. Kowari provides a scalable, transaction-safe storage infrastructure for RDF statements and an expressive query language for their analysis, with or without the use of a subset of the RDF Schema and/or OWL languages. OWL Lite plus the full cardinality constraints from OWL Full are currently supported via the interactive Tucana Query Language (iTQL) or the Simple Ontology Framework API (SOFA). Kowari’s native quad-store indexing scheme has been shown to scale to hundreds of millions of RDF statements on a single machine. Kowari is an Open Source project sponsored by Tucana Technologies and is licensed under the Mozilla Public License, version 1.1.

It's basically an overview paper containing an explanation of the components that make up the system focusing on the query engine, the resolver framework and the XA statement store. So it's very much like similar papers on Sesame and the 3store. We're pretty hopeful it'll be accepted, we think we needed to provide a good overview of this important piece of Semantic Web infrastructure. (Update... The paper wasn't accepted to the stream we submitted it to at www2005, but was accepted to xtech 2005).

Systems

2004-11-03T21:01:00.000+10:00

The recent election here in the US had me thinking about some of the problems that I've observed with the systems that operate here. The electoral process is my first target here. It amazes me that a country as wealthy as the US, still struggles with things that I would have thought were solved problems. The electoral process here is really screwed up, a fact that I'm starting to see highlighted more and more in the news media. For a start, there's the antiquated electoral college system. I'm not 100% sure of its origins, but I imagine that it would have made sense a couple of hundred years ago, but seems a bit silly to an outsider in this day and age. Personally, I find the Westminster system of the party that wins the most seats winning government the easiest to understand. The voting system here is also weird. The person who gets the most votes wins, regardless of whether they have a majority. In Oz, we have preferential voting, which basically means that people vote in order for who they like, and the candidate who gets the least votes, gets their second preference re-allocated. This goes on until someone gts a majority. Our system appears very similar to what is know here as Instant Runoff Voting (IRV) (a much sexier name than "optional preferential"). My second target is the general law enforcement system. Back home in Oz we basically have two police forces. Each state has a police department that looks after all matters of policing and there's also a federal police force, that takes care of cross-state issues, drug running, border enforcement and lots of stuff I probably have no idea of. Contrast that with the US. In the Northern Virginia area, we have lots of different police forces, seemingly unable to cooperate. We have park police, state police, town police, highway police, airport police, tollway police and I'm sure there's at least 14 more that I've missed. So when I got my car insurance card in the mail the other day and I read that the first step in reporting an accident was to call the police, I thought "which one?". Given the number of separate agencies, is it no wonder that the events that lead to 9/11 happened here. America is full of minor bureaucracies that all compete for power at the expense of the people they purport to represent. Surely this is a solved problem too.

Bush on Vietnam

2004-10-22T21:28:00.000+10:00

Heard a good one today from a buddy, who in turn read it on Eric Alterman's blog.

What's the difference between Iraq and Vietnam? Bush had a plan for getting out of Vietnam.

On a similar note, kottke.org is pointing to this nice jem about Kerry and Bush being distant relatives.

Beverage

2004-09-24T20:56:00.000+10:00

Told myself that I'll write down all my observations on the US while it's still slightly novel. I've found that I've started to forget many of the things that I found so bazaar when we first arrived. Here is my latest observation... Was at Starbucks yesterday afternoon to grab a quick coffee while waiting for Janelle and her Dad to drop off a rental car. The line was really slow, so by the time you got to the front the cashier had forgotten the order. So he started asking people "What was your beverage?". I think this is a good reflection of American culture. Instead of saying toilet here, they say bathroom, as if the daily necessities of life are somehow a bad thing. I see this as an example of the effect of nice-ening up everyday experiences to the point of nausea. So instead of using the word drink or even coffee (for what was plainly a coffee), he chose to sugar coat the word to make it (subjectively) more palatable.

RDFlets and Kowari Descriptors

2004-09-23T20:52:00.000+10:00

Saw an interesting post yesterday on the DAWG list from Alberto Reggiori at @semantics. He talks about a technology they've developed on top of RDFStore called RDFlets. Very cool... At a first glance, these look very similar to our descriptors, which are XSL stylesheets that allow the execution of iTQL queries, getting the results in XML, and the massaging of the results using XSLT. When Keith and I initially designed them a few years back, Simon came to us with the idea of applying a transformation to an RDF datastore, in an analogous manner to the way transformations are applied to XML files. This is the way I like to think of them, although others like to explain them as stored procedures. And of course you can pass whatever parameters to them you like, and use them in whatever way XSL allows variables to be used. As the documentation states, descriptors really consist of:

Extensions to XSL: 1) a <query> tag to issue iTQL commands from within XSL stylesheets; and 2) a lt;descriptor> tag to allow one descriptor to call another;
Java objects that invoke descriptors and return their results;
Java Servlets that invoke descriptors using HTTP;
URL addressable SOAP endpoints.

So, how do RDFlets and descriptors stack up (points stolen from K...)?

RDFlets don't seem to be integrated into XSL so they require a container of sorts, i.e Apache using SSIs;
RDFlets don't allow for the cascading of queries like descriptors do (unless SSIs give you this for free). So say we're querying metadata on a document, you can use a top level descriptor to get the top most document properties, then call off to other specialised descriptors for each of those properties. This lets you get optional properties back very nicely.
Descriptors expose their interface (parameters passed in and out) in a queriable fashion, as it's defined in RDF. This lets you do cool things like "show me all the descriptors that work against RSS feeds and return XHTML".

Of course no technology is perfect, and descriptors certainly have their faults, not least of which is trying to explain to people what the hell they are...

John Kerry

2004-07-29T21:27:00.000+10:00

Well, watching the Kerry speech, he's just about to arrive. They've been making a big deal about his war record, extolling it on one side, and ridiculing it on the other. Frankly I think one of these sides is attempting to create something from nothing...

Bush is at it again

2004-03-25T21:25:00.000+10:00

Yeah, everybody has seen it, but this guy never ceases to amaze me.

Administration sources tell TIME that employees at the Department of Homeland Security have been asked to keep their eyes open for opportunities to pose the President in settings that might highlight the Administration's efforts to make the nation safer. The goal, they are being told, is to provide Bush with one homeland-security photo-op a month.

Mac OSX for Geeks

2003-07-17T21:30:00.000+10:00

Forwarding Address: OS X has a link to a great set of tips for unix geeks new to OSX.

Creation Science

2003-07-10T21:22:00.000+10:00

This is pure gold... Someone at work pointed me to the Creation Science Fair. It has a boatload of great stuff, from showing that "Pine Cones Are Complicated", hence proving the existence of God, to the three week long, scientifically rigorous "Life Doesn't Come From Non-Life" experiment. I've seen a couple of documentaries on these nutters, and it still amazes me that these people truly believe these things. They even have a latest newssection, detailing the latest advancements in creation science, more gold. And I just found that Apple is anti-Christian and a cult:

However, these propagandists aren't just targeting the young. Take for example Apple Computers, makers of the popular Macintosh line of computers. The real operating system hiding under the newest version of the Macintosh operating system (MacOS X) is called... Darwin! That's right, new Macs are based on Darwinism! While they currently don't advertise this fact to consumers, it is well known among the computer elite, who are mostly Atheists and Pagans.

There's a great deal more on this site, it's kept the guys here entertained for a few hours already.

Quote of the day

2003-03-24T21:20:00.000+10:00

Someone on a mailing list I'm on wrote this nice little summary of how a lot of people are feeling at the moment:

Now we see a pyrotechnic display on TV that is the war as we are allowed to witness it. But THAT is not the war. The real war we never get to see. We don't get to see magled bodies, we don't get to smell rotting, burned flesh, we don't see the pain of the families of killed Iraqi soldiers. No, the in our war the people that get killed all somehow deserve it, as we are good and therfore they must be evil.

The whole thing makes me fucking sick.

Americans look elsewhere for news

2003-03-18T21:17:00.000+10:00

I've been telling Keith about how I've noticed that a lot of American bloggers (James Duncan Davidson in particular) are looking at blogs to get their news fixes. I've considered that this stems from US news networks/sites/etc. not objectively reporting the current Iraq "crisis". Of note, Andrew pointed me at an article outlining the way the current US administration is using the media to push its agenda (I'd also heard of recent press conferences where the Bush camp handed out pre-baked questions, and only called for question from reporters who were willing to ask the questions). Looks like wired have picked up on the thread, and have posted an article on the subject. What is interesting to me (apart from the obvious) is that the ABC news site is getting a lot of hits from US-ians looking for objective coverage. buzz.weblogs.com has more articles on the matter. In fact this site looks rather scathing... an interesting read!

North Korea

2003-02-07T21:16:00.000+10:00

Looks to me like North Korea is pulling the piss out of the US again. They're now saying that they'll launch a pre-emptive attack if the US places more troops into the region. I'm not entirely sure if the North Korean government is taking the US' tactics as an example and is serious (which they probably are), or whether they're just exposing the hypocrisy of the current US Administration's stand on Iraq.

Senate passes no confidence vote

2003-02-05T21:14:00.000+10:00

Well it looks like the senate has finally passed the no confidence vote on Johnny's handling of the Iraq "crisis". The vote was carried 34 to 31 with the support of the opposition, the Greens and the Democrats.

Map of Terror

2003-01-31T21:12:00.000+10:00

Just found this Map of Terror. The site is that of a fellow Aussie who seems to have the same take on this whole "War on Terror" thing as me (ie. it's a joke). On a similar note I saw a large tow truck the other day with a perspex "Infinite Justice" sign mounted on the front of the bonnet. Of course he had the obligatory US flag to hide behind mounted on the rear of the cab, as well as a large eagle.

Bush coverup

2003-01-28T21:06:00.000+10:00

Well seems like good ol' George has done it again (ie. something stupid), this time he's re-labeled a bunch of boxes stamped "Made in China" with "Made in America" (thanks Scot). Of course this wouldn't happen in Australia, we're far too un-patriotic for this kind of behaviour, or so says a random interviewee in a story aired over the weekend on the closing of Old Sydney Town - something along the lines of "They wouldn't stand for this in Ameria. They love their country and wouldn't let their history go down the drain". While I can support the need for this kind of history to be preserved, there's a lot of other history that needs to be sorted out also - eg. the stolen generation. To take this comment out of proportion, I'd hate to see us taking a more US-styled approach to everything we do (which unfortunately seems to be the approach we're headed down under the current government). I watched an old western on the weekend (Colt .45) staring Randolph Scott as the gunslinging hero, and began to think that maybe ol' Georgy boy considers himself such a hero, rescuing the town and getting the girl. I just hope America doesn't wich too fondly to be back in the good old days of the wild west where bad guys were hunted down and hung. One scene in the movie - where two gang members are brought into town by a deupty and the townsfolk demanded a hanging - reminded me of the current world climate. Hopefully common sense will prevail and see civil liberties retained instead of being erroded in the name of the "War on Terror".