NIH

Before I knew what National Institutes of Health was, before I moved to the US, NIH stood for "Not Invented Here."

The reappearance of this phenomenon on my radar is coincident with my deepening involvement in the Agile/XP world. Coincidence? I sure hope so. So step forward all you XPers and agilistas and tell me that you are at least as allergic to NIH as I am.

In the last month I have spent hours (with a pair) figuring out why http session management is broken. I have spent days (with various pairs) trying to outwit a half-baked Hibernate session management infrastructure. Today I spent several hours (with a pair) getting some file manipulation utilities to work.

Even back in the days of Struts 0.5, session management was pretty damned solid. And the Spring-like management of Hibernate sessions by tying them to a request/response cycle (in the simplest case) was hashed out years ago. And the Apache commons I/O library is pretty mature at this point. Hell, it's already on our classpath!

So how did this mess come to be? The answers I got in the past were vague hand-waves along the lines of "well we needed something simple and lightweight so we built our own."

Yeah, yeah, "the simplest thing that could possibly work." But surely the definition of "work" also includes "does not require constant repair and debugging"?

Maybe it's just innocence. Maybe those people really believe that an app framework is "too big" and so building out a Struts-lookalike from scratch is excusable. Maybe they really believe that they just know they will never need to manage Hibernate sessions with a broader scope than a single call. Maybe they really believe the file manipulation is just a quickie, undeserving of a library.

To me the simplest thing that could possibly work is to use code that someone else has spent years refining and is known to work.

Please tell me all this NIH is not hubris.

Labels: xp

0 comments

Migrating from FileMaker to MySQL

FileMaker Pro (FMP) has the ability to access a real RDBMS via ODBC dynamically. An "external" table can be accessed in the same way as a native FMP table. FMP calculations, normally associated with a native FMP table, can be added to an external table as "supplementary fields".

So all this sounds terribly glorious. Surely one can migrate one's FMP db to MySQL (or other) so that the data is stored in an industrial-strength RDBMS. That, naturally, allows all kinds of other enterprise apps to hit the same data as the FMP. In fact, one can think of the FMP db as a front end to the enterprise db.

I know folks who have made this work. But migrating is another story. There is no programatic interface to the FMP schema - the definitions of the tables and "table occurrences" (really aliases that allow for the definition of associations that become joins at run time). And FMP's use internally of ids rather than names for fields, scripts, tables, etc – one of its key benefits that allows transparent renaming of objects – means that once a relationship is defined using one table, the only way to switch to another (external) table is to delete and recreate it.

That spells a ton of manual labor. I'll outsource it if I can find someone.

We'll still be left with all the presentation intelligence (FMP layouts) as well as the business logic (scripts, calculated fields, portals) firmly in the FMP world. As we build a parallel implementation that does not rely on FMP, we'll be rebuilding all that stuff. Only this time we'll do it in a transparent, modifiable, refactorable way, using code that can be edited using industry-standard tools like (ooh, gasp) a text editor.

So if you're thinking of building a real enterprise app in FMP, don't. It will bite you.

And to answer my own questions from a few months ago regarding FMP in an XP world: The more I bang my head against the closed, constrained FMP model, the more I realize that its only use is as a spiking tool.

Labels: filemaker, xp

0 comments

Wired

All of my on-stage experience to date has been unplugged. That is, I have never dealt with amplification of any sort. I have sung the lead in operettas and musicals of all sorts as well as numerous musical roles, of course. All of it unamplified. But I have never – until tonight – had to sing onstage with a full band, had to deal with mic technique, had to try to follow exactly where the hell we are in the song with a monitor blaring the bass line alone...

The "Music Masti" at Agile2008 was my wired debut. Damn, it's hard to figure out where you are, to keep track, to be able to hear un-monitored instruments when you are singing in a band.

Now I have designed sound for stage productions, even mic'ing actors where needed. But now I know what it's like to be on the performing end. Who'd a' thunk it was so hard? This is what happens when you let popular culture pass you by.

Sigh. There's a lesson in this. There is no substitute for doing, even when it seems that "it" is a natural extension of what one already knows. Maybe it's the geek mystique: "How hard could it be?" But when you pick up that mic and you realize that it's actually not that self-evident,.. it's no longer an extrapolation from the known the assumed, but genuinely into the unknown, it's a bit of shock.

Regardless, it was unbelievable fun.

Rock on!

Labels: xp

0 comments

Disgruntled employees

I caught a bit of this morning's Forum on KQED (Download (MP3)) which was mostly about some civil servant in SF who sabotaged the City's computer systems, and what that means for employers and "health and security of the workplace."

The interesting bit for me was a Towers Perrin survey of 90,000 employees across 18 countries indicating "widespread disengagement".

• 21% "engaged with their work"
• 38% "partly or fully disengaged"

Also cited was a Gallup survey saying that

• 19% of employees are "actively disengaged" i.e. trying to sabotage the workforce

I bring this up because as XP evangelists we run into people who we want to see succeed and excel, but who might be dealing with the own demons. And a much-cited antidote mentioned on the show was "job rotation" and "employees training each other". So pair programming is not just good for the code, good for the dev, good for the team, good for the enterprise, but it might just stop you from going postal. ;-)

Those numbers also tell us that there is a vast amount of waste. Large numbers of people punch clocks and little else. Again, as XPers, we talk about productivity, but what does that really mean for someone who is comfortably numb?

Labels: xp

0 comments

Why FileMaker ultimately sucks

When you have a FileMaker "solution" in place, you have a large investment in UI design and usability. You might, depending on the quality of the original design, also have an investment in the database schema. Porting to another environment - say Java + Spring + Hibernate - is daunting, mostly because re-implementing the UI is such a vast undertaking.

As an XPer, I need to be able to test-drive development. The only reasonable way to do that is by using external AppleScript (or Python + appscript and maybe PyUnit too) tests. I don't know about you, but that sounds like far more trouble than it's worth. Just navigating the maze of what works as expected and what requires arcane tricks in the AppleScript world gives me a headache just thinking about it.

XPers also refactor. Not surprisingly, there are no tools at all for refactoring. Worse, the structure and syntax of FileMaker's scripting language actively discourages refactoring. For example, while it's possible to pass multiple arguments into a sub-script, they have to be packed on the caller side and unpacked by the callee - multiple lines of script. And having to drag commands into place rather than just type like a normal human...

Then there is basic source control. A FileMaker file consists of several elements, the most important being

• Tables and relationships
• Scripts
• Layouts

I need to be able to see what's changed from one version of my FMP file to another. Since all the above elements are stored in a single binary file with a proprietary format, it's impossible to do a diff. Sure, one could export a DDR XML file and diff that, but diffing XML?!

And there are indeed tools that purport to do this. Unfortunately, they are universally hard to use and slow, if you cam get them to work at all. (Of the 3 main contenders, I have only managed to get the demo version of one of them to work consistently. The others are so obsessed with license keys that they fail to work at all.)

Labels: filemaker, xp

3 comments

Selenium-RC

My "Mastering Selenium" tutorial has been approved at Agile2008. I am very excited about that.

Meanwhile, in preparation, I have been putting Selenium-Grid through its paces at Industrial Logic. We are formulating a high level lingua franca to use in describing executable Storytests. This is a big deal. It was the promise of FIT, but for one reason or another (and theories abound!) Customers stayed away from FIT in droves.

Our old Selenium tests had been constructed using a wiki language and template processor. This provided a powerful toolset with which to construct Stroytests, but we ran into 3 major issues:

1. Selenium Core is slow
2. With the HTML Selenium tests, there is no guaranteed tear-down as there is in xUnit
3. Our toolset, while providing a fair bit of power and reasonable readability, did not come close to the expressiveness of a language like Python, Java, or Ruby in Selenium-RC.

So we went full out for Selenium-RC, using JUnit to drive the tests against a local Selenium-server.

To get the ball rolling, we grabbed the HTML from an old Selenium test by simply firing it up in Selenium Core and grabbing the code from the browser. It's a trivial matter to translate it into one of the Selenium-RC languages - we're using Java - with the Selenium IDE. Some deft text editing massages the Java file into shape so that it fits into it's place in the class hierarchy. More about the hierarchy later. We found reworking these legacy Stroytests to be an illuminating process. It meant bottom-up - starting with a very long list of Selenese commands, and doing extract method on clumps of code. The original source file with its template language was found to be lacking in many cases. The tests had grown over time and had not been refactored. We found some very interesting issues:

• Code that verified conditions that had absolutely nothing to do with the story at hand. These were presumably checks that had been copied/pasted from some other test where those checks were relevant.
• Some tests attempted to do too much and veered off on a tangent.
• Once we put in place some decent setup and (dependable!) tear-down, the test body was short and to the point.

So what does the class hierarchy look like?

The immediate superclass of each Selenium test class contains domain-specific helper methods. Its superclass is essentially a proxy for the Selenium server (or a hub, which is in turn a proxy for the runners.) For that class, we started with the SeleneseTestCase that is provided in the download, but decided that we didn't like it for a variety of reasons. In particular, we needed to do some magic so that a new Selenium session (and hence a new browser) is only launched if none already exists for the current thread. That's because we exploit the good ol' JUnit 3.x suite and test decorators to run allow for parallelization of tests. Sure, we could have used testng but its seemed a little haphazard to us and that business of generating an intermediate xml file just doesn't seem right.

The next chunk of learning relates to just what is needed to get tests to run in parallel so that they don't stomp all over each other.

Labels: java, xp

0 comments

Macworld | Mac OS X Hints | Add more power to 10.5's screen sharing

[From Macworld | Mac OS X Hints | Add more power to 10.5's screen sharing]

I live by VNC, since we are a distributed XP team. So I was thrilled when I found this MacWorld article on tweaking the build-in VNC client on Leopard.

Now if only it would get over some of its temperamentality so I can use it reliably over an SSH tunnel or to a non-Mac machine.

Labels: macos, xp

0 comments

TDE - test-driven exploration

So I was geeking out last night, as Mike would say. I was writing some code to fix some corruption in my calendar. It's in iCalendar format, so I found an ical4j framework on SourceForge and fired up Eclipse.

The first thing I did was to copy some snippets of sample code from the ical4j docs into a JUnit test and see what happens. I did this for 2 reasons: (a) the docs for FOSS are often wrong, and (ii) I needed to get all the config set up (e.g. to make a test calendar with events in it containing the properties I was interested in.)

As I worked, my "production code" kept evolving inside my test class. When I had successfully TDD'ed a step, I extracted the code into a real production class. I could move code around with Eclipse's refactoring tools and I had solid tests to save my butt.

Nothing startling in any of this. The interesting thing for me was that I was exploring ical4j. It was something of a spike - I didn't know ahead of time if the framework would be up to the job - yet each success immediately resulted in moving the code to the production classes.

So what do you think of this approach of evolving the code in the test and moving it as soon as it's green?

Redux: Keith Ray says: It's good. I think Kent Beck or someone called these "Learning Tests"when learning an API via TDD.

Labels: java, xp

0 comments

The Launch Continues

One normally thinks of a "launch" as a discrete event, an atomic occurrence, something which starts and ends almost instantaneously. A product launch takes days to get out the door. It's more like childbirth. Today, Josh sent out the press releases to Dr Dobb's and other publications, email to numerous mailing lists, and the mass mailings. The launch continues.

Labels: elearning, xp

0 comments