There was a column a few months back on CMCrossroads on tool selecting, and these thoughts escaped our Agile SCM column for various reasons. Meanwhile, Brad has commented on his blog, so I thought I would weigh in too!
Agile SCM requires minimal disruption of flow to developer�s lives, and thus tools that help, not hinder, this process. The right processes are obviously key to effective and efficient development and it would appear that if you have a good process, then the more you can enforce it the safer life will be. However, as we wrote inThe Illusion of Control, too much safety leads to much reduced productivity.
As Joel Spolsky writes regarding defect trackers and enforcing process:
Historically, I am opposed to custom fields in principle, because they get abused. People add so many fields to their bug databases to capture everything they think might be important that entering a bug is like applying to Harvard. End result: people don’t enter bugs, which is much, much worse than not capturing all that information.
Best of Breed vs Integrated Suite?
This is a classic conundrum and there will always be good arguments for both sides. Indeed it is not possible to come down on one side or the other without knowing the details of a particular organization and all the nitty-gritty requirements.
However, let�s consider how some ideas might help us in our decision making process.
Who Owns the World?
Many tools make the mistake of thinking that they own the world � well at least the environment that they are going to run in. The user (developer) is going to be safely cocooned inside the wonderfully productive environment of the tool and never going to have to leave for the big bad world outside. Thus the vendors make little attempt to provide any interface to the outside world.
Now obviously considering that some things are �beyond the pale� has some advantages. Unfortunately the disadvantages are often considerable.
Over the years many vendors have come up with wonderful development paradigms, development environments, 4th generation languages and similar which were indeed very productive. Often these environments included (very) rudimentary version control. The problem is that they implemented it badly (and often unreliably) and provided no hooks to external SCM tools which were used elsewhere in the organization.
This is a bit like the XP principle of YAGNI and refactoring as opposed to big upfront design. As Martin Fowler writes in Is Design Dead?:
People aren’t good at anticipating, so it’s best to strive for simplicity. However, people won’t get the simplest thing first time, so you need to refactor in order get closer to the goal.
Thus an attempt to anticipate everything developers are likely to need would seem to be rather difficult.
The rise of Eclipse which is making the basic environment a commodity with a good design for plugins and extensibility is perhaps a result of this worldview. Companies such as Borland are now repositioning previously standalone components such as J-Builder as add-ons to Eclipse rather than as competition.
This is somewhat similar to the arguments for performing builds using a general purpose language. The grand daddy of them all, Make, has its own arcane syntax and issues which have evolved and been tweaked and extended to try and solve the ever expanding set of problems posed by building systems.
In Martin Fowler experiences with Rake (a Ruby based build tool) he suggests:
The fact that rake is an internal DSL [domain specific language] for a general purpose language is a very important difference between it and [make and ant]. It essentially allows me to use the full power of ruby any time I need it, at the cost of having to do a few odd looking things to ensure the rake scripts are valid ruby. [�] Furthermore since ruby is a full blown language, I don’t need to drop out of the DSL to do interesting things – which has been a regular frustration using make and ant. Indeed I’ve come to view that a build language is really ideally suited to an internal DSL because you do need that full language power just often enough to make it worthwhile – and you don’t get many non-programmers writing build scripts.
My personal current favourite for such tools is Scons (written in Python) which has a similar approach that I have found to be very powerful.
Advantages of Integrated Suites
Of course integrated suites can be a big win if your problem domain is sufficiently close to what the designers had in mind. In addition, if the suite and its existing processes can be tailored easily to your requirements, then of course you should rate it highly in your selection criteria.
But you need to be careful. There�s an awful lot of �shelfware� out there consisting of tools sold as a �silver bullet� and never really used. The psychology seeming to be that if you pay enough money then the problem will be taken away from you.
Wolf Suites In Sheep�s Clothing
Then we have the category of suites that purport to be integrated but under the covers all is not what it first seemed. The classic example is of different tools which a company acquires by buying the original vendor, and with a lick of paint and a wave of the magic wand over the marketing materials suddenly has an �integrated suite�. The challenges of integrating tools not designed to do so are considerable and it can take many years for good integration to happen (if it ever does).
Customizability
Is also a two-edged sword in that you can spend all your time customizing the tool and not enough actually doing the work.
As has been noted by various people on CMCrossroads, workflows implemented by scripting can have a cycle of �script a little, test a little, repeat�. This would suggest that tools which offer the ability to design workflows graphically are immune from these problems. However, that is often not the case, and has been pointed out before, some of these workflow designers are in fact very difficult both to change control and to debug. A complicated workflow is an inherently hard problem to both comprehend and manage (which does point towards keeping it as simple as possible).
The key area of customizability is that of being able to link the tool (or suite) to the external world in the form of other tools. Thus providing sensible hooks to allow third party tools to link in would seem ideal. And yet this can be rather difficult to do in practice, and thus gets pushed to the back of the queue by the vendor. In addition, I suspect there is often a business rationale that they think if they don’t make it easy to link to external tools then the user will be forced to stick with the vendor’s tool.
It’s not quite on topic, but I could resist commenting on a classic example of third party hooks that don’t work very well – Microsoft’s SCC integration to Visual Studio .Net. This is based on a lowest common denominator API which used to be released under NDA but is now relatively freely available. It worked reasonably well with Visual Studio 6, but was totally re-implemented for Visual Studio .Net and rather badly it seems.
Conclusion
Thus I would personally tend to come out on the side of linking best of breed tools together rather than going for an integrated suite, since my experience is that integrated suites don’t try hard enough to provide clean interfaces to the outside world. Of course, some best of breed tools make integrations with other tools a bit like teaching a pig to sing – “frustrates you and annoys the hell out of the pig”!
So the real answer is that every potential customer for SCM tools needs to draw up their own requirements and evaluate against them. Delaying commitment by avoiding lock-in is a very valuable feature, but it doesn’t always pay off as it should perhaps in theory!
Do not turn brain off when choosing tool! An ounce of requirements analysis and evaluation is worth a ton of tweaking down the track.
