Entries Tagged 'Tech' ↓

The other keynote at the  XPDay conference last December was Mary Poppendieck on the theme of her book Lean Software Development.

“Make More Money”

The title certainly grabbed people’s attention. The key point being that you make more money by increasing your ratio of outputs to inputs.

Productivity in software has had its peaks and troughs in recent years, whereas in manufacturing, productivity is continuing to rise (there was a plateau around 2000/01 but not it’s heading up again). Companies who do well look at their core business processes which are what drives productivity. They then chose where to match industry norms and where to lead (and they pick a few areas to excel at). They focus on end to end improvement. You lead in a decade not in a year so it’s not a short term thing. She referred to the book by James Collins, Good to Great.

Looking at the productivity levers for software:

• What are the processes involved in turning an idea into a product?
• How do you translate customer needs into software? (This isn’t what they ask for it’s what they need!)
• You need to manage your development portfolio (resources and capacity)
• Deploy complete solutions – are you fully invested in the success of your customers?
• Managed the full lifecyle – design for maintainability.

Another large lever is marketing but that wasn’t the subject of the talk.

A key way to improve your productivity is: Do Less Work

Reduce Direct Costs

Reduce the direct cost by providing only what the customer will pay for. A quote from Jim Johnson of the Standish Group: users typically use only 25% of the system – 65% of features are rarely or never used (see Martin Fowler’s writeup of XP2002).

Where do all these extra features come from?

• we ask the customer what they want
• we reward them for thinking of everything (”scope”)
• we penalise them for adding features later (”scope creep”)

She referred to the following book which talks about MMF – Minimum Marketable Feature Set (Mark Denne & Jane Cleland-Huang, Software by Numbers; Low-Risk, High Return Development, Prentice Hall, 2004). This assumes you are deploying early and often.

Reduce Indirect Costs

Streamline your processes to eliminate waste.

Streamline the core processes to provide rapid delivery of value (cost reductions that reduce customer value do not increase productivity – beware of mindless cost cutting!).

The measure of an organisation’s maturity is the speed it can reliably and repeatedly execute key processes. For software this is the speed that customer needs get turned into deployed code.

She referred to the Value Stream Mapping process in her book (see link above), and how to change your processes to remove delays.

Get support people involved in the development and design of a product to ensure that it can be deployed appropriately.

Increase Output (Value)

Do this by shortening the customer feedback loop.

Improve customer productivity – when they win you win. A case study – Dell realised after some customer visits that there was a whole new business in delivering preconfigured PCs. This increased both revenue and customer loyalty.

How can you help your customer?

• Map their value stream
• Extend the value that you offer

Conclusion

Another interesting keynote. Covered a fair amount of stuff from her book, but with some interesting twists that made it very relevant. I also attended her workshop, and some key points from that:

• to get predictable outcomes don’t predict – make decisions based on facts not forecasts
• if you can’t deliver fast, you can’t delay commitment
• speed of software development can have a bad name but is a fundamental competitve advantage
• improve depolyment by testing earlier and making rapid small releases
• to move fast you can’t tell people what to do – they have to know for themselves (”information radiator”)

I went to the XPDay conference last December where Martin Fowler gave one of the keynotes. I enjoyed the conference as a whole very much. While there were some XP zealots, I also found plenty of more rounded views which were of great interest (to me).

Martin’s talk was interesting and I took some notes which I have been meaning to write up.

There was nary a power point slide in site as he strode around the room talking to us as he went – rather like your archetypal absent minded professor. He did refer back to previous conference talks he has given – “that was when I used to plan my talks as opposed to just give a generic title and talk about my current preoccupations!”.

He started his talk by saying that he was getting a bit tired of XP (and yes he was being a bit provocative!).

He compared software development with book writing, and concluded that the process of development varies legitimately because of who is doing it rather than what they are doing (his style vs. Steve McConnell, both of whom are successful authors). Martin thinks about something and then writes a complete section on that topic. He repeats this process, and then starts refactoring the topics, out of which comes his book. Steve apparently refines an outline view down to a very detailed level for the whole book. Then he fleshes out each detailed point with a few paragraphs and is done. Two very different ways of going about it. When asked if he would co-author a book with Steve he was doubtful because their styles are so different, whereas he has successfully co-authored with Kent Beck).

How do you measure the value of software? This is frequently rather difficult since reducing some business process down from 2 days to 45 minutes is down to a number of factors, not just the software that helped enable it. Software methodologies tend to be very difficult to compare and are not really measurable. Thus relative arguments are frequently founded on sand (there is a lot of heat in online discussions to very little benefit).

So, what makes good software design? He mentioned Eric Raymond’s The Art of Unix Programming as discussing what people have done to succeed as opposed to laying out what they should do to succeed.

In his view, the XP community consider design and programming to overlap to a huge degree (growing out of Smalltalk/Lisp/Unix world and experience). Others have the experience that programming without design leads to huge problems, and therefore conclude that the XP way can’t work.

Can evolutionary design work? Absolutely, and Unix is a prime example.

Martin discussed this topic in “Is Design Dead?”. He now considers that article fine up to a point, but he realised that he had missed a major issue to do with people. This was highlighted for him by Enrico Zaninotto during XP2002 that you need something to make the process converge. Various XP practices influence this: test driven development, refactoring and continuous integration make evolutionary design possible. Things like YAGNI seem counter intuitive, but anticipatory design is seldom correct, it adds complexity, and the “cost of carry” is frequently large. Unix worked by releasing early and often, and building simply and evolving.

In Martin’s current view, the most important factor in getting evolutionary design to work is to have people on the team with the will and the ability to “make things converge”. Ability means that people recognise imperfections in a system (this is more important than knowing how to solve them – you can ask for help with that). Will means looking at what’s going on and have an active desire to fix it, persuade others etc. Thus motivation of a project team is a huge factor. You get problems if you don’t have people poking around looking into things and how to make them converge.

If design and programming happen together, how can a manager tell that any design is actually being done? The manager needs to get a sense of “do people care?” and “is action happening?”. An important clue is that code being thrown away indicates design is happening. Next clue is the sense of the team dynamics or motivation.

And that was it. I certainly found it very interesting.

Having a spam filter in place is vital these days, and the one I use is SpamBayes. This has an excellent Outlook plugin which seems to work very well. The accuracy is very good (I have 2.5k spams and 10k hams for training), though I do get a reasonably number of SpamSuspects which are mostly Spam.

The problem I had was that it is a client only solution – thus I had to download all that email to Outlook before it was correctly identified as spam. This is a particular problem when I am on site with clients, and my only email access is typically via webmail. Scanning real email amongst the multitudes of spam is not something I enjoy.

So, I wondered how I could get the spam filtering to happen on my ISP’s server without me having to download it. Like many ISPs they do have a SpamAssasin option, but I really don’t find this very accurate when processing, plus the fact that it was changing my emails meant I would have to retrain SpamBayes.

Fortunately, my ISP (Beyond Perception) is accommodating about the software I can install there, so I worked out how to install SpamBayes filtering directly on the server using the spam/ham training database that I had accumulated locally in Outlook.

.forward

This file sits in my home directory and just forwards emails to be processed by Procmail which is used by the ISP. Some trial and error and Googling showed the following magic spell does what I need.

~$ cat .forward
|/usr/bin/procmail -f-

.procmailrc

This controls how Procmail works.

~$ cat .procmailrc
# .procmailrc for using Python SpamBayes filter

MAILDIR=$HOME/mail
DEFAULT=$MAILDIR/inbox
LOGFILE=$MAILDIR/log

# Run the spam filter program on all emails which will then have X-Spam headers in for
# processing by later rules below. This just executes the Python SpamBayes program specifying the
# appropriate database.
:0fw:hamlock
| $HOME/utils/spamfilter

# Anything classified as spam put directly in spam box (might eventually
# just delete it.

:0
* ^X-Spambayes-Classification: spam
${MAILDIR}/spam/inbox

# The above just puts the mail directly into a file. If I used the following action it would forward
# the email (depends on your mail handling setup)
# ! spam@vaccaperna.co.uk

Transferring spam/ham classification DB

The above works fine but needed "training". I realised I could shortcut this using the spam/ham classification database used by my local Outlook installation. All I needed to do was export the database to a text file (using dbExpImp.py which is a script which is part of SpamBayes), ftp that file up to the server and recreate the local database there. This is a slightly manual process, but the whole thing is working well enough that I did it initially and only transfer updates every month or so.

The database is sitting in:

C:\Documents and Settings\<username>\Application Data\SpamBayes

The command to export it is:

dbExpImp.py -e -D default_bayes_database.db -f db.txt

This required a slight tweak to the SpamBayes files to ensure the correct Database format was used (Berkely DB).

The reverse procedure on the server converts the text file back again.