Tools vs Processes and People (again)

Thu 05 July 2018

It's a year since I wrote the "People vs Processes and Tools" article. Apart from being busy with other things, the main reason I don't write very much here is that my criteria for a post are:

  • cover something that I want (or need) to assert in public
  • include something new/innovative/challenging, that isn't widely understood
  • stand the test of time, i.e. hopefully still be valid in a decade

Suprisingly the main thing I feel compelled to write now is actually another assault on the "People, Processes, Tools" dogma, which has been been lobbed at me a lot, recently, as a magic spell to thwart my reasoning or recommendation about a specific tool or set of tools.

In other words, the underlying message is...

"You're wrong to even be thinking about the tool(s) at this point. We need to figure out the team and organisation first, then establish the processes that matter... and then will be the time to choose the tools."

My internal monolog takes this further...

"...And by then, Paul, you'll no longer be one of the People involved. And in any case we'll have established Processes that you would either refuse or fail to follow. And just to completely box this off, we'll be choosing a different tool, that you don't like, to demonstrate how wrong you were. And we'll make sure you aren't even allowed to access it."

AFAIK the People>Processes>Tools ordering was originally cited as part of the Six Sigma approach. It may well be fantastic in that context - I have no idea. But for large scale software work, particularly in established organisations, it seems largely irrelevant to me.

For a start, there aren't many effective levers we can pull with People:

  • lead, manage
  • incentive and environment factors
  • educate/train
  • hire
  • fire

Without adequate leadership, management, incentives and environment the remaining factors don't actually get us very far.

If we have the wrong people leading, no amount of education/training will help. Inadequate managers will struggle to even recognise, let alone hire the right people. And firing established staff is usually surprisingly stressful and difficult.

Most of the Processes I've ever seen documented were at least in part

  • unworkable in practice, and/or
  • not relevant to the actual objectives

People who define processes without understanding both the details of the actual work, and the stakeholder objectives and values, will usually produce bad processes. Ironically, these documents are usually described as "Quality Processes".

So what's the answer?

Based on some decades of experience, and with a number of successful (as well as unsuccessful) projects delivered in various scenarios, I'd like to describe my alternative:

Philosophy => Key People => Tools and Environment => More People => Processes

Philosophy: let's make sure that we're as clear as we can be about what is the best/right work to do, before we start. This is where our leadership needs to get its principles, intentions and objectives straight. We may need to adapt/improvise later but doing the thinking early can avoid a huge amount of waste and grief later.

Key People: now onboard the (core, small) set of Key People, selected for their knowledge and ability, to lead/drive/implement the initial phase, which will be...

Tools and Environment: choose the best Tools for the work, and set them up in an Environment which

  • is dogfooded (used and tested by the Key People)
  • is reliable, reproducible, backed up etc
  • causes minimal friction for the work
  • is locked down so normal users working in good faith can't actually break anything
  • has appropriate gates for review/quality/validation/signoff (but note that gates can cause bottlenecks and friction)
  • is simple and obvious for users who are competent and experienced in the work
  • is self-service (minimal handover/wait-steps) so users can get on with their jobs
  • provides enough basic documentation for new users to get going quickly
  • provides immediate and correct feedback from user actions so that problems and documentation can be improved quickly and effectively
  • collects and maintains evidence about the work done

More People: Onboard the rest of the team, preferably in phases, and:

  • fix the problems they encounter
  • document the things they find confusing or unclear
  • measure the bottlenecks and do everything practical to reduce them and increase overall throughput

Once this is working, the remaining step is...

Process: Now that everything is running, we can relatively easily document what the actual processes are. However, this may not actually be worth doing:

  • we already have the best tools for the job, in a low-friction environment
  • we already have with evidence of appropriate reviews/approvals and outputs
  • we are already measuring our throughput and reducing bottlenects

If you're still here, your internal alarm may be wailing

"Hah! He's hidden 'People, Processes, Tools' in his "Key People => Tools and Environment' steps. The people chose tools to implement their implied processes."

Actually, that's true;

I work with the best People I can find, and I've learned a huge amount from them. As a result I'm relatively experienced in both software engineering (doing the work) and the business implications of software engineering (stakeholder). As a result I'm relatively confident in what software enginering Processes at scale need to involve; in my view they boil down to

"continous integration, automated test, and continous delivery... with evidence of provenance, reviews and approvals.""

And so, in the end, I recommend specific Tools that help to achieve that.

People vs Processes and Tools

Mon 10 July 2017

I recently attended a demo of a software tool for requirements management. After a few minutes showing the menus and screens, the demo became a presentation of a slide deck about the software.

It turned out that the engineer demonstrating the tool does not use it himself. His weapon of …

Read More

Who can we trust for software?

Thu 15 September 2016

TL; DR We need to figure out how to guarantee that software can be trusted

In the over-simple graphic above, I've tried to map some of the key technical roles being performed in the delivery and maintenance of complex systems and services. I'm struggling to work out who precisely sits …

Read More

The Dalton Cycle

Fri 29 July 2016

Thanks to Niall Dalton for this comment at the end of his post on the trustable-software list

"I go off on this tangent to raise the whole bucket of pain involved in building a trustworthy system like this. Not that we necessarily want to try to build a 100% trustworthy …

Read More

Trustable Software - Help Wanted!

Tue 14 June 2016

For a whole load of reasons I think we need to get to trustable software, and would appreciate comments and suggestions, preferably on the mailing list, which is public.

Note, I'm not "marketing" or "selling" anything here. The aim is to trigger interest in a cross-industry set of grown-up engineers …

Read More

Are we there yet?

Sat 16 April 2016

Many large projects are late. Most of the time the the actual journey turns out to be much harder and much longer than it looked when we started out.

From a management perspective we are constantly looking for validation that we're hitting the plan, whether it's traditional milestones and deliverables …

Read More

Technical debt for whole systems

Sat 19 March 2016

"It ain't what you don't know that gets you into trouble. It's what you know for sure that just ain't so." [Mark Twain]

If you've been in involved in large-scale software projects over the last few decades you've probably learned the hard way that upgrading the platform (operating system, libraries …

Read More

Software Engineering As Legitimate Engineering Discipline

Wed 20 January 2016

For 2016 I've decided it's time I take my public communications seriously.

Perhaps some of my customers and colleagues realize that my previous attempts at wit are born from decades of struggle on difficult software projects.

But most of the internet has no idea who I am, and my previous …

Read More

Why Should GENIVI Members Contribute, Again?

Tue 23 June 2015

GENIVI Members need to make attractive, useful software to run reliably in cars for a long time. The amount of software needed is increasing faster than our capacity to bring in extra engineers, and our ability to increase the productivity of the engineers already engaged. Given the economics, automotive organizations …

Read More

Leading Tools in GENIVI

Sat 02 May 2015

I've had a stormy relationship with the automotive industry - love-hate doesn't begin to cover it.

It's not that I'm a car guy - I can barely manage an oil change. But cars these days contain a staggering amount of software. And in spite of what some may say, I am a …

Read More

Three Times Done

Sun 11 January 2015

In my experience many organisations experience systemic difficulties in getting work 'done' to an acceptable standard. The Scrum solution for this is 'agreed definition of done', which typically leads to a checklist eg

  • the work has been checked in to the VCS
  • it builds ok
  • the automated tests pass
  • documentation …
Read More

Refactoring is an anti-pattern

Fri 13 December 2013

I thought I'd written more than enough words on refactoring already, but maybe I was too cryptic.

So here's my version 2.

Let's start by noting that most of the folks who claim that they are 'refactoring' don't actually know what it means.

I, on the other hand, found the …

Read More

What does "BSP" stand for?

Sun 23 June 2013

During a late-night drinking session technical discussion this week there was confusion about the meaning of "BSP" for Linux and Android.

Some folks think BSP stands for "Board Support Package".

But, as with SCRUM, the popular view is wrong.

In the real world, BSP means Bull Shit Project.

BSPs are …

Read More

The Worst Metric

Wed 06 February 2013

Lines Of Code is an awful way to measure software - but look at the pretty picture...

Read More

The software commandments

Fri 15 July 2011

OK, I admit it - some of the Agile thinking did influence the recent Software Commandments 2.0 update.

I accept short timeboxed sprints.

Agreed definition of DONE - great.

Prioritised list of requirements - awesome.

Daily standups - fine if you are all in the same room, and you stick to the rules …

Read More

The best revenge...

Thu 14 July 2011

Over the years I've worked with many amazing people, but from time to time I've also had to deal with some complete assholes unpleasantness.

Frankly it's amazing at how downright dishonest, manipulative and plain stupid some folks can be.

Actually plain stupid applies to me too.

I've been repeatedly robbed …

Read More

I predict the future of embedded Linux

Fri 06 May 2011

I've been reading the excellent Future Babble by Dan Gardner.

Perhaps unsurprisingly, it confirms that we're all wired as complete idiots when it comes to looking into the future.

The science shows that the more confidently someone makes a prediction, the more likely they're talking utter balls. But sadly, it …

Read More

I believe in aeroplanes

Thu 27 January 2011

How can you be sure that something is true?

Actually it is surprisingly difficult... Do I mean true as in fact or true as in 'I believe this'? Maybe we're in the matrix, and everything is a simulation... maybe the people you trust are lying.. or maybe they're just wrong …

Read More


Sun 16 January 2011

After a typically ignored secretive softlaunch, I'm finally taking the wraps off devcurmudgeon. The idea is to be funny, incisive... and maybe change the world one hacker software developer at a time. But I'm not optimistic. This project may fail, like so many others :-)

Mostly I expect to be ranting …

Read More

El Reg "not your grandad's project management"

Tue 14 December 2010

I confess, I get a lot of my worldview from The Register... saves me coming up with my own jokes.

Anyway, was a little disappointed watching this...

Had to turn off when the Microsoft guy said "you have to build the business …

Read More

My briefcase is older than you

Tue 30 November 2010

I recently demonstrated my desperation gravitas to a roomful of undergraduates with the immortal line "My briefcase is older than you." Disappointingly they continued fiddling with their gadgets.

Even my rollercoaster antics with Prezi failed to amuse...

But the exercise was not entirely a waste of time.

A spot of …

Read More

Picking a fight...

Sun 14 November 2010

Maybe I'm alone on this one ... but aren't Open Source and Agile pretty much directly opposed?

Agile = 5-9 people in a team, locked up together, everything thrashed out in face-to-face conversations, daily standups, timeboxed iterative deliveries.

Open Source = no fixed team size, distributed around the world, everything thrashed out via …

Read More

Musical differences...

Thu 04 November 2010

I used to be in a band, when I still had enough hair for a stupid haircut.

We had real instruments. None of this electronic bleepy noises nonsense. We used to lug huge amplifiers everywhere, because the guitars always had to be loud enough to shake the building.

Whereas the …

Read More

Open Sores

Fri 22 October 2010

I need to get with the program. All these years working on secret projects with someone else's name on the mastheads have created an obvious problem - nobody knows who the hex I am.

But wait!

There's an obvious solution too.

Why don't I just take my old spaghetti code - all …

Read More

Apple vs Contortionists redux

Fri 01 October 2010

Previously I asked "Does Apple use Scrum?"...

An ex underling just slipped me the recipe for Apple's secret sauce....

Wanna hear? Of course you do... you too could make gazillions... but you've got to promise not to tell anybody else.

It goes like this...

  1. Three independent design teams produce mockups …
Read More

Software Chaos Ridiculously Under Managed

Sat 28 August 2010

So this post is just a place-holder for me to launch my new methodology - SCRUM.

I'd hate to have some fan copy my ideas (again) and find myself unable to prove categorically that I thought it first, so here goes...

Google tells me (so it must be true) that no-one …

Read More

Emperor's New Scrum... iteration 1

Fri 27 August 2010

I've been wading through the literature on Scrum/Agile/Lean/Kanban/Hokey-kokey for months now, and I'm still no nearer to making sense of it...

How on earth can folks seriously believe that the whole commercial world should turn itself upside down so programmers can have an easy life?

No …

Read More

On time and on budget - yeah right!

Sat 07 August 2010

Developing complex software well is actually very difficult to do.

Seems to me it requires intelligence way above average, combined with enough pig-headedness to refuse to be defeated by dumb chips and other people's code spitting random error codes at you - often for hours or days at a time.

It's …

Read More

Some background...

Thu 05 August 2010

After years of advising folks NOT to blog, tweet or display their backsides innermost secrets on facebook, here I am poised to break my own rules. No doubt I'll come to regret it later, but really, I need to vent.

Later posts will explain why...

Read More