Tuesday, June 04, 2013

A Decade in Agile: 2. CIO/IT Directors - What Agile Means to You

This blog is part of the Blog Series: A Decade in Agile and was first published in April 2010. My updated reflections are in blue italics.

Agile is a double-edged sword!
On the one hand, following a period of intense change, budget spend and organisational upheaval... your department will have quicker cycle times to market, be more predictable operationally, produce higher quality software, have better working relationships with its customers and will be more adaptive to changes of business objectives and requirements. On the other hand, all the department's problems that you know currently exist will become very visible and transparent very quickly. All those skeletons in closets you've been dampening down, or battles you've been fighting to secure funding will rise to the top of agendas.
If you're looking for an easy ride or at least some calm seas, don't start agile. Increasingly, I am discovering that the issue faced by CIO’s and IT Directors is not that they are needing or wanting to cover issues or mistakes, rather that there is a fundamental lack of visibility and transparency in the end to end process of software delivery which means they don’t know where the biggest problems are. There are lots of voices saying how bad things are, but little consensus on the scale of the problem or how to quantify it. Increasingly, the more pressing need of CIOs is for visibility and transparency in order to understand that changes are required. This is leading to the need to use Lean Thinking and Application Lifecycle Management (ALM) tools. Lean Thinking brings Value Stream Mapping (VSM) to the fore, which provides an overview of the entire E2E process by clearly articulating customer expectations and metrics covering process, delay and lead times as well and complete & accurate percentages. By using VSMs for agile adoption we are able to clearly identify and quantify where the biggest problems in the pipeline are and attack those problems first by applying A3 problem solving and agile solution patterns. clip_image002 Creating current state value stream maps can be a painful experience when you apply cost data to a process with high levels of waste and low quality levels. ALM tools are the means by which software is created from idea to consumer usage and there are an increasing array of tools in the market – “Business innovation now drives the application life-cycle management (ALM) market. The contribution that application development and delivery makes to a company’s business goals – making workers more productive, creating engaging customer experiences, and brining new software products to market more successfully – must be more direct and successful. Software increasingly plays a central role in a firm’s ability to deliver new products and services or exploit new channels. Firms can no longer accept historical gulfs between business and application development and delivery teams as, increasingly, firms now expect to manage application development and delivery as a business and treat it as a competency.” (The Forrester Wave: Application Life-cycle Management Q4 2012) I expect strong growth in the areas of ALM tools and DevOps capability over the coming years as companies grapple with the integration of the business, application development and delivery/support teams in order to achieve Continuous Delivery. Agile methodologies expose the weaknesses in your value stream of delivering software, it exposes poor infrastructure, asset management, governance, processes, delivery, and most fundamentally, it will expose weak people...
Agile not only highlights these problems, it rubs your face in it by reporting all the dirty linen as impediments and waste, and directly attributes slow delivery to the specific problem. These could include lack of testing environments, or poor VM performance, or production support interruptions, or no automated tests, or poor architecture etc, etc, etc.
Not only that, but if you walk this road, your department will start to judge you and the success of the agile initiative by your personal ability to secure funding and remove organisational impediments i.e. your executive skills. This is a key point. I’ve seen first-hand the effect on morale and motivation when there is a lack of drive and courage from senior management. Teams that were excited and making great strides in changing the way they worked were brought to faltering halt. When you coach an individual person, regardless if they are from the business, learning to become a Product Owner or a seasoned Senior Developer, they ‘get’ agile and will accept/go-for a certain level of personal change. When you coach a Scrum Team, you can see the mind-set changes, the learning take hold, and the retrospectives become effective improvement triggers. Over a few sprints the team starts to race along faster than those teams around them… And then they start hitting organisational barriers: environment issues; business involvement; archaic deployment processes; bureaucratic change control… It is at this point that management/executives need to break-down some silos, build or re-build relationships, drive some change, spend some money. If these barriers or impediments are not over-come fairly rapidly, the change stutters and comes to a halt. Often this is most clearly identified and felt at the retrospectives where the same impediments and organisational issues are impeding the team sprint after sprint. So before you start down this road I think the first question for a CIO should be "Should building software be a core competency of this company?"
If the answer is no, then maybe the answer lies elsewhere in out-sourcing your software development to a good agile house. Forrester is a good start for compiling your short-list of agile companies such as Thoughtworks.
However, if there is a need for a strong software development capability because technology is fundamental to your business strategy, and the board of directors are in agreement, then agile is the best approach. This is because Agile development, specifically Extreme Programming, is one of the most disciplined approaches to software development in the world.
So, let's assume you're a pro-active CIO/IT Director, with a supportive board behind you, ready for the challenge... Where do you start?
Fundamentally, you're initiating a change programme to completely overhaul the way your company builds software. Don't fool yourself into thinking you're doing anything less!
The change will affect everyone involved, your consumers, your internal customers, finance, marketing, HR, Operations and your own area.
Forget the Hype - Moving to Agile is a large organisational change programme that will overhaul your approach to software delivery.
Make no mistake, it's about people, its about change and therefore we all know it won't be easy!
The good thing is that each change is incremental, there's no big bang, you just have to keep removing barriers, impediments and keep the budgets going.
It's continuous improvement, not revolution. Every company is different, and every implementation of agile is different. Some companies adopt it with ease, others find it hard, and again others take the bits that they ‘like’ or are relatively ‘painless’ and stop there. If you want to get the best results and productivity from adopting agile then it really should hurt a bit! You're going to need help!
There's plenty of agile consultancies out there at the moment flooding the market, but fundamentally I think you need to look for experience and named resources to cover 3 fundamental areas; the organisational change, the delivery process and the engineering practices:
1. Experience of managing organisational and cultural change including the stakeholder communications, business case creation, and internal marketing. And be able to demonstrate the energy it requires. (regardless of if its agile or not).
2. Delivery management expertise, commonly Scrum is the predominant process adopted by agile teams. So look for Scrum Practitioners or Coaches with at least 5 years experience and references to back it up.
3. Engineering Practices - Extreme Programming expertise, automated tools usage, and deployment specialists... basically strong agile engineering experience - ex-Thoughtworkers and Conchango Devs are a good start.
With competence in these 3 areas, you won't go too far off track but do not underestimate the need for the Engineering Practices as it is fundamental to the quality of product and reducing cycles times to market.
And as with all change programmes... this will take months not weeks. "The propensity to exploit Agile delivery to achieve business success is directly proportional to the capacity of a company to change and continuously improve."

Wednesday, May 29, 2013

Brains are Machines: Blog 2: Innovation, Automation & Investment

Personal development is often viewed as ‘fluffy’, with little relationship to the bottom-line, not something a CIO would hang a vision and strategy on. This Blog series: Brains Are Machines, attempts to demonstrate that the reverse is true. In this second blog, we highlight that despite our rapid innovation and adoption of automation within software development, the need for higher levels of investment in developing people remains.

Innovation

Technology innovation is rapid, the speed of innovation is accelerating and technology is ubiquitous. So whichever way you look at it, you either need to excel at creating & distributing software or you need to excel at assessing, purchasing and integrating software into your existing architectures, processes, people and culture. And this trend is only going to increase.
So if you were given a strategically critical project this year, i.e. the future P&L of the company depends on its successful implementation and return on investment, who would you trust to deliver it? Would you out-source it? Would you bring in contractors? Or would you give it to your own teams? I.e. Your own ‘machinery?’
If you wouldn’t give it to your people i.e. your brains, your machinery, what does that say about the state of your machinery and your ability to maintain it? What does it say about your capabilities for creating a team or organisation that meets the needs of the business?
Furthermore, what is the reason for not trusting your machinery with the job? Lack of knowledge? Lack of capability? Or lack of capacity?

Automation

Some of your competitors are now able to deploy enterprise-level, mission-critical applications on a daily basis. Through Continuous Delivery [1] (an encapsulation of many agile practices) the business can ask for a change, have it designed, developed, tested and deployed live, on an enterprise scale within 24 hours if required.
Test-driven development, continuous integration, automated acceptance testing, automated environment provision, automated configuration, automated load & performance testing, automated application and database deployments enable this capability…
Each of these individual innovations has removed many manual processes from the value stream of creating and delivering software, and the effect has been to rapidly reduce cycle time to market. These innovations implemented in concert, represent a fundamental shift in software development capability and capacity. Continuous delivery is starting to incrementally evolve business models.
But even accounting for this automation and reduced cycle-time, you still need people to establish and maintain that capability… The brains are the machinery, and are needed to write the automated tests, configuration and deployment scripts, to design the environments to be automatically provisioned and to establish the continuous integration process, as well as design and write the code.
And you’ll need resources to take you on beyond this point, adopting the next level of processes and technologies that drive the business forward in this ever-changing technology landscape.

Investment

So if the brains of your teams are the machinery of business, if technology is moving so fast, if you’re in a talent war and your people are implementing these technologies or assessing and integrating them… Do you really have the right level of focus on, and investment in the attraction, retention and development of your people?

Friday, May 17, 2013

Brains are Machines Series: Blog 1: Shifting Sands

Original post: http://businessvalueexchange.com/brains-are-machines-series-blog-1-shifting-sands/

The industrial revolution saw the landscape of Britain transformed from rural & agricultural to urban & industrial. The mass exodus of people from the country into the city represented a fundamental shift in the social fabric of the country.

The companies themselves were able to recruit and retain low-skilled workers, working on high-cost machinery that required constant maintenance and investment. The machines had maintenance schedules, were constantly oiled, monitored, measured, and had parts regularly replaced because they were the core contributor to the profitability of a company.

 A worker leaving the company had little impact on the capability of the business to create its product.

The information age has brought with it a shift in power toward the knowledge worker due to the nature of work. The information worker is a higher-cost resource, holds more tacit knowledge and intrinsically more value to a business, as their knowledge is part of the whole make-up and fabric of a company. Put bluntly “their brain is part of the machinery”. And therefore a business has more reliance on its people as resources, rather than its machinery (or infrastructure).

A worker leaving the company has a larger impact on the capability of the business to create its product or service. Tacit knowledge is leaving the business, a new worker is required and their induction into the collective takes far longer and costs a lot more.

So if brains are the machinery of your company, what kind of shape are they in? How much are you investing in them, what does their maintenance schedule look like?

The software industry has a scarcity of resource, and this scarcity of resource has existed for a number of years. In the manpower survey of 2012[1], 25% of EMEA employers reported difficulty filling jobs due to a lack of available talent. It has led to you being in a talent war, having to pay c20% fees for recruitment of team members before they’ve walked through the door. Also, your best ‘machinery’ is at risk of being head-hunted by the competition.

So if you’re paying c20% to a 3rd party just to bring talent through the door, and your best talent is at risk of being pinched, how much should you be paying to develop and maintain that talent?

[1] Talent Shortage Survey 2012, Manpower Group

Friday, May 10, 2013

HP Business Value Exchange - Thought Leadership & Featured Contributions

I’ve been invited to become a CIO Thought Leader and Featured Contributor on a new IDG produced site in collaboration with HP Enterprise Services. The site is called The Business Value Exchange and you can find out more about it here. I’ve written a couple of blogs including a piece on the Cellular Business model and the start of a blog series entitled “Brains are Machines”. I hope you enjoy the thinking!

Tuesday, March 19, 2013

Technical Debt: Strategies & Tactics for Avoiding & Removing It

For clarity, this blog will focus on how to manage technical debt i.e. make it visible, communicate it, quantify it, and prioritise its removal. This is because  in my experience it is not only a challenge to fix the debt, it can also be very challenging to get time and resources assigned to fix it unless a high profile issue has occurred which highlights the debt.
Blog Structure:
  • Technical Debt – Definitions
  • Strategy 1: Evolving a Mature Definition of Done
  • Strategy 2: Preventing Unacceptable Technical Debt
  • Strategy 3: Prioritising & Quantifying Existing Technical Debt
  • Strategy 4: Tactics for Fixing Technical Debt
A Definition
There are many definitions of technical debt available, my favourite is by Martin Fowler. Please have a read if you’d like a bit more clarity on the term: http://www.martinfowler.com/bliki/TechnicalDebt.html

Accruing Technical Debt

Most companies are creating technical debt all the time – lets take an example:
“The team I’m part of has just started a project. We don’t have a test environment but we’ve got our dev boxes and a build server so we can get cracking. Our definition of done is okay, we’ve said we’ll use coding standards, will build features to the acceptance criteria in the story and will have a review of features with the Product Owner. We’ll write unit tests on new bits of code, and prepare test scripts but because we haven’t got a test environment we won’t be able to do integration testing or regression test the features to ensure nothing has been broken in the process of creating new features. We’ve done a couple of sprints and there’s more bugs in the solution but once the test environment is available we’ll sort them all out. “
For some clients we work with, this in not a rare scenario, but inherent within this approach is the creation of large amounts of technical debt and its growing exponentially. I think the following slide from Colin Bird’sCSM course highlights the problem very succinctly.
image
As these items of increasing bugs, lack of regression testing, unclear coding standards, unclear understanding of the quality of code being produced due to a lack of pair programming, TDD or even peer review and a complete lack of visibility of how the solution will perform in an integrated, production environment increase sprint on sprint (phew!)… so the mounting technical debt will start to impede velocity and add more work on to the end of the project in order to move the solution from an immature definition of done to production ready.
The typical result of this approach left unaddressed is a set of hardening and stabilisation sprints that take months to get into production. This causes knock-on effects of losing stakeholder trust and support, blocking other projects from starting, keeping resources longer than expected messing up resource planning, and generally losing kudos and trust across your internal teams – and that’s before we talk about the impact on consumers of your product!

Strategy 1 – Evolving a Mature Definition of Done

image
One of the first major conversations I expect to have with a Scrum team is about the definition of done. Scrum specifies that the team should create ‘potentially shippable product’ every sprint. This is, in essence, the heart of the problem of adopting Scrum. It is the reason why so many technical, engineering and test capabilities must change and adapt in order to be able to achieve this ‘rule’.
An immature Definition of Done speaks volumes about the team’s overall ability to reach hyper-productivity levels of software development. If the gap between a story done in sprint and a story deployed into production is big, then after the final ‘development’ sprint, you should all expect a considerable period of hardening and stabilisation.
We’ve recently been using Value Stream Maps to help articulate the flow of work through development teams. The Value Stream Maps show customer expectations, information flows, physical flows, productivity metrics and value stream metrics including waste.
The physical flow often shows software being created and then moving through the environments of development, test, user acceptance, pre-production and into production. So, with your team, where is code deployed within the sprint i.e. when its tested and set to done in sprint, how far away is it from production?
  • And what is the cost or amount of pain to be inflicted before it reaches production?
  • How many unknowns exist between your team’s definition of done and production ready code?
  • What are you doing to evolve the Definition of Done to reduce the gap between sprint done and production done?
Again value stream maps come to the fore when considering these questions as does Application Lifecycle Management (ALM).
In order to avoid the long and painful hardening/stabilisation period, the team must be focussed on continuously evolving the Definition of Done to bring feedback loops and risks into the sprint and reduce the gap from sprint to production so that they are truly creating ‘potentially shippable product’.  

Strategy 2 – Preventing Unacceptable Technical Debt

Again, I’m afraid I’ll have to refer you to Martin Fowler’s blog which introduces the Technical Debt Quadrant:
 image
Ahh, the 2 x 2 grid! Takes me back to hours upon hours of MBA study! Thank god that’s done with!
I have often used this grid to work with teams on prevention. i.e. preventative measures in order to reduce future technical debt. As with all preventative activities, they usually take longer to implement but when adopted have a significant impact.
Use these categories to decide which sources of debt are acceptable and which are not acceptable within your organisation, and then establish tactics for preventing unacceptable behaviour…

Prudent & Deliberate

Generally, I don’t have a problem with behaviours that sit in the top right quadrant… assuming that is really where they belong. Often this quadrant is misunderstood. It is not about having to ship because stakeholders are expecting a shipping date, or someone’s bonus depends on it shipping on a set date – those scenarios actually belong in the top left quadrant.
The top right quadrant is reserved for business drivers that have a compelling ROI for why a product has to ship immediately – i.e. responding to a threat to the business bottom line, or exploiting an opportunity that positively affects the bottom line. If in doubt, a good challenge would be to inform a board member of the situation and ask if they are aware that the product is shipping due to a compelling business agenda.
Examples of where the top-right might be justified:
  • Entry into new market (first to market may mean that less quality or features with more work-arounds might be acceptable)
  • Regulatory or Legal Requirement (a compliance date has been set whereby non-compliance would represent significant negative brand, operational or financial impact to the business
  • Peak-period opportunity – within retail the Christmas period is increasingly becoming the most critical part of the annual cycle and failure to launch prior to Christmas could have a significant negative impact on the company’s performance.

Reckless & Deliberate

Top left is indicative of poor management, usually corners are being cut in order to hit a deadline that is related to perceived operational needs rather than an underlying clear business case. Rushing teams because someone somewhere has communicated a deadline and then driving to hit the deadline because of the deadline’s sake rather than a compelling business need.
This is a very common cause of Technical Debt. If the board or shareholders were informed that a project was cutting corners and creating technical debt which will slow the company down in the future, they’d want to know why. In fact, I think that if the subject of technical debt was discussed more openly and quantified, they’d be far more examination of portfolio management capabilities.
A lot of the time, this is occurring not because there is a business case for hitting this date, rather due to how the company delivers projects. Examples of Reckless & Deliberate:
  • Rushing the project to completion because there’s lots more projects to deliver and the matrix resource plan needs to transfer resources
  • Cutting corners in a project because the programme manager and/or director have incentivised objectives based on the project being delivered into production this year
  • Pushing the project through because the client wants it on a set date, no one has built a relationship with the client to discuss the details, nor has the client been informed of the affect on quality if the delivery is rushed
None of these are necessarily easy to change, but, stopping these behaviours will have a significant, positive impact on the long-term velocity and well-being of the company.

Reckless & Inadvertent

Incompetence at one level or another is the key contributor to debt created within this quadrant. You don’t know what you don’t know and could therefore be blissfully unaware that you are creating a huge amount of technical debt. As a manager/leader, I’d want to prevent the reckless & inadvertent creation of technical debt and there are many tools to help me do this.
Essentially, this is about investing in your people, processes and tools – again something not done enough!
Pair programming, code reviews, static code analysis, continuous integration, automated test suites help to provide feedback on code and design quality. Communities of practice, clear role description, personal development plans, training budgets with technical strategy alignment are tactics for helping people get better at what they do and lightweight iterative methods with visual management help to ensure processes are continuously reviewed and improved.

Prudent & Inadvertent

This is a natural occurrence. Regardless of what walk of life we are in, or what job we do, over time, we’ll return to a previous piece of work and see a better way of doing it. It is the natural sequence to gain more domain knowledge about a particular piece of work.
Just because we know that with hindsight and increased experience in a year or two’s time we’ll look back and see a better way, doesn’t mean that we procrastinate or spend large amounts of time trying to second guess the future.
Rather, we keep to the agile principles of:
  • Our highest priority is to satisfy the customer through early and continuous deliver of valuable software.
  • Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale
  • Working software is the primary measure of focus
  • Continuous attention to technical excellence and good design enhances agility.
  • The best architectures, requirements and designs emerge from self-organising teams
  • At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behaviour accordingly

Strategy 3 - Prioritising & Quantifying Existing Debt

So, lets assume we’ve implemented some tactics for reducing our Reckless & Inadvertent debt and our Reckless & Deliberate debt and now we’re going to spend some time ‘speeding-up’ the company.

Where Does it Hurt?

This is how I have approached the problem in the past.
imageThe first thing to do is to try to establish how painful a particular piece of technical debt is. i.e. if you have to touch this part of the solution in the future, how ‘painful’ will it be? How much additional time will be spent understanding the area, re-writing parts to get is to work or integrate with other new parts? This ‘pain’ factor is represented by the yellow part below and we’ll refer to this as the interest.
Next for each item of debt, we need to understand the effort involved in fixing it, and we’ll refer to this estimate as the Principal. 

How Much Does it Hurt?

Having established how much interest is payable upon the use of any part, and knowing what it takes to fix it - the principal - we can now work out which pieces to fix, and why. 
image
Looking at the slide above, which piece of debt do you think should be fixed first?
The answer with this amount of information is A, as it has the highest amount of ‘pain’ or Interest, with the lowest effort to resolve ‘Principal’. However, what we haven’t considered yet is time and the frequency of payments. i.e. what if in the next 6 months item A on the left will only be changed/touched once, but the item on the far right ‘D’ will be touched 10 times?
Now which item of debt do you think should be fixed first?
image

Who Cares?

So we’ve established the frequency of Interest payments and found out that the most painful item of debt is item D on the far right. Have we finished?
No. Because until now we’ve focussed purely on the IT decision-making side of Technical Debt i.e. how painful it is, what it takes to fix it, and how often it hurts our development efforts. What we have yet to consider is whether our prioritisation takes into account business value. 
A fair few years ago as Head of Software Development, my team created a product portfolio. i.e. we took all our disparate applications (and there were many, and they were disparate) and grouped them into families of applications that served specific business needs.image
This was a very powerful exercise and it helped us to clearly discern where we were adding value and supporting the business. It also allowed us to quantify the value of software applications based on which business units were using them and how much revenue was being generated by those units.
In this way, we were able to prioritise fixing technical debt based on a clear alignment of business value.
So now we can quantify the size and frequency of Interest payments and we know what it will take to pay-off the principal to remove the debt. In addition to this we can prioritise the debt in terms of the value of the application to the business at the moment, but what about the foreseeable future.

Strategic Intent

The final consideration when prioritising technical debt activities across your application portfolio is the future needs of the business upon the application landscape. Which systems are critical to the future of the business? Which will be decommissioned as the portfolio moves through the foreseeable future?
In answering these questions, we will understand the future needs of the business upon the application landscape and be able to focus the entire development effort on improving the highest value systems of the business both for today and tomorrow.

Technical Debt Portfolio Prioritisation

By taking the factors we’ve already discussed: Interest Payment; Principal; Frequency of Payment; Business Value;  and Strategic Intent - we can now create a high level, business-derived prioritisation matrix for articulating, quantifying, prioritising and resolving Technical Debt.
The first step is to map current business value/usage against the future strategic needs of the company.
image
This mapping will provide an initial view of the value of applications within the landscape.
image
Once these applications have been reviewed, categorised and the output discussed with business stakeholders, we can initially provide a generic approach to dealing with the debt:
  • Operationally Maintain
  • Fix the Debt
  • Don’t Touch the Application
  • Focus on Prevention
image
 

Strategy 4 - Tactics for Fixing Technical Debt

We now have a means of categorising all the applications within the landscape and assigning relative value to them based on current and future usage requirements. Within these categories we are able to specify the ‘pain’ of technical debt, as well as the effort required to fix it.
We also have some strategies for dealing with the debt:
  • Ignore – for applications that are Dogs
  • Reduce – for Cash Cows to ensure the application is operationally maintainable
  • Remove – for the Stars
  • Prevent – for the Problem Children
Hide the Work - I’ve seen teams hide the work to fix the Technical Debt. This tactic can be successful but I feel it goes against the grain of transparency and honesty and ultimately the fixing of debt should be a business driven activity. But, I have seen this tactic work so it should be considered within your context.
Leave the Code in a Better State - The on-going development strategy should always be to leave the code in a better state. It’s a simple statement that is rarely adhered to. The rule is that every time you touch a piece of code, you leave it in a better state. It could be as simple as an additional unit test or some clearly articulated comments or as complex as writing a suite of unit tests and refactoring a component. If every single developer adopted this strategy in your organisation the state of the code base would improve significantly over time.
Ask to Leave Code in a Better State – This tactic is closely related to the one above, but is cognizant of the fact that some companies micro-manage their teams to the extent that any time not spent directly working on the addition of features can be rapidly identified and can cause friction. This tactic addresses the scenario where a developer identifies a piece of technical debt during a sprint, that they think should be fixed. Rather than just sorting out the problem (which is desirable outcome), the developer would escalate to the Scrum Master and Product Owner. If you have to resort to this tactic then there is a lot of work to do in educating the business and IT management about the problems of technical debt, as well as a recognition that you have not provided an environment that allows self-organising teams to flourish.
Create Story & Justify – Typically, leave the code in a better state is not a fully adopted way of working across development teams and the more typical way of fixing technical debt (other than complete re-platforming) is to create individual stories and treat them as ordinary backlog items. When you get this up and working fully, involving the Product Owner in the reasoning behind the work, both parties learn a great deal about each other’s perspectives and it can be a valuable lesson in understanding the business and technical domains.
Allocate Release Ratio – Another tactic I have introduced in orimageder to achieve some progress in fixing technical debt is to ‘Allocate a Release Ratio’. When articulating the value of addressing technical debt and prioritising it within a Product Backlog, I have often seen that despite best intentions, the Technical Debt items sink toward the bottom of the backlog and do not get resolved.
There are a number of reasons why this might occur so to mitigate against this occurrence, I seek agreement from key stakeholders to allocate a certain amount of a release backlog to technical debt items. This approach has certainly been successful in ensuring TD items are addressed. Ultimately though, the best tactic remains to leave the code in a better state as it is more efficient.

Summary 

Ultimately this is about understanding the value of your application landscape, understanding where its weaknesses lie and finding a way within your organisation to ensure it is fit for purpose now and in the future. I hope that the next time you hear the term Technical Debt you’ll have gained a clear understanding of what it is from Martin Fowler’s blogs and you’ll also have some tips for how to address the problem from this blog.

Thursday, January 31, 2013

A Decade in Agile: 1. Breaking the Iron Triangle–Project Manager v Scrum Master

 
This blog is part of the Blog Series: A Decade in Agile and was first published in February 2005. My retrospective comments are in blue italics.
As a PM my general approach to initiating projects has been to build a Project Initiation Document (PID – Prince). A PID details the project objectives, initial scope, organisational structure, roles & responsibilities, deliverables, project processes (lines of communications, risk management, issue management, change control), plans, costs and timelines.
And starting an agile project I would still identify most of the above and ensure it is clearly communicated to all stakeholders.
The key difference in approach is the way it is done, not the artefacts themselves. These pieces of information are more likely to form part of a kick-off workshop with the team defining their roles & responsibilities, the Product Owners defining the Vision and Minimum Marketable Features (MMFs), the team identifying risks and then rather than logging them, creating stories that investigate the risk to minimize or remove it. I think the key shift underlying this paragraph is the removal of the “I create” to “we collectively created…”. In Agile, the Scrum Master is not the leading voice, not the commander leading the troops. The Scrum Master is the facilitator, obstacle remover, a servant of the team.
But… I have learnt that adopting a scrum approach changes two fundamental things for a project manager:
· The Iron Triangle
· The Role of Project Manager
The iron triangle of cost, time and scope (or quality) is typically the responsiclip_image001bility of the project manager to manage. Hopefully tolerance levels have been set for each element of the triangle, and if the tolerance level is about to be breached or is broken, the project manager escalates their issues to the project board or steering group.
Previously on traditional projects I owned and was responsible for this triangle and reported to the project board if any risks or issues were likely to break the tolerance levels.
I have a wry grin on my face as I read this… I have recently seen a few Scrum Master role profiles describing the need for Scrum Masters to manage the project to time, scope and budget. At least when you read the job spec, you gain an indication of the level of agile adoption. In a traditional project a deliverable-focused approach to planning is used (Prince = configuration plan), whereby all the intermediate work products and “pieces” of the solution are identified and a plan drawn up to show the route to delivery usually utilising a gantt chart or network diagram. From which timelines, resources, and costs can be identified and critical path analysis and risk management conducted.
Now scrum doesn’t throw this all away… rather we open out the Iron Triangle and share responsibility across the team, and recognise that no matter how well we plan the project, things will change, and problems will occur.
So… lets make time and cost fixed for a set period, say 4 weeks. The resclip_image002ources I’ve identify for the project will work for 4 weeks together which will cost a fixed price…
Now, let’s not set the scope as fixed, but accept that scope is variable within this part of the project. The amount of work and quality of deliverables achieved within the fixed cost/time axis will depend on people… it will depend on the team’s productivity and practices.
Let’s remove focus from the iron triangle established at the beginning of the project, and instead focus on making the team as efficient and productive as possible.
This is the key statement within the entire blog. Rather than trying to plan up front how long everything will take and then spend all your time trying to make sure the plan is followed… why not understand what the customer wants, how long we’ve got and then spend all your time helping the team achieve this by removing obstacles, inspiring them, giving them autonomy and ownership. So the role of the Scrum Master is one of maximising the productivity of the team by removing obstacles, inspiring, motivating, and ensuring the scrum methodology is followed.
The responsibility for what is built (the scope) is then given to the business. Scrum identifies a Product Owner, a business stakeholder who has a list of the requirements for the project prioritised by business value.
Maintaining quality levels can be aided through the use of agile development practices such as automated testing, continuous integration and refactoring.http://www.martinfowler.com/articles/continuousIntegration.html
A common pattern for defining and maintaining quality in modern Scrum teams is through the “Definition of Done” (DoD) and the Extreme Programming practices mentioned above. The team defines its quality criteria for each software feature created collectively and hold each other accountable for that quality. The DoD is often printed out and placed somewhere very visible for all team members, so that everyone can be challenged on the quality of product features being created.
Obviously, many teams are not developing in isolation and therefore standards from the wider organisation, or across multiple teams of developers will naturally be included in the DoD.
Some typical problems I’ve witnessed in adopting a Definition of Done:
Aspirational: The Definition of Done should be the quality criteria by which every software feature is completed. Every team member should understand the DoD and be adhering to it every day. One of the problems that re-occurs regularly is that someone has written a beautiful DoD (back to the old PM style – commander in chief) which bears no relationship to what the team can or will achieve. I’ve regularly seen this solved by pulling the team together (usually at a retrospective) and walking through each line of the DoD asking the question “do you do this right now for all features?”
The result is a much smaller DoD, of maybe 4 or 5 lines that everything will meet and everyone can adhere to. You then encourage the team to build it up again with the rules that it is not an aspirational list, it is an operational list.
Evidence / Metrics: I’ve now grown accustomed to seeing one particular line in a Definition of Done: Adheres to coding standards [What does that mean?]
When I see this line item, I usually to ask to see a copy of the coding standards… Responses vary. Assuming that we have some standards to reference, I’ll then ask “to what level are you all following these standards?” And finally, “As a team, how do you know you’re all following the standards and none of you are creating poor code that will impede the team at a later date?”
In 2013, there is little excuse for any team not to be using static analysis tools. These tools can be configured to match your coding standards if required. Your automated builds can treat any violations of coding standards as errors to ensure no poor code is allowed in the source code repository, and beyond that, code productivity tools such as ReSharper help developers identify problems with their code as its written so it can be resolved before going into the build.
So for every part of the DoD (not just coding), we should look for ways to automatically measure adherence to the DoD. The advances in Application Lifecycle Management (ALM) such as automated builds, automated testing, automated deployments, and automated virtual environment provision & configuration, means that teams can much more easily measure the quality of what they are producing… And if you can measure it, you can improve it!
Pressure to Deliver: This problem exists in both the traditional and agile worlds, that of sacrificing quality for a short-term delivery that creates technical debt and will slow you down in the future. Often the person applying the pressure that leads to the short-cutting is unaware of the long-term issues being created, and in less mature agile teams, developers are often not in a position to directly influence the decision. Short-termism is becoming an increasingly common problem which I will cover along with technical debt in a future blog. 
The project team select the highest priority requirements (as specified by the product owner) they think they can build in 4 weeks. So time is fixed, cost is fixed, the highest priority business needs are defined, and how much is achieved (scope) is based on the team’s productivity level.clip_image003
So whereas PMs own the Iron triangle and spend their time, directing, planning, mitigating, and monitoring progress, Scrum Masters give the responsibility for the scope of the project to the business, set time and cost as fixed, and focus their efforts on the productivity of the team. The team is responsible for delivery, the business defines the direction or scope, the scrum master helps the team meet its objective for the 4 weeks.
The role change from PM to Scrum Master is not easy… releasing control and sharing the responsibility with others can be a little disconcerting. However, I’ve seen at first hand that the productivity gains in adopting this approach are impressive.
I cannot under-emphasize the fundamental mind-set shift a traditional PM will need to undergo/experience in order to become an effective Scrum Master. Many of us became strong PM’s through taking control of the situation, understanding every facet of a project, exuding assertiveness, confidence, and strong leadership and epitomising dogged determination. As a Scrum Master you have to release control, or as in my case, realise there was no control in the first place, only the perception of control.
Scrum Mastering requires a completely different style. Putting the team as the focus of attention and sinking into the background to quietly remove impediments, coach, motivate and ask questions is not for everyone. It’s the team’s responsibility to deliver the software, it’s the Product Owner’s responsibility to define scope, the Scrum Master helps the team follow the process, and removes impediments.
This can prove even more troublesome in some organisations where promotion, reward and recognition systems  have not moved with the adoption of agile methods. Companies that do not embrace wholesale change will still reward heroics, hitting a plan and getting it done OVER resolving root cause problems, innovation and sustainable growth. Promotion through how many people report to you OVER Promotion through providing business value or profitability. The business sees its highest priority requirements built and delivered every 4 weeks. The team continually improves and increases its productivity levels, and every 4 weeks everyone knows exactly how far the team has progressed because the software is there for everyone to see, test and review

Wednesday, January 23, 2013

Blog Series: A Decade in Agile


10 years…
Ten whole years of my working life dedicated to creating software through agile methods. A lot of things have changed since those first few sprints. Lots of lessons learned, (usually the hard way), plenty of experience from working with lots of different people, with difference perspectives, in different locations, with diverse cultures and in unique companies.
As I went through the highs and lows of agile adoption I wrote a small but detailed blog along the way. These ‘epistles’ were usually written when I had considerable motivation to tell the story – when on a high from success, or a low from learning things the hard way. These blogs became a vehicle to vent frustration or expound virtues of some new insight or shift in mind-set.
Over these first weeks of 2013, I’m going to take some time to review these blogs, and retrospectively comment, amend and critique the thinking at the time, against the years of experience that have followed. Maybe I’ll learn something new along the way and I hope you’ll enjoy the journey too. There are 2 main categories of blogs I’ll be reviewing:

Category 1: The Basics & Tactics


These blogs are more for the people in the trenches actually using Scrum in anger (or bliss!) and consists of lessons, insights and patterns that have proven useful.
·         How Adopting Agile Increases Business Benefits
·         The 10 Commandments of Agile Planning
·         Which Agile Method?
·         Launch, launch, launch
·         Agile Metrics & Scorecards
·         Constraining Agility
·         The Machine that Changed the World
·         A Nod to a Solid Team

Category 2: Transformational & Strategic


These blogs are aimed at the executive or change management audience emphasising key concepts, mind-sets and expectations when leading agile change initiatives:
·         CIO/IT Director – What Agile Means to You
·         Implementing Agile: Top-down or Bottom-up
·         Inversion Management: 21st Century Leadership
·         Agile & Transformational Leadership
·         Lost in Translation
·         Organisational Patterns
·         Fearless Change
·         Emotional Intelligence
·         The Cellular Business Model