28 February 2008

Is the BA an endangered species with the growth of Agile?

First, I'm a project focused software developer, team lead, designer, architect, jack of all trades, who has been on projects that have used various methodologies over the years, including of late some agile projects.

I'm not a big blog reader, or a big blogger, but like most people I have an opinion on things, and for some reason that opinion is valued by various people I have met in my life's journey, so I have been invited to provide input into this forum from a technical perspective.

Doubtless, my blog posts will be intermittent, and possibly contentious. So this marks my debut posting for the Better Projects blog. I have been reading of late various blogs around the place (usually when hunting for other information) on the imminent demise of the role of the BA, with the growth of Agile and the perception that the role of a BA is not required in an Agile project.

From a BA community perspective there seems to be a sense of impending unquantified unease, if not one of impending doom. For the IT community, there is growing conjecture on whether the BA is really an unnecessary evil that can be dispensed with in an Agile project.

To paraphrase Mark Twain, rumours of the death of the BA is greatly exaggerated.

To understand why I'm of this opinion, I want to consider four things. What is the business? What is a BA? What is the role of a BA in an Agile project? What is the future of Agile?

What is The Business?
To generalise, the business are focused on two things:
  • Operations; which equates to staffing, training, process compliance, materials, machinery, transport logistics, supply, SLA's, demand or orders, stock levels, etc.
  • Management; the analysis of cyclical trends, identifying markets, advertising campaigns, market share, competitor analysis, capacity, contracts, etc.
For the company to be successful, management has a need for accurate statistics to be derived from their operational areas, and operations has a need for efficient, low waste production, yet retaining flexibility to meet varying levels of demand.

The Business are not necessarily good at identifying where they can improve, how to go about change, or even to identify what process' they do undertake. They often however, identify the need for change but will usually struggle to articulate what this change will look like. This is when they usually turn to the BA.

What is a Business Analyst (BA)?
I will probably be telling the readership of Better Projects how to suck eggs on this one, but it may or may not be insightful to hear the perspective of someone from the outside describing what they believe a BA is. I am happy to have my view corrected though.

A Subject Matter Expert (SME) while often employed under the guise of a BA title, in my mind is not one. The software developer who has grown tired of the never ending chasing of the technological tail and has "retired" to the BA role, is also not necessarily one.

The two main attributes required of a BA are:

1. A facilitator; they act as a communications bridge between management and operations in the business, as well as between the business and technology. Sometimes even as representative of their company, in the communications with external parties. For this they require a number of skills and tools such as being able to:
  • manage meetings with people from a range of disciplines
  • identify appropriate stakeholders
  • networking across disciplines and business units
  • communicate to and alleviate concerns of the various stakeholders with tailored messages to each group
  • manage stakeholder expectations
  • create presentations, and present to various stakeholder groups, of varying size audiences
  • convey concepts, ideas, details
  • generate enthusiasm, sell a vision, keep the dream alive
This requires some knowledge of various disciplines, from the business domain through to the technical domain, as well capability in the tools of the trade.

2. An Analyst; with an understanding of the business domain, the ability to analyse inputs, outputs and stakeholders, map processes, identify deficiencies, envisage and communicate potential solutions, perform cost analysis and projections, understand and represent The Business in its needs, being able to identify and articulate the detail that they cannot, and helping to ensure that the target vision is being met with any deliverable.

But most importantly, a BA is not an island of knowledge. A BA is a member of a multi-disciplined intelligent team, where ego is often a major contributor to internal friction, but where it has no place. They need to avoid being proscriptive in their proposed solutions, and for the best chance of success within a project environment, they need to take a collaborative approach.

Both The Business and the IT team have a dependency on the BA's, but when the BA acts alone and arbitrarily as with any other member of this team, it is to the detriment of the deliverable and all who are involved.

The Business needs to be involved for the whole project life to validate what is being developed and potentially identify new needs in light of what they have seen.

The IT team need easy and ready access to The Business. For many things, the BA acts as a representative of The Business, but for the IT team members, it is often extremely beneficial to also be able to directly interact with the target audience. Ideally (though in practice it seldom if ever happens), all team members would have some exposure, if not hands on experience, within the target environment so that they can have some appreciation of the potential impact of their decisions.

It is also important for the IT team to be reviewing requirements, even if in draft form and contributing to estimates and decisions from the inception of the project. Often the IT team are brought in only after the requirements have been "finalised".

Which neatly segues into the next section.

What is the role of a BA in an Agile project?
There are a growing plethora of urban myths around what an Agile project is and what it can achieve, and this posting cannot do any justice in addressing them, so I will not attempt to. There are also quite a number of criticism's emerging around Agile, some of which are in my opinion quite valid. Again, I am not going to attempt to address them in this post. The following will likely make me sound like a strong advocate of Agile processes. In truth, while I see much benefit from the process approach, I am also somewhat critical of it. But that can be left for another day.

To understand a BA's role in an agile project, it is important to understand what an Agile project is.

I guess the best place to start is the Agile manifesto. There are a number of Agile methodologies, but the two most commonly encountered are probably eXtreme Programming (XP) and Scrum. There are a number of methodologies jumping on the bandwagon of Agile, that are mostly Agile in name only. While it seems to be growing in popularity in the business domain, Six Sigma in my mind is one such example that is not an Agile methodology, despite its coupling of late with Lean, out of which the principles of Agile have their origins.

Essentially, the main focus of an agile project is to reduce the feed back loop from the time something is requested, to the time it is validated. At the micro level, this involves practices such as continuous build environments, Test Driven Development, daily stand up meetings, and Pair Programming up to the macro level where iterations are kept to short durations and feedback elicited from the target user as often as possible.

There is also greater emphasis on interdisciplinary cooperation within an Agile project. The traditional process is horizontally layered, where each layer is the responsibility of a dedicated team of specialists, and they have little interaction with team members of other layers. Typically it would be layered something similar to the following:

The Business (receipt of project)

Production Support and Migration Teams

Testing Team

Software Development Team/Infrastructure Team

Business Analysts (requirements gathering)

The Business (request for project)

In an Agile process, it is more vertical, with the above structure turned 90 degrees. Ideally, you would have a representative from each layer in a collaborative co-located team, and the team as a unit is responsible for seeing one "story" proceed from inception through to completion. In a full XP model, these team members would be swapped between teams by up to twice a day so that everyone has an understanding of all aspects of the project and the key man risk is mitigated.

This does not mean that requirements gathering, or documentation is no longer needed. However, the process of capturing them and the format they are captured in will not be likely to be as per a traditional process. Decisions still need to be made, recorded and prioritised, rules still need to be captured and referenced, processes still need to be mapped, communications still need to be compiled and disseminated.

These are the skills that BA's are meant to be competent in, if not their speciality, and cannot be replaced simply by the developers and testers having direct access to, and talking directly with The Business to validate things for themselves. Those communications will still require facilitation. In a pure XP implementation, there would be no BA, only The Business involved in requirements elicitation and functional and user acceptance testing. However, in reality, even SME's from The Business often lack the skills to adequately do this task, and this is where the role of the BA can help fill this space.

An Agile project is fast paced, consisting of consecutive short sharp development cycles for the members of these sub teams. One of the biggest challenges for a BA is to be able to capture the requirements to a sufficient level of detail for the developer and tester within the current iteration, inside of the required time available, without leaving the developers waiting, while still being available to field any questions with minimal response time from both developers and testers.

If anything it arguably requires a BA to be even more capable and disciplined on an Agile project than with any other methodology.

Additionally, not all solutions involve IT. It is my opinion, that IT solutions should only be seen as a last resort, not because of the human impact factor, but because of the high risk associated with IT project deliverables, not just in cost and schedule but also acceptance and fit for purpose, when often there is likely to be a cheaper, lower risk alternative in business process change.

Unfortunately, we seldom model the risk aspect into our valuations of alternatives. But even with a purely business orientated process change, there is no reason why such an initiative could not also be done using Agile principles.

I should point out though, that I am painting the picture of a BA's role in a large corporate environment. For small web development outfits, who usually do not employ BA's anyway, going Agile for them will not change their perceived lack of need for a BA.

What is the future of Agile?
It is always dangerous to Crystal Ball gaze, but here goes for the next few years.

Many large corporations have commenced using some form of Agile in some of their development. This penetration into the corporate IT industry will continue to increase.

Many small outfits risk tarnishing Agile with cowboy behaviour masked under the aegis of "Agile", when in reality they are not following anything but a chaotic methodology.

The hype will begin to dissipate as per every previous "silver bullet" to hit the industry over the last 50 years, and the Agile zealots will be marginalised and voices increasingly ignored as Agile is absorbed and morphed into something else that deals with distributed development teams of varying mediocrity, and silo corporate entities across which even the smallest of corporate projects find the need to navigate.

The focus of project management will begin to change to be more inclusive of all stakeholders, which can only be a good thing, and there will be an increasing expectation from customers for earlier deliverables of incremental releases, which will change how system testers need to approach regression testing, and production access is modified to allow more frequent migrations. Project managers will have a wider range of tools and strategies from which to choose to apply to a project, and the increasing support of the corporation in using them.

There will be increasing tension between the likes of CMMI/SPICE/V-Model processes and Agile based methodologies, including with the likes of government contractors, as Agile projects struggle for official endorsement and recognition as valid methodologies.

But at the end of the day, there will still be a need in the corporate space for a BA, regardless of the process used on the project.


10 comments:

  1. A few observations that may be of interest:
    1. The activities of business analysis are critical on agile projects, on all projects for that matter. That doesn't imply the need for someone in the specific role of BA.

    2. Developers do in fact have communication skills, and can in fact pick up BA skills.

    3. We're moving away from specialists towards being generalizing specialists. If one of your specialties is doing BA work that's great, but you need more than that to fit in on an agile team. See

    4. For agile@scale situations having someone in a more specialized role can make sense. The implication is that we will need some people in the role of "agile BAs" (or equivalent) but nowhere near as many as what the BA community is currently trying to push for. These roles will likely be called "product owner" and not BA.

    5. You can scale the agile role of customer/product owner via agile BA skills. See Scaling On-Site Customer (and Product Owner)

    6. We've been talking about agile approaches to business analysis within the agile community since 2001. See Agile Analysis.

    - Scott
    http://www.ibm.com/developerworks/blogs/page/ambler

    ReplyDelete
  2. I see Scott got here first... :-)

    My view on this topic is that only a small percentage of IT solutions involve new custom software, and likely only a sub-percentage are good candidates for Agile. I would think the main problem faced by Agile is continous business involvement.

    So, that leaves the majority of projects where a BA role and other specializations are effective, especially COTS package acquisition; as a result, I have no fear of my role fading away because of Agile. What I can see, if not yet fear, is continuing improvements in the communication between Business and IT; if no translator is needed, then maybe BA's will be needed less.

    ReplyDelete
  3. I appreciate your comments Scott. I do agree with you in that the actual role that a BA plays does need to change, in my opinion regardless of what process is being followed, Agile or not. It is a topic that has been suggested to me to blog about. The current blog was getting too big as it was, and the topic is worthy of its own space.

    However, I don't swallow the Agile mantra hook, line and sinker. It is fantastic in theory. For instance, I'm something of an oxymoron myself, in that many I work with regard me as a generalised specialist. However, I'm rare in my cross discipline abilities. As such, where do I find others like me to populate an Agile team with?

    The argument exposes what is in my mind one of several weaknesses of Agile. I think it was Martin Fowler who said that one of the key dependencies on the success of an Agile project is having high calibre team members? I would have thought that regardless of process, having a team of high calibre members would be a big plus. Unfortunately, the world is not full of high calibre individuals and you need to cope with the mediocrity of the masses. The majority of these people regardless of discipline who are available are specialists, and usually not even good ones at that, not high performing generalising specialist oxymoron's like the Agile movement suggests are required. And in a corporate environment, the years of experience is also usually a big variable.

    If Agile is to have widespread penetration in practice, not just in name, then it needs to cater for the "normal" population, not be an elitist process requiring only above average team members.

    Even though I am a developer, it is also a concern to me that the Agile movement appears to be a completely developer driven process, and then when you say the agile community have been talking about... BA's, where has been the collaborative inclusion of BA's into these discussions? Did you have a BA voice or representative during your deliberations on a BA's role in an Agile world? Or was that voice the BA's trying to push for more than in the eyes of the agilists are needed?

    Personally, I believe that a minimum skill set for developers is communication skills. Anyone working corroboratively regardless of discipline requires the ability to communicate. However, most of the Agile software developers I have worked with, such as from TW, are generally arrogant and abrasive. Sure they can "communicate" fine, but they have no idea of relationship building and tend to play out the very weaknesses mentioned of BA's in one of your links, in having undue influence in decisions. They do not elicit requirements but dictate them.

    What I have seen to date has not provided me with any evidence that putting requirements analysis completely in the hands of developers, as some of your links suggest, is such a good idea, even with "high calibre" Agilists. Though I do agree with many of the issues raised regarding deficiencies of BA's in your links. However, I see these issues related to anyone doing requirements gathering, not just confined to the practice of BA's.

    So in this regard, I think we will have to agree to disagree, even though I'm probably in almost complete agreement with you on the issues with the BA role, it is in the conclusions we draw that we differ. I refuse to chase the theoretical ideal as seen through the agile prism, but try to apply what aspects I can to the real world constraints with which I have to work with. And in that world I see the BA role maturing and changing, but not going away.

    ReplyDelete
  4. COTS = customised off the shelf.

    ReplyDelete
  5. I always thought the "C" in COTS was "Commercial"...

    ReplyDelete
  6. David a quick google check and I learn you are right. Just me assuming customised sionce everything always is.

    ReplyDelete
  7. As a business analyst, I don't see my role disappearing any time soon.
    What I see is an evolution of development lifecycles.

    I think that Agile is just the next step in how to address risk factors in projects. It's a good way to get early feedback on what the team is doing. That makes it much more likely to get a good product at the end.

    That doesn't seem to be mutually exclusive with what I do in any way. I work with the customer to help them explain their requirements. I also help manage their expectectations as well. It is usually my job to explain that yes, you will have a product you can look at in 4 weeks but no, it won't do everything we spoke about.

    Can a developer do that? Some of them, yes. Is that where they should be spending their time? Depends on the size of the project and the team you have working on it.

    Remember, this is not a one size fits all world. We are not all interchangeable parts.

    ReplyDelete
  8. Blogs are good for every one where we get lots of information for any topics nice job keep it up and Pretty good post, this is one of the best articles that I have ever seen! This is a great site and I have to congratulate you on the content.
    I appreciate it!!!!!!!!!

    ReplyDelete
  9. Anonymous8:52 am

    Business is a broad field. Many surpasses and many hinders. Success is achievable with great skills.
    sonicare diamondclean best price

    ReplyDelete
  10. Ellen Gear9:50 am

    Thanks you for your views. I learned a lot with your post. Reading the article, it fills mind.
    buy lalaloopsy silly hair dolls

    ReplyDelete

Search This Blog