BA — a crucial ingredient in a successful scrum project?

By now, we all should be able to recite the Scrum roles in our sleep, however, the role of the “business analyst” is suspiciously absent from that list. Does that mean that the skillset of a competent Business analyst has no place within modern agile processes?

The business analysts, what are they good for?

Let’s begin by stating that this text should apply to any iterative/agile framework, but we’ll focus on Scrum since it’s (for better or worse) the best-known and most widely adopted one.

I’ve written (rather extensively) on the general role of the business analyst in the solution design process, however, I haven’t explicitly covered the topic of how the business analyst fits into the agile methodologies in general and scrum in particular. This text should fix that.

It stands to reason that in order to figure out where, and if, the business analyst “fits” in the scrum process, we first need to define what skillset and overall capabilities your “garden-variety” business analyst (*) brings to the table.

(*) please note that the BA role/job description tends to vary significantly between organizations, so what I’m describing here *is the BA role as I see and practice it* and, consequently, the way in which it is “configured” in the organization where I work.

So without further ado, these are the main skills that a typical business analyst should bring to any game:

1. An ability to elicit/discover functional requirements and business rules — through communication with one or more “knowledge owners”
2. An ability to document functional requirements and business rules — ie. the mastery of requirements documentation techniques and formats such as a) use cases, b) acceptance criteria, c) user stories, d) actor-goal pairs, d) business rule examples, etc.
3. An ability to manage functional scope —ie. ability to “slice” or partition the functional scope into adjacent, autonomous (as much as possible) functional segments of various sizes
4. Ability to adjust the detail level for any particular functional segment (ie. quickly and easily switch between “birds eye level” and “low-level” views of the particular feature or a functional segment)
5. Ability to validate functional completeness of any given functional segment and the system as a whole
6. Deep and thorough understanding of both the top-down and bottom-up design processes

Also, it’s worth noting that a good Business Analyst should be able to perform all the skills mentioned above, quickly and flexibly in an agile setting — just as well as in the large “design everything up-front” waterfall-y scenarios.

So with that in mind — let’s see whether those listed BA skills might come in handy in your typical scrum process.

The scrum process is composed of a well-defined set of events/ceremonies, let’s try listing them and see whether there are some that might benefit from a Business analyst’s skills we’ve listed before.

• Backlog design workshop — typically happens before the first sprint (or is sometimes the sole purpose of the first sprint — often called “sprint zero” in this case) and its goal is to create a rough functional scope definition for the project and produce the material for the first refinement session. This is typically done by producing a full set of (or at least as complete as possible) high-level user goals (a.k.a. “epics”) which define the scope in low detail and a partial set of medium-level user goals (a.k.a. “user stories”) that define one or more epics, or their segments in more detail. 
• Backlog refinement session — its goal is to produce the “raw development material” for one or more upcoming sprints in the form of fully defined story cards (or analogous work items). This is typically done by taking one or more items from the backlog, slicing them up, and detailing them, resulting in a number of “high detail level” or “development ready” story cards (a.k.a “work items”). Said work items usually need to have at least the following defined: 1) user story — defining the goal and giving the context for the work item, 2) a comprehensive set of well-structured acceptance criteria, 3) any additional information (business rules and examples, UI wireframe references, etc.) which anyone who gets to implement this particular work item might require in order to complete their job without needing to look for any additional information elsewhere (a.k.a “the single source of truth”). 
• Planning session — happens every sprint and its goal is to prioritize the backlog items and plan the upcoming sprint 
• Retrospective — happens every sprint and its goal is to figure out whether the sprint went well, and whether we can improve something in the overall process
• The demo session — happens every couple of sprints (exact timing depends on a number of factors — but it’s not important for our discussion) and the goal is to introduce the project stakeholders to the product increments with the ultimate goal of giving them the ability to gauge the direction the product is taking as early as possible, and subsequently to change it if they see the need for it.

Please note that those sessions mentioned above are not all the sessions of a classic scrum process, there are also daily standups and we didn’t mention the development at all. Also bear in mind, that these listed sessions and events tend to come in many varieties and their details and nuances can and do vary between different scrum projects and organizations that implement them, but for the purpose of this text, those mentioned above should suffice.

Right off the bat, there are two events that stand out as good candidates for employing the skills of a Business analyst, and those are:

1. The initial backlog design workshop (or workshops — plural) and
2. Backlog refinement sessions.

The initial backlog design workshop is actually very similar to the functional system design process, the main difference being the required level of detail of the resulting end product. Typical functional system design results in a fairly detailed SRS document and the initial backlog design workshop usually focuses on the actor-goal pairs (or user stories — it’s mostly the same thing if you think about it) and leaves the detailing (ie. acceptance criteria, use cases, etc.) for the backlog refinement sessions.

So for the initial backlog design workshop — any business analyst should feel right at home and should be able to add significant value to the design process and the resulting deliverable — and as a consequence, make the product owner’s job much easier.

For the backlog refinement it’s basically the same story, but this time the focus is on details, scope management, and precise acceptance criteria, rather than on functional completeness and high-level scope, and this is the portion where all that requirements documentation mastery really comes into its own. 

Funnily enough, the main benficior of this involvement is the same as for the initial backlog design session — ie. the product owner (although the scrum master and the development team should really feel the positive impact as well)

Other sessions/events don’t benefit much (at least not directly) from the presence of the business analyst (or someone with the same skills), but these certainly do.

So, if what I’ve written stands, then the conclusion that follows is: 

By now, we all should be able to recite the Scrum roles in our sleep, however, the role of the “business analyst” is suspiciously absent from that list. 

Does that mean that the skillset of a competent Business analyst has no place within modern agile processes?

Sadly, in my personal (and rather extended) experience, it doesn’t.

I’ve directly participated (or in some cases, I have been an active observer) in a number of scrum projects of various sizes and complexity, and more often than not they struggled at least to some degree. What I found interesting (and telling) is the fact that the common thread in all those projects which did struggle was that the work items were poorly defined, and those projects which had their work items (story cards, etc.) defined well, usually also performed well.

Please note that “well defined” does not mean “heaps of written information” — just that the *information that was provided* on them, was of good quality, functionally complete, and consistent.

Typically work items (a.k.a “story cards”) on the projects which struggled fell into one of these two categories:

• work items contained insufficient information — information typically consisted of a (malformed or incomplete) user story, or some other version of “goal statement” and either comically sparse and underdefined acceptance criteria or (seriously) no acceptance criteria at all
• work items contained a lot of incomplete or contradictory information — those work items utilized some sort of formal format (ie. gherkin for acceptance criteria, etc.) and even made a big deal of using that particular format, while the information conveyed within that format was incoherent, contradictory, incomplete or (often) all of the above

The primary reason for those problems was that no one in the team really knew how to define requirements correctly, identify functional holes and write proper acceptance criteria.

And to make things worse, the scrum framework is very vague and ambiguous when it comes to those topics. It’s simply assumed that said knowledge and capabilities “just exist” within the individual participants’ skillsets, that the design and definition of well-formed user stories, management of functional scope, etc. can be done by the product owner, and that the acceptance criteria and functional decomposition are a collaborative process between the product owner and the development team.

Unfortunately, that doesn’t really work IRL. And the simple logic behind that statement is this:

If no one in the scrum team has sufficient skills required for functional design, documenting the requirements and overal scope management, then no amount of collaboration between those same team members will somehow produce those skills.

This is a problem since *those skills are essential* in avoiding the poorly defined work items I’ve mentioned before.

The Scrum framework in general, and agile manifesto, in particular, puts heavy emphasis on communication, collaboration, and self-organizing teams, and that’s all fine and dandy when you’re talking about a team of experienced individuals where at least a subset of them possesses all the skills I’ve mentioned so far, but unfortunately, the reality is that you most likely won’t be working in such team, but rather in a team of individuals with varying degrees of skill and experience and where very few (if any) of your team members will have any requirements documentation/capture/management skills on any significant level.

Let me be clear here, I’m not stating that the scrum doesn’t work, it does, and I’ve been a part of successful scrum teams and have witnessed and participated in their success — so the core idea of a process like a scrum is not invalid or problematic in my experience.

What I’m saying here is rather that the scrum framework was designed with a team of competent, well-rounded, self-organizing individuals in mind, where those individuals possess all the skills needed in order to make the process work – but “in the real world,” the framework is implemented in an environment where those assumptions simply aren’t true.

My argument here is that this “missing/crucial ingredient” or rather “the missing set of skills” is that of the business analyst and that by adding one in the mix and mitigating those issues I’ve mentioned, you can significantly improve your scrum teams’ performance.

When it comes to the details and questions of whether the business analyst should be a permanent member of the scrum team or an “external domain expert”, I’ll leave those up to you — both approaches work IMHO, but I’ll implore you to try the following.

If you do happen to have Business Analysts (or functional designers) in your organization, the next time you encounter a struggling project which uses scrum as its methodology and especially if that project exhibits the classic “symptoms” I’ve described in this text, just try adding the BA to the mix for a few sprints and refinement sessions in a way described in this text, and you might be pleasantly surprised(*).

(*) or if you are not — you can always contact me, explain to me that I’ve got no idea what I’m talking about and that you curse the day that you read my blog, but please augment your well-deserved critique with some examples of what happened as I’ll be genuinely interested in hearing what went wrong and perhaps even be able to point you in the right direction.