acceptance criteria | Geek 2 Man

I had a manager once who would constantly harp on both the product team for not writing stories with good acceptance criteria and on the engineering team for accepting these “poorly written” stories. He would claim that his engineering team was the engine, and it would only perform as good as the fuel you gave it.

Better fuel makes things perform better, whether it’s high octane gas for your car or great acceptance criteria for your engineering team

If you’re familiar with Agile, then you’ve probably had the general story format of “As a [USER] I should be able to [DO SOMETHING] so that I can [ACCOMPLISH SOMETHING].” beaten in to you. It’s a great concept, and as a Scrum Master, I can say that it does work. Of course, you don’t have to take my word for it (even though I hope you will). Mike Cohn of Mountain Goat Software, one of the authorities on Agile and Scrum discusses the advantages of the As a User, I want User Story Template.

Here’s the one problem I have with using that format: Engineers get their hands on it. They bastardize it. They take it to be gospel. They use it to not have to think.

I’ve had engineers who would write system level stories with things like “As a System, the System should have Logging, so that the Developer has something to look at with things go wrong.” Gee, that’s nice. What are you trying to log? How will you access the logs? (which should really be another story quite honestly…) What sort of things do you mean by “go wrong?” And of course, my favorite bit, “As a System, the System should…” Is the web server actually sentient and have a concept of self? (Try and bend the process and I’ll pick anything and everything apart…)

Then there’s the individuals who just don’t want to follow the format. “Why?” they whine. “What’s wrong with just attaching a picture of the white board from the meeting we were all just in to the story? The devs who will build it are in there.” You should definitely add the picture. It’s an artifact to the process. But don’t rely on it! Weeks could pass before the team gets to that particular story. A crucial team member could get hit by a bus or win the lottery. You need context to the story.

I’ve seen issues even following the format to the letter. This arises from the statement being too vague and the product owner expecting more than they wrote. The engineers take those vague statements as gospel, and then hide behind it as the story doesn’t pass, or ends up having three more stories written to clarify the original work. They use it as a crutch to not have to think about more than what was written and potentially not deliver greater value. I once saw a story rejected because the statement was “As a User, I want to be able to log in and see what I have permission to see.” A little vague, but tolerable. The story was failed because of a different issue; the product owner didn’t like the error messages displayed if it was a failed login attempt. Funny, I didn’t see anything in the sentence regarding failed login attempts… You know what would have alleviated this issue? Acceptance Criteria.

By explicitly stating after the “As a User…” statement what the expected results would be this could have been avoided. Acceptance Criteria for the above log in story could be as follows:

The user will be able to enter their user ID (email) and password and select the button labelled “Login” to log in to the system.
If the user enters an incorrect ID and password combination they will be presented with the login screen and an error message above the fields stating “You have entered an invalid ID or password.”
When the use successfully logs in they should see content appropriate to their User Roles:
- Standard users will see their dashboard with content they have subscribed to defaulting to the last 30 days ordered from newest to oldest
- Staff will see the internal dashboard
- Admins will see the internal dashboard augmented with extra tabs for user account management and subscription payment management

So how do you write good acceptance criteria?

Think of acceptance criteria as a helping guide for how you or someone else would test this work. No amount of detail should be spared on your road map of how to verify the story. In some cases it could be as detailed as an AAA TripTik (If you ever had a paper one made for you, you’ll know what I mean). It often helps to draw out a flow chart or build out a spreadsheet of desired actions and possible outcomes. For the login example, the spreadsheet might look something like this:

Login Acceptance Criteria Example

Page	URL	Action	Expected Result	Data Needed
Login Page	/login	Login as a standard user	Standard users will see their dashboard with content they have subscribed to defaulting to the last 30 days ordered from newest to oldest	User: standard@sample.com Password: password
Login Page	/login	Login in as a staff user	Staff will see the internal dashboard	User: staff@sample.com Password: password
Login Page	/login	Login as a system admin	Admins will see the internal dashboard augmented with extra tabs for user account management and subscription payment management	User: admin@sample.com Password: password
Login Page	/login	Attempt to log in as an invalid user	User is redirected to the login screen and is given an error message above the fields stating “You have entered an invalid ID or password.”	User: baduser@sample.com Password: badpassword

While this will often fall to a designated QA individual or team, not every organization is set up to have a dedicated QA group. Regardless of if you have QA available to you or not, be sure to write the acceptance criteria with enough detail that anyone in your organization could follow the steps. It will help with testing, and it will help with development. If you do have QA available, engage them to help you write Acceptance Criteria that will help all of you. They’ll have plenty of tips to give.

So back to the fuel… You can give your engineers basic requirements, or you can give requirements and acceptance criteria. One will act like 87 octane in a Corvette; you’ll get there, but there will probably be some sputtering and a few issues down the road. The other is going to be like putting in 93 octane in that same Corvette; it’s going to purr and give you much better performance in the end.

You’re probably thinking to yourself right now, “Great, he’s about to give us another recipe example for breaking down tasks and achieving the desired acceptance criteria.” Well, stop that. The food is merely a vehicle for this post.

First, a little background: I’m currently on a business trip, and as my hotel room has a kitchenette I did a little shopping at the grocery store; a Publix to be precise. Why Publix you ask? Well, we don’t have those in our neck of the woods, so it’s something different for me. After all, I can go to Kroger’s any time I want. Back home, all the neighbors have been asking my wife where I’m travelling and what I’m doing (they do the same to me when she travels for work). During the course of one of these conversations a neighbor brings up that there’s no good store-bought hot cross buns where we live, and that the best ones he’s ever had were from Publix. After all this, my wife texts me:

Publix Market has the best hot cross buns… Do I need to bring some home? You’d make [our neighbor] happy. I’ll head there tonight for some. LOL

Now, stop and read that again. This time, read it as a product owner and an engineer. The hot cross buns here are almost a feature request, but not quite. It could be a preliminary fact-finding discussion to see how feasible it was to achieve an objective; getting hot cross buns. Or maybe it’s just a discussion over some cool feature that was seen elsewhere that the product owner just wanted to share. Usually it’s something like “checkout this cool slide show transition.” This time it’s just a statement of where to get the best hot cross buns.

It just so happens that I’m in a position to find and obtain these incredible edibles. Being the engineer that I am I immediately see this as a problem that needs solving, and offer a solution: I’ll get some and bring them home from the trip. It’s not so different from, “You like that slide show transition? Let me see if I can slide that in to the sprint for you. Don’t worry about a ticket…” Nowhere in the text conversation did my product owner ask for hot cross buns. We had a phone conversation afterwards, and the conversation continued, and I said that I would stop and get a box. I figured I’d pick them up after dinner on my way back to the hotel. Note: my product owner neither confirmed nor negated this suggestion. In fact, I think she may have been working on another project.

Off I go to dinner (I’m starting work on the slide show, and said that I’d add the undocumented, un-asked-for transition). The restaurant I went to (recommended by coworkers) had an hour wait, and being the hungry engineer that I was I passed and went next door since it had empty tables. Of course, with my engineer’s luck, it was a bakery. And wouldn’t you know, they had hot cross buns… Now, these aren’t the ones that would satisfy my acceptance criteria. They’re not Publix hot cross buns. But, being the engineer who always wants to please, and always wants to deliver more, I eye those hot cross buns. Of course, I still want to do the right thing, so what do I do, I contact a product owner. I text the neighbor. Not the product owner I had the initial discussion with, but now a totally different one! I chose to ask a different product owner on my own, not because the original product owner was unavailable or had passed me off to a new one – or was out of the office and designated a replacement. Now I’ve made promises to two different product owners (my wife and neighbor), for two different acceptance criteria (Publix and bakery hot cross buns), and still no official request within my sprint (dinner time/business trip). But I’m the engineer, I’m supposed to say how things get implemented, whereas the product owner gets to tell me what gets implemented. I decide to treat the purchase of the baked goods as a how, not the baked goods themselves being the what, and I buy some from the bakery. Better still, I don’t buy just one, but three boxes of hot cross buns. After all, if I were programming those transitions, then three unasked for transitions and some other features would be better and make my product owner happier than the one discussed, but still unasked for transition I offered to do. Right?

Guess what? My product owner wasn’t very happy when I told her what I had done. Where my engineer mind thought I was going above and beyond, in reality I was adding my own scope creep, wasting time and money on something that wasn’t asked for. If this was development, I would have put my sprint and deliverables in jeopardy, and wasted the company’s money on my salary for doing things I wasn’t tasked to do. I still put my deliverable in jeopardy, as these aren’t Publix hot cross buns, but fresh bakery ones, which won’t last nearly as long.

Plus, since the Hot Cross Buns were not in the original sprint plan (business trip), I have circumvented a process that is in place for a reason. Remember that as a developer, the process is there to protect you too, but if you start agreeing to do things outside the process and without documentation, you could find yourself in a lot more trouble than you initially bargained for. Plus, there’s always the “but you did it for me with the Hot Cross Buns…. Why can’t you do it for me with the Coffee?” argument that the product owner can make. Or the you did it for product owner 1, but you won’t do it for me…. argument (which could get you into all sorts of HR issues.)

So the next time you want to do a favor for your product owner, make sure to agree to what is being requested, then prioritize the work, and document it. Don’t go assuming that they want something just because of a conversation or email. Don’t go involving other/different product owners with your questions because you’ll only get conflicting marching orders, and acceptance criteria. If you think there’s a better way of doing something, discuss it with the original product owner, as there may be a reason they are specifically asking for something, and there’s no need to add in all the extra work you’re considering. Instead make suggestions on how to improve upon their improvement. And finally – Just remember the hot cross buns, and your product owner won’t get cross with you when you deliver.

As for all those hot cross buns? It looks like I have a lot more breakfast items than I had planned for…

Tag: acceptance criteria

The Rocket Fuel of a Project – Good Acceptance Criteria

Login Acceptance Criteria Example

Hot Cross Buns as Acceptance Criteria