Demos are for Getting Feedback

bigstock_Showing_966692I get the opportunity to talk with many teams and product owners about their software development efforts. A common thread of questions tends to  be around demos (also known as sprint reviews, showcases, or any number of other terms).

Before delving into specific questions, it may be helpful to address what value demos provide. Demos exist primarily to get feedback from your stakeholders when you can’t get feedback from them during the course of a sprint.

Note I didn’t say that they are to get feedback from the product owner. If you are truly doing your job as a product owner you are providing feedback along the way so that your team has a shorter feedback cycle and has an opportunity to act on the feedback you provide within that sprint. If you (as a product owner) is surprised during a demo, something is wrong.

With that thought in mind, here are some frequently asked questions answered from a product ownership perspective.

Should we always do demos?

I am going to answer this question with a question.  Should you always look for feedback?

Ok, a little snarky I’ll admit.

If you as a product owner have already had an opportunity to provide feedback to the team during the course of the sprint, and you were able to get feedback from the key stakeholders interested in what the team was working on, you probably don’t need a demo.  You already have the feedback, so now you may just be doing the demo because that’s what teams do at the end of sprints. Unless of course you are using a flow approach, in which case you better be getting feedback after every item is done.

My guideline: Do you need more feedback? Do a demo.

What if we don’t have stakeholder (customer) visible stuff, do we still do a demo?

Some teams (teams working in a mainframe environment or teams with a LARGE, complex code base come to mind) may find themselves working on stuff that provides value to stakeholders or customers, but is not very demonstrable.  If you find yourself in that boat, think through what would be a good way to get feedback on the work you did. Mainframe teams I worked with used before and after screen shots.  Admittedly not very sexy, but it was sufficient to get the point across and to generate conversation and encourage feedback.

My guideline: Do you need more feedback? Do a demo. Figure out how to show something meaningful to get that feedback.

Who should attend demos?

It’s good for the entire team to attend.  Is this some “agile mandate”?  No, (there is really no such thing) it’s helpful for the entire team to hear the feedback they are getting from stakeholders (customers) so that they get a better understanding of what stakeholders are looking for. They may also pick up on something that other team members don’t.

It’s also helpful to consider which stakeholders have an interest in the stories the team is working on during the sprint. This is a good discussion to have during sprint planning so that you can specifically invite the people who have a vested interest in the backlog items the team will work on during the sprint. Taking this approach instead of creating a blanket invite to hundreds of people who may possible be interested in what you are doing some time helps you have more engaged attendees at your demo, and is more respectful of everyone’s time and calendars.

If you are working on a software product, identify customers that you would like to include in your demo. You may not include customers in all of your demos, as you may not have something to show meaningful to your customers (hopefully that doesn’t mean you didn’t work on anything that delivered value.)

Remember whomever you invite, don’t make it hard for them to give feedback.

My guideline: Include anyone that you need feedback from and who would benefit from hearing their feedback.

Who should do the demo?

It depends what you are trying to accomplish (beyond receiving feedback that is) and the makeup of your team.

I’ve seen the developers demo their own stories.  This is good if you’d like to get some recognition for the developers and they won’t go off the rails and give a 30 minute discourse on the nasty details of how they completed the story.

I’ve seen analysts or testers do the demos.  This is usually the case when they have a strong relationship with the stakeholders and the team feels that they can best convey what the team accomplished.

I’ve seen product owners do the demos. Same reason as why analysts or testers do the demo.  Just make sure you as product owner aren’t doing the demo because you don’t trust the team to do it properly.  This is a sign of a dysfunctional relationship with your team.

I’ve seen members of the business unit who are working closely with the team do the demo.  This is often a way of subtle organizational change management.  If other members of a team see someone from their team using the new solution, they likely conclude that it must not be that scary after all and is something they could use.

Perhaps the stakeholders (customers) should do the demo themselves by actually just playing around with the software.  In fact, that may be the most effective way of getting meaningful feedback, and certainly something to consider if your solution is in a state that stakeholders could start using.

My guideline:  The person who is best positioned to ensure you receive the most useful feedback is the one who should do the demo.

What questions did I miss?

Do you have other questions about demos?  Do you have a different perspective on something I suggested above?  Do you think demos shouldn’t be called demos?  Share your questions or thoughts in the comments.

Agile2015 Highlights

I’m writing this as I fly home from the Agile2015 conference, or what Jake Calabrese rightfully calls “summer camp.”  I headed into the week planning to focus on sessions related to product ownership or one of it’s related fields, as I described a couple of weeks ago. As a result I found myself attending several user experience related sessions, which helped to expand my understanding of that field.  I also attended some sessions on topics I’m a bit more familiar with such as ideas from Lean Startup, and I also attended a couple of sessions that just sounded fun.

Below is a quick summary of the sessions I got the most out of. These sessions tended to discuss fairly simple techniques but did a deep enough dive that I was able to get some good nuggets out of them.  A few of these sessions prompted some ideas that I’ll explore in more depth a bit later.

The link to Session Details takes you to the conference’s online conference and depending on whether the presenter uploaded it, you’ll be able to get the presentation from that location.

Consensus That Sticks

Session Details

Jeremy Kriegel @sonarc

The Gist

Graphic facilitation can be very helpful for helping a group identify options and reach consensus (i.e. a decision all can support). Affinity mapping is a useful technique that consists of three key steps:

  • Idea capture (Real-time/Individual)
  • Organizing (Unstructured/Structured)
  • Prioritization (Dot Voting/Forced Prioritization)

The different variations of those steps are most helpful in different contexts based on what you are trying to accomplish.

Key Takeaways

I’m fairly familiar with affinity mapping, so my key takeaways from this session are related to some insights about the various steps of the technique.

  • Remember that early ideas stick – people in the team will tend to base new ideas off the first few, so you may limit the variety of options you receive
  • Be careful not to get too vague in the header categories.
  • Dot voting is a good way to get a feel for the room, but really shouldn’t be used as the sole means of determining a specific decision. I like to use it as a way of deciding the order in which we discuss things further.
  • Using structured organization where you place items relatively on two axis (such as Value and Ease of Implementation) is a good way to have discussion about characteristics of the ideas.
  • Deciding which risks to mitigate and assumptions to verify can come down to a tradeoff between the cost of being wrong and the value of knowing.

Prototyping: Iterating Your Way to Glory

Session Details

Melissa Perri @lissijean
Josh Wexler @josh_wexler

The Gist

Iteration is the key to learning successfully. Cognitive biases block your ability to iterate and to learn. Prototypes are a great way to help us communicate and learn and avoid cognitive biases.

Key Takeaways

Three kinds of prototypes:

  • Conceptual
  • Experiential (wireframes)
  • Technical

Three qualities prototypes can exhibit:

  • Visual: Sketched – Styled
  • Functional: Static – Interactive
  • Content: Sample – Actual

It’s helpful to establish a persona and describe the story of how they may interact with the solution. You may start out with a text version of a customer journey or story map.

As a starting point it’s helpful to do your prototypes in black & white so people don’t get too distracted by the color.

Create the prototypes and then sit with customers and have questions in mind to ask them. Make sure the questions are open ended and are not leading.

Example Mapping

Session Details

Matt Wynne @mattwynne

The Gist

Gherkin alone does not make good acceptance criteria. Likewise writing Gherkin and chucking it over the wall does not mean you’re doing BDD. The important bit is the conversation that leads to shared understanding.

Key Takeaways

Rules and examples are both important, and it’s best to be in a situation where you have both.

Three Amigos is a black box where a story goes in and additional stories, rules, examples, and a shared understanding comes out.

The examples that come out of Three Amigos should not necessarily be written in Gherkin. Rather, it’s helpful to use the Friends Episode naming pattern (The one where…). Likewise, rules can be named “What if…?”

Not every rule needs examples.

Helpful References:

Slides showing how to run an effective Three Amigos discussion

Cucumber School

Product Owner Value Game

Session Details

Dajo Breddels @dajobreddels
Paul Kuijten @paulkuijten

The Gist

A game that teaches people to think about backlog refinement in a different way. The game is most effective when it is combined with active facilitation while groups are playing the game.

Key Takeaways

The deck of game cards.

#1 Learning objective for a game for Product Owners is how to become more value driven.

Backlog Refinement (specifically sequencing) comes comes down to: What’s most valuable to do at a specific point in time?

Not everything that has value can be measured. Not everything that you measure has value.

Helpful Resources

To find out more about the Product Owner Value Game go to

Organizing for Innovation

Session Detail

David Bland @davidjbland

The Gist

Organizing for innovation means:

  • Actionable strategy to define what you will and will not do.
  • Teams give an account with leading indicators
  • Creating urgency & expectations with incremental funding on Horizon 3

Key Takeaways

High retention low satisfaction organizations are prime for disruption

ROI is the worst measure of progress for innovation because it is a lagging indicator. Need a leading indicator to make decisions.

Giving Account > Holding Accountable i.e. Leaders should want their teams to give an account of how they are progressing.

Corporations don’t lack ideas, instead they have trouble deciding which ones to invest in.

What’s my MVP

Session Details

Jeffrey Morgan @chzy
Ardita Kiraj @ardita_k

The Gist

While the idea of the Minimum Viable Product came out of the world of Lean Startup, it certainly also applies to enterprise situations, with some modifications.

Key Takeaways

  • Form hypothesis and validate them often
  • Customer value is priority
  • Start with the end, but look at the whole picture
  • Replace larger systems one piece at a time – not big bang delivery.
  • Keep the users involved.
  • Continuous integration & continuous delivery
  • Keep your MVP small.

Research is not just for the UX Team; Strategies for everyone to understand end-users

Session Details
Amanda Stockwell @mandalaceys

The Gist

A great deal of user experience is doing research. You can basically organize that research into three groups:

  • Business models & goals
  • End user goals, needs and behaviors
  • How well we’re serving those users.

There are a lot of UX Research methods, and you don’t have to be a UX professional to do them (but they can be very helpful).

Key Takeaways

Steps to UX Research

  • Determine research goal
  • Determine research category
  • (Cheat) Rely on Experts
  • Do it
  • Incorporate

Keys to Research Success

  • Ask the right questions
  • Ask the questions the right way
  • Practice logistics

Helpful Resources

A great cheat sheet listing UX Methods and their applicability:

Getting the Most out of Agile2015

Agile2015 is next week in Washington DC so I thought I’d share some ideas for getting the most out of the conference.

I shared my 5 tips for getting the most out of the Agile Conference last year (they all still apply) so I thought I’d provide pointers to others thoughts on attending conferences and end up with some perspective on the sociology of the conference.

Good Lists of Tips

Karen Greaves shared her Top tips to benefit from conferences back in March. I especially like Tip #4:

For every session or talk you attend, write down 1 action that you want to implement. This way, when you look back at your notes this one action will stand out and remind you.

This tip applies to those corridor conversations and Open Jam sessions.

Scott Belsky posted a similar list in his 5 Tips for Making the Most of a Conference. While his tip #4 Plan private gatherings with like-minded folks is a good idea, I’d encourage you to also reach out to people whom you don’t normally talk to. By reaching outside of your tribe, you just may learn something very helpful for you back at work. My goal for Agile2015 is to soak in thoughts from product management and user experience. Two areas I could use more exposure to.

Finally, Rebecca Knight explained How to Get the Most Out of a Conference  in a Harvard Business Review blog. I like her ideas because they provide a lot of good advice for introverts attending a conference. Pay special attention to her suggestions of do’s and don’ts.

The Sociology of the Conference

As the Agile 20XX conference has grown, the nature of the conference has changed a bit. Chris Matts explored this shift in a couple of posts earlier this year: Communities of Need & Community of Solutions and Agile – The Broken Learning Machine. These posts rather accurately describe the nature of the Agile Conference and I understand the temptation to wistfully look back at the “good old days”. I also realize that the conference is what it is in order for the Agile Alliance to support gatherings of Communities of Need where they arise. I can accept this model because it means that the Agile Alliance is able to support smaller local gatherings (the ones that tend to embody the Communities of Need that Chris describes) without having to travel the primrose path of certification.

I share Chris’ two posts because I want to encourage you to have conversations with the other practitioners at the conference and share your stories, both the successes and the failures. Like Chris, that’s where I’ve gotten the most value out of the Agile conference in the past.

Along those same lines, I’d like to point you to Jake Calabrese’s post Don’t Let a Few Thought Leaders Make Us Stupid. His basic suggestion is

Learn. Listen. Ask questions. Listen more. At the end of the day, take in what is out there, but don’t just do it because ‘they said so’

Good advice, but perhaps even better advice is what he closes with

All you need to do is help people be awesome.

Share your successes, and what you learned from your failures. You’ll find you get much more out of the conference if you do.

Now it’s Your Turn

Let me know in the comments your suggestions for getting the most out of conferences. And if you are going to Agile2015 let me know that as well.

Here are the sessions I’m interested in (I’ve got multiple sessions identified so I can decide at the last responsible moment). Let me know if there’s a session I don’t have listed that I just shouldn’t miss.

What is Product Ownership?

Another Way to Say “Building The Right Thing”

Successful software product development and IT Projects face a constant, and healthy tension – you want to build the right thing, you want to build the thing right, and you want to build the thing fast so you can get more feedback. (Thanks to Henrik Kniberg’s excellent video Agile Product Ownership in a Nutshell for putting words to that concept). All three are equally important so certain people on team find themselves naturally focusing on one of those three aims and then working together to make sure that healthy tension results in the best outcome.

My goal for is to provide a set of resources for those who are primarily focused on determining the right thing to build (and determining if building something is the right thing to do in the first place). A challenge I’ve run across is finding a concise term to call that set of activities, and I’m fairly certain I’m not the only one.  For example over the history of the Agile Alliance’s North American conference there has always been a track that inherently been focused on building the right thing, but one program committee after another has struggled with what the heck to call that track.  For Agile2015, it’s called Working With Customers.  I’ve never found that name entirely satisfying.

I’m hesitant to lump it under an existing field (product management, business analysis, user experience) because I’ve found that activities from all those fields come together to effectively determine the right thing to build. I also don’t want to assign it to a specific role, specifically product owner, because of the implication that a single person can do all of those activities on a team. Not terribly realistic in most settings.

Others in the community have thought about this problem and have suggested the term product ownership. While not perfect, I think this term works good for my practices.  It’s fairly recognizable, and it represents a group of activities more so than a role. So that’s what I’m going to go with. It’s also a name I’d suggest to the Agile2016 Program committee.

Product Ownership Defined

With that settled, I suppose it would be helpful to state what specifically I mean when I say product ownership. My brief description of product ownership is determining the right thing to deliver.

My bit longer description is:

  • Understand stakeholder needs
  • Determine if the need is worth satisfying
  • Determine the best solution to satisfy it
  • Build a shared understanding of the solution
  • Validate the need was satisfied

For an even more fully fleshed out view of what product ownership is, take a look at the learning objectives included in the International Consortium of Agile (ICAgile) Value Management Track. Why don’t I use the term Value Management? I find that it is too conflated with concepts from other fields, (including the horribly misnamed Earned Value Management which has nothing to do with earning value.) In addition, the term as applied to determining the right thing to build is not as commonly known in the software product development and IT project communities. That said, I think the learning objectives grouped together under the ICAgile Value Management Track are a good collection of activities and techniques and I consider product ownership = value management.

Product Ownership is a combination of other fields

As I mentioned above, a primary reason I adopted the term product ownership is I see it as composed of activities from primarily three fields:

  • Product management
  • Business analysis
  • User experience

You can think of it this way: (is there any sort of award for the gratuitous use of a venn diagram in a blog post?)


The extent to which you use the activities from each of this area depends on the context in which you are working.  The simplest distinction I can make is between software product development and IT Projects.  Software product development tends to use product management and user experience activities much more than business analysis. IT Projects tend to place a larger emphasis on business analysis, not nearly enough on user experience, and hardly any on product management. Chris Matts took a more thoughtful approach using the Cynefin model as a guide. He suggests that business analysis activities are helpful in complicated domains, whereas product management and user experience are the best tools for product ownership in the complex domains, with the difference heavily determined by the amount of choice the users have in what solution they use.

Unfortunately, these collections of activities are not as clear cut as someone who likes to group and characterize things would like to believe.  There are activities that cross between these three groups (personas for example originated in user experience circles, but have been adopted by both product management and business analysis). So there are definitely some overlap between these areas, and there are some activities that these three fields include which are not included explicitly in product ownership.

Following is my attempt to identify a set of activities for each field that may be helpful to be aware of when talking about product ownership.

Product Management Activities

I tried to pick non commercial views of the activities in a given field, but given the extent of my knowledge of product management, the best collection of activities I can come up with is the Pragmatic Marketing Framework. The nice thing about this model is that it has several very discrete activities which provide a basis for determining what should be included in product ownership, and which fits outside. The downside is that there are several of those activities listed.  All the same it is possible to clearly delineate which activities are relevant to product ownership as I think about it (activities such as User Personas, Requirements, Use Scenarios, Product Roadmap and Status Dashboard)  and those which are important to a business but fall outside the realm of determining the right thing to build.  (Everything else on the Pragmatic Marketing Framework, such as  Market Definition, Sales Process, Channel Training and Support).

This graphic from a presentation that Rich Mironov gave at Agile2009 The Agile Product Manager/Product Owner Dilemma ( Slide 19) provides a nice view of which activities from the Pragmatic Marketing Framework could be considered part of Product Ownership:


Another view of the relationship between product management and product ownership, at least implied, is this one from Steve Johnson in his presentation titled Is Agile Breaking Product Management (See Slide 20: )


While it doesn’t indicate what activities fit in product ownership (all the stuff that would fit in the box labeled product) he calls out all the stuff that doesn’t fit in that box that are still important.  Namely:

  • Define Business
  • Define Market
  • Design and Deliver Promotion
  • Design and Deliver Sales
  • Design and Deliver Support
  • Refine Business
  • Refined Market

Business Analysis Activities

I’ll freely admit that of the three fields I’m discussing here, I’m the most familiar with business analysis.  It’s the arena in which I have worked the most during my career.

The best description of activities associated with business analysis is the Business Analysis Body of Knowledge.  That work is organized into a set of knowledge areas. These knowledge areas include:

  • Business Analysis Planning & Monitoring
  • Elicitation & Collaboration
  • Requirements Life Cycle Management
  • Strategy Analysis
  • Requirements Analysis and Design Definition
  • Solution Evaluation

As well as a set of underlying competencies.

These knowledge areas contain a set of tasks (you could also think of them as activities) some of which are relevant to product ownership, others of which are overhead activities used in phase based approaches.  There is also a set of techniques that go along with the activities. Some of which are extremely useful for product ownership.

User Experience Activities

I had a hard time finding a commonly accepted set of activities for user experience, or at least I was immediately aware of one.  I finally rested on referring to which seems to have a fairly comprehensive view of the usability field.  I have to admit that I was a little skeptical of using a resource from the US federal government as a resource on good practices on anything, so if you are a UX practitioner, please let me know if there is a better resource for information. references a set of elements of user experience put together by Jesse James Garrett as the primary activities that make up user experience.

Those elements are:
  • Project Management
  • User Research
  • Usability Evaluation
  • Information Architecture (IA)
  • User Interface Design
  • Interaction Design (IxD)
  • Visual Design
  • Content Strategy
  • Accessibility
  • Web Analytics

I should note that these elements are intended primarily for the context of the web, and it is also 15 years old, so I’d love to know if this is widely accepted and it there is a broader, more updated view of user experience.

I can see where some of these elements are always applicable to product ownership and others are applicable in certain contexts, but some input from user experience practitioners would be very helpful to sort things out.

So What?

So why am I trying to categorize, sort and group a bunch of activities (the analytical part of me rearing it’s ugly head)?  As I mentioned above a goal for is to serve people who are practicing Product Ownership. While I can certainly share my experiences, and plan to do so, I realize that I haven’t had the opportunity to experience every kind of software product development situation or every type of IT Project.  I want to provide some suggestions about what activities and techniques are most appropriate for different situations, without regard to whether those techniques are product management, business analysis, or user experience. I thought I’d start the discussion on whether this categorization makes sense and is even worth while, and if it is mine the minds of experts to determine what some appropriate use of those techniques are.

Here’s Where You Can Help

So here’s where you come in.  Let me know in the comments if you think this sort of categorization would be helpful.  Also let me know if you think I have the right collections of activities to begin with.  I’d especially love to hear from product management and user experience practitioners on any different frameworks that exist out there. And if you look at things with a business analysis frame of reference and think that the BCS or PMI view of things outweighs IIBA’s, feel free to chime in as well. (Note: My reference to these organizations and the content of their certification exams should in no way be construed as endorsement of their certification programs.  For insight on my perspective on certifications, see here.)

I also plan to explore this idea during Open Jam at Agile2015, and would love to hear your thoughts about it there.

Ten Powerful Questions for Discovering Acceptance Criteria

A friend of mine is an agile coach for a large software product company.  When she coaches teams, product owners, and managers she stresses that teams need to feel ownership for their work. She likes to teach product owners and other leaders to lead through the use of questions, and has compiled a list of “powerful questions” they can use to lead through questions.

Recently one of the Product Owners she was coaching asked her about powerful questions that a product owner could use when working with their team to discover acceptance criteria.  She asked me what I thought, and  I couldn’t pass up the opportunity to answer a question with more questions.

Ten questions

1. If I deliver this user story what things should I account for?
This is a good question to ask first because it reinforces the use of acceptance criteria to further describe the user story. The way it’s worded also reminds you that backlog items are options, not commitments. 

2. If this user story has already been implemented, what would I try out?
This question is a different way of posing the first question, and is trying to position things from the perspective of what you expect the user story to do.  It is inspired by the innovation game Remember the Future.  A variation on this question is “Pretend that we have already delivered this product; how would you actually test it?”

3. How do we know when we have successfully delivered this user story?
This question reinforces the idea of Acceptance Criteria as a set of statements that tell you whether a user story meets the expectations of your stakeholders.

4. How should we verify that this story is implemented completely and correctly?
This is a variation on question #3 that provides a clearer description of “successfully delivered.”

5. Are there cases with this particular user story that we have not identified how the system should behave?
This question aims to get you to think beyond the obvious conditions and identify obscure conditions that could be very important.

6. What about…?
This question may followup question #5 or may be asked as  your team identifies various scenarios.  In order for this question to truly be effective, you should follow up with “Is that within scope?”

7. Anything that has not been considered so far?
This is another way of asking “what have we not thought about yet?”

8. What happens when…?
This question, along with “what about…” end up getting asked eventually, but usually fairly late in the process.  This question is a good way to spark discussion about behavior the solution should exhibit in specific situations.  While your team can discuss these situations when they arise, if you identify them with acceptance criteria, you run a better chance of having a well thought out response.

9. What scenarios are relevant?
This question is a good way to determine if your acceptance criteria meets what you are trying to accomplish or if it is verifying something that won’t happen or is outside your true goal.

10. Are all scenarios necessary (right now)?
This question is a good way to determine if some acceptance criteria represent aspects of the user story that are not necessary to satisfy immediately.

Two bonus acceptance criteria  techniques

In addition to the ten questions listed above, here are a couple of technique described in other posts that are also helpful for discovering or working with acceptance criteria.

Acceptance criteria and story splitting

A couple of years ago Chris Sims wrote a series of blog posts discussing different story splitting techniques.  One technique he described in the post Splitting User Stories with Acceptance Criteria was how to use questions to identify the parts of new user stories that can be identified from the acceptance criteria of a large story.

  • Ask “Who wants this” to come up with As A
  • Ask “Why do they want that” to come up with So That.
  • The Acceptance criteria itself is the “I want”.

This technique is a good reminder that acceptance criteria can act as more specific descriptions of a user story and can often identify aspects of a user story that aren’t entirely necessary.  Splitting stories via their acceptance criteria is also one of the 21 story splitting patterns I cataloged.

Acceptance criteria and mind maps

I included a technique brief on acceptance criteria in Beyond Requirements: Analysis with an Agile Mindset. That technique brief focused on the use of mind maps to guide the conversation surrounding acceptance criteria for user stories. When you match the use of mind maps with the questions listed above, you’ll greatly increase the chances of having a highly productive conversation. I posted the Acceptance Criteria Technique Brief as an excerpt from the book so that you can take a look at it.

What questions do you use?

Those are the questions I use to discover acceptance criteria. I’d love to hear some of the questions you, as would my friend and the teams she coaches.  Leave your thoughts in the comments to this post.