marcusoft.net

Some common Kanban questions–my suggestions

2012-01-23T08:00:00.000+01:00

I have been talking about Kanban a lot the last couple of years. Sometimes to the point where I think that I must have talked a hole in the head of the those closest to me. But from time to time you get great response, for example;

Anna (an Avega Group colleague) attended one of my Kanban introductions and became inspired. She started to implement parts of the practices I’ve suggested at her client and soon ran into some questions. She sent me an email and asked for my suggestions on how to act in certain situations.

I of course answered but also felt the questions was not only good, they were also common. So I thought that I’ll share my suggestions here. Bear in mind, as I also told Anna, that Kanban is really just a couple of simple rules – how you apply them in your context is very much up to you. This has led to a plethora of practices and ways to solve problems. These are the ones I have picked up in the great network I have surrounded myself with.

So important to note is that Kanban as methodology doesn’t answer any of these questions. It’s up to you.

How do you highlight new activities on the board?

Anna also said; we don’t want them to get mixed up with the ones we already have there. I suspect that she was talking about the work items that might get prioritized over already committed work.

Actually this is one of the benefits of using a flow approach such as Kanban. It’s gives you control and power to say Yes! Yes to change prioritization in the middle of the flow. Yes to add new stuff in your queue of items. I know that David J Andersson consider that as title for his Kanban book; Kanban – how to become a Yes man!

When you have control over a visualized flow it’s easy to see what can be added and what then need to be removed.

One way to handle that is to let the stakeholder have complete control over the Inbox. In my current assignment our project manager is allowed 6 items in our To-do-column. How he prioritizes them is up to him. We simply know that the top one is the first we should work on.

If you feel that, that would make your flow too unpredictable you could allow for a fixed number of items that they cannot change the order on. But what they do before your inbox is completely up to them.

Is it ok to add work items whenever – or only during daily standup?

In my opinion you should at least talk to some other person in the team when adding or changing the order your work items. It doesn’t have to be a formal meeting but could. I’ve experienced anything from big prioritization meetings with several participants (see below) to just letting the stakeholder add notes when the feel like.

One thing I’ve tried that force a communication to take place is that a note must be estimated in size before added to the board. That make the stakeholder talk and describe the item before adding it to the board.

When it comes to moving the board I encourage moving of the items when the state of the work changes. Not just at daily standups. There’s some people, mainly in the Scrum community, advocating moving of the work items only at the daily standup. I was one of them. But then I realized that the board should convey the status of the work – and it should be up to date all the time. So I suggest you move the sticky when the state of the work has changed. That’s also much easier to remember and gives instant feedback of the progress made on the item.

Do you have any suggestion for a simple, powerful agenda on a daily standup?

This one was actually easy. You simply “walk the board”, enumerating work from the right to the left. This will emphasize the pull-principle so that we see work have been pulled into new states, closer to the goal.

Instead of asking all members of the team what their status is we concentrate on the work being done.

If you are in a hurry or have a large team you can simply stop with any deviations from normal. Anything blocked? Anyone not respecting WIP-limits? Work items without attention? Missuse of the board? People sitting on more than one notes? Items having stayed for a long time in the same place?

Stuff that just flow as normal doesn’t need our attention right now. Keeping to just the deviations has led to really big team (40+ people) having daily standups in under 5 minutes.

Keep the meeting short (10 min max) and focus. Put off any discussion to after the daily standup. What I’ve found is that pretty soon will small groups gather after the daily standup and talk about the work. Or better – the process and how to improve it.

What’s a suitable range in size for work items?

It depends, said the consultant and sent his check… This is very hard to give a general answer to, of course.

For small tasks I would say that if it takes longer time to manage the sticky (creating it, adding it to the board and move it) than the work itself – don’t add it to the board. But make a note of work being accomplished. My good friend Joakim Sunden as written an excellent blog post on this matter. I suggest you check it out.

In the other end you have to consider what value you want from the board. A board is should be an information radiator. If you have work items sitting in the same state for several weeks it will not give you any value. I usually have 1-2 week as maximum, estimated time through the board.

Of course sometimes you have waiting stages (such as “Waiting for deployment”) and then you can batch up many items there.

Even though I suggest that you try to slice up big work in to smaller end-to-end slices of functionality and manage each separately, you could also: split them into tasks and move the task. Here I’ve read about people keeping a number series to keep track of when the main item can be moved.

Say you have a big work item entering development.
You give it a number (5 – Allow for credit card payment).
You then add tasks for that item;

5.1. Update database,
5.2 Create GUI
5.3 Create business layer,
5.4 Integration with PayPal.
etc.

As the team start work on the sub items they move them over to Development done.
When the last item has move the whole work item can be moved (#5 that is)

This pattern will help you to keep track of the status of the work even for big items. Also by divide the work item into smaller tasks you have to think hard about what is needed to complete the task. That is a exercise well worth the time in itself. Hence I never allow for task bigger than 8 h (1 day).

How do you handle recurring activities, such as weekly meetings etc?

I usually don’t add these to the board. It’s not stuff that “make our customer happy”. But of course if you want to track the how much time is spent on meetings… then by all means add them.

In one team I’ve been in we had recurring activities (generating customer invoices) that needed to be done by certain dates in the month. So they created stickies that they reused for those activities and kept them in a corner of the board. With the next due date written above them. When that date approached we simply move the item to the top of the Inbox.

We have several stakeholders that supply us with work – how do you make them prioritize among themselves?

This sounds exactly like the situation where the first Kanban system was ever created in software development. It’s described in great detail in David J Anderssons Kanban book. Simplified they had a maintenance team (in India) that got their work from 4 different managers. Of course all of these wanted “their” items to be prioritized over the other work.

The manager of the team solved this by send them an email when the team had room for 3 new items. They then decided among each other what to prioritize now. Pretty soon the mail was automated. And also the stakeholders started to deal among each other (if I vote for your item this now can I count on your support later?).

The reason, I think, that this worked has to visibility and building trust by delivering value often. The stakeholders knew, roughly, how long it would take until the opportunity to add item. So if they missed out this time it wouldn’t be long until the next occasion. The didn’t have to wait so long until the next train, if you like.

It’s of course better to meet in person if you can. But invite all your stakeholders. Describe how your process works and how they can see work flow through the process. Show them how you will deliver faster (maybe even promise a max time?) if you can work with fewer items running at the same time.

Conclusion

I hope you could make sense of this even though the questions were out of context. I would love some comments on other ways to handle these situations.

“If you quit today”–make someone happy

2012-01-01T21:54:00.001+01:00

I have been working as a contractor for the better part of my career. Due to the nature of consultancy and the IT-business in general I’ve seen a lot of colleagues come and go. Brilliant people in many cases, people that you miss so bad when they have quit.
So when the ubiquitous “Thank you for the time here”-mail comes I always try to write them a short sentence saying that I’ve really appreciated their company, competence and just being around them.
And it hits me every time; why didn’t I say anything of this while we were colleagues?
It’s so stupid. It’s so simple. And it takes about 5 minutes to write. I always find the time when they send the notice that they will quite.
So I will try this out – send people a “If you quit today … I would love to have said this first”-note before they go. Yeah, yeah I know it sounds like a New years resolution but I actually decided to do this in November.
I found a great spot for doing it also. On the LinkedIn recommendation page. Then it will be available for others to see as well. It might have a twofold effect for the person in question.
Speaking of twofold this reminds me about a chours we sang in the Salvation Army when I was small (my translation):

Make someone happy
Try to do it today
There’s so many way you can try out
And if you do it you will see
That you’ll get happiness in giving
So try today to make someone happy

I’ll give this a try. ~~Follow my tweets with the tag “#IfYouQuitToday”.~~
After a great suggestion from Christer Unestrand I have now changed the twitter tag to "#HaveIToldYouLately" since that's a more positive take on it. Thanks Chibbe!

What is agile?

2011-12-29T11:08:00.001+01:00

About a year ago I got asked by my project manager to “just give a quick intro and overview; What is agile? 20 minutes or so…” That didn’t feel to bad – but when I got home and started to prepare it struck me; it’s really hard to sum up agile!

For me agile has been become more a less a lifestyle and much of the things I argue for or fight against is about agile or being agile. And also, now that agile is 10 years old, the buzzword has gone into the next phase and almost anything good or cool that people does, related to software is branded with agile ™.

Is TDD agile? If your not doing TDD are you not agile? Standups? Boards? The list goes on and on and I couldn’t come up with an intelligent answer. Either everything had to be included to be agile or nothing was left.

Like the story of the old women who collects wood:

She went out and started to pick up sticks from the ground:
“I’ll manage that one”,
“And if I managed that one I sure manage that one”,
“And if I managed that last one I will manage this one to”
…

After awhile she had way to much to carry. So she started to take sticks away again;
“If I cannot manage that one, I’ll take it away”,
“And if I couldn’t manage that one, I will have to take this one away too”
…

So I dodge that presentation.

But the question stayed with me and itched me like your knitted Christmas stockings from grandma. Later during the autumn (or rather the winter) I got another opportunity and this time I took the chance to sort my thinking on the matter. In this post I’ll share my thinking on the subject. And yes, sorry for the long post – it’s a 45 minutes presentation. But now on paper.

Shu Ha Ri

I read an excellent book called Coaching Agile Teams by Lyssa Adkins and that introduced the martial arts thinking of Shu Ha and Ri. That is different phases that you go through when learning new skills. From the Wikipedia page about it we have a wonderful translation that sums it up, short and sweet:

“first learn, then detach, and finally transcend.”

Here’s a quick intro to the three levels:

On the Shu level you learn fundamentals and techniques. My mind often wander to Karate Kid and the famous wax-on, wax-off scene, when I think of this level.
On the Ha level you have reached a level of understanding of the fundamentals that you can start to challenge the rules a bit. “What if I turn my hand like this?”, “Yes, but in this situation I might try to go this way instead”
The thing here is that you are using your knowledge you picked up on the Shu level and break the rules based upon that knowledge.
Ri level means that you are leaving the rules, improvising and creating new ones as needed. But coming from, or being in the culture of the rules that you learned and challenged on the first two levels.

Wasn’t this about agile now again?

Ok, ok .. I’m getting lost here. But I think that the Shu Ha Ri model says something about how and why many agile adoptions might fail; we break the rules before we know them. Or rather we break rules without really knowing the reason of the rules. WHY is it like that?

We love our tools – and a word on diets

Moreover our industry loves and craves tools, guidelines and best practices. We are still longing for a short and sweet description that says; “do this and your software development project will be a success”. A silver bullet if you like, when I learned 1986 from mr Brooks that no such thing exists.

And really … we know this, don’t we? We’re just hoping… Oh we hope sooo much. Like people still searching for a diet (only bananas, low carb high fat, pills, TV-shop) when we in reality knows; eat less, work out more. Bam – works every time.
But we’re still hoping – maybe this blue pill will do the trick. Or this bicycle thingy that folds and plays my favorite music at the same time – it will make me slimmer. I know it…

And that’s the reason for failure of many other software methodologies and processes; RUP, for example, wasn’t intended to be used the way many people used it. Nothing in RUP saying that you should have year long iterations for example. And it even says explicitly that the documentation templates was just templates, to be modified - it still we use all of them, and all of the headings as well. Just to be sure.

In fact the dreaded Waterfall method is in itself a misconception and erroneously applied model. The man that “coined” the phrase used it as a reference to a flawed model, but still our industry took to it as a fish to water. Why? Because it was something to hold on to.

“The waterfall method you say…? I’ll take it”

“But it’s flawed!! It was meant as a joke…"

“Don’t care! It’s at least something”

Dan North did an excellent keynote on this at OreDev 2011 – we he said;

“We rather be wrong than uncertain”

Scary but true.

Applies to agile processes

So when agile came along, starting with Scrum (at least in Sweden), we all went;

“Yes! That’s right! And with some rules along with it. Lets go”

So everybody started apply Scrum. Or at least tried. Or at least where it didn’t hurt to much. I have actually never seen a Scrum implementation as it was meant to be; it’s always some part missing;

Yeah, we do Scrum, but we don’t do standups/have a board/sit together
It was too hard to find a good Scrum-master so we have our own applied version of it.
It’s Scrum-ish; we don’t meat our customer that often, and we have skipped iterations.

This have to do with not embracing the fundamentals that agile is based upon. Or if you like (and to finally get back to the martial arts mumbo-jumbo about ShuHaRi); we’re breaking the rules of agile without knowing why?

What is agile?

So, with that in mind, answering the question “What is agile” is not that hard:

“An approach to software development based on the values and principles expressed in the Agile Manifesto”

And the agile manifesto talks about, as you already know:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

But what do they mean in the Agile manifesto then?

Well, I will not be so presumptuous as to say that I know what they were thinking. But if we try to think about WHY they have written as they have… Maybe we’ll learn something:

We are uncovering better ways of developing software by doing it and helping others do it

What’s the thought behind this statement?

Firstly “uncovering” means that they (and we) are still learning. This is not the truth once and for all.
Secondly “developing software by doing it” – all the signees of the agile manifesto were developers, to my knowledge (!). They still had their hands in the dough. If that’s good or bad is another question…
Thirdly – “helping others” – they haven’t been sitting alone and dreamt this up. It’s based on experiences from a lot of projects where they together have been involved in.

Individuals and interactions over processes and tools

Why do they write this?

Software is a knowledge creating process – nobody knows the finished product before it’s done. Not even the customer. In order to succeed we need to communicate. A lot and often.

So … therefore it’s more important to focus on the individuals in the project and their communication, than to focus on following a certain process, or use a certain tool. If you communicate well within the group – the process and tool becomes secondary by itself.

What do you do when a project is in danger of being late? You put all the smart people you need to create the product in one room. Leave them alone and let them work. There you have it; Scrum in a nutshell.

What would change if you could talk to your end-customer/stakeholder each day?
What is the number one thing hindering you from doing so? How can you change that?

Working software over comprehensive documentation

Oh man – this one has been abused so much it isn’t fun anymore. It DOESN’T say “No documentation” – it says; “Working software over comprehensive documentation”. Of course documentation is needed.

Why do they write this?

Working software is actually the only real metric that you could use for progress. How much software that works (that the end user can take advantage of and be more productive with for example) are you putting into production each week?

That’s why we are here; to help and delight the customer with working software. Think for yourself how happy your customer would be for a great requirements specification. … Exactly – not that much. Still, that is what is being DONE after several months in some projects.

And why is it important to put it out there often? To get feedback. Feedback is the creator of knowledge. Without it you can do changes but you don’t know if it’s the right change, or how much you should do.

I love a Michael Feathers tweet about this (couldn’t find a link to it now sadly):

“This definition of done this scares me. Code is in production or not!”

When we take software into production we know that it’s there. Ready to be used. And we’re run through our process (analysis, design, development, testing, building, deploying). All of this will give us valuable feedback to improve upon.

This is also a foundation of Lean, to move small bodies of work through the whole production chain often and swiftly.
But, very important - WHY? To learn and improve your process!

That’s why even well working units at Lean plants will continue to put more pressure on themselves (decreasing WIP-limits for example) when they already work well. To further improve.

How can you measure how efficient you are creating working software?
What is the need of documentation from a end-customer perspective?

Customer collaboration over contract negotiation

So what is the reasoning behind this then?

Well as with the first point this honors the fact that nobody actually knows what the finished product will look like. Not even the customer. Not even in “re-writes” (trust me – I’ve been doing my share of those).

But why writing this “contract negotiation” in here? Well that the common way of “protecting” yourself against uncertainty in the software industry. Write a contract or an SLA. And really what – is bad about that? Here’s a story that I’ve took part in:

After a project at one of my client we sat down in one of the rare opportunities we had to meet the customer. They were happy – because we had delivered before the estimated time. But we started to talk about how we worked on “each side” customer vs. IT-department (yes, sadly the same company).

“So how did you do estimates? You were about 20% under budget. How can you guess so badly” – the customers project manager asked

“Well – first we make a guess and then we always add 30% just to be sure” said the IT project manager

That made the customer side burst into laugh in unison. When they had calmed down they said:

“That’s exactly the same thing we do with all your numbers. We add 30%, because otherwise we just get disappointed”

I wish I was making it up but I was in the meeting. So both sides added extra time and resources to the estimates to protect themselves. But here’s the most stupid thing about this – all that time they could have been cranking out “working software” instead. Now we were writing contracts, budgets and estimates.

Really this whole business of having a IT-department that the business side “buys” services from is wasteful in itself. It doesn’t promote a collaborative culture. It’s writing contract instead of collaborating.

What could you stop doing if you sat next to your customers?
What would you change to improve collaboration among different departments in your project?

Responding to change over following a plan

And the final one – which I have found to make the most profound change in my thinking. Why have they written this? This is at the heart of the philosophy behind agile; what kind of person/project/ company are you?

If problem arises with a plan you made – what are your first measures? Some people would make even more plans, stricter budgets and less maneuverability for the next project. Others would say; well that was to strict let’s give ourselves more flexibility for the next time and not plan so much in advance.

I’ve heard of companies who’s CFO’s war cry is; “no exceptions” (everybody goes through the same exact process and is treated in the same way).
And on that same company there were team leaders that said: I only do exceptions all day long – that the way we roll.

So again, there’s is nothing wrong with planning. In fact, I think I have planned more when I started with agile than I did before. But I have completely shifted the horizon. I make detailed plans for the next few days or weeks and very rough ones for anything bigger than that – because I know it will change. That’s the only thing I do know about the things down the project road, in fact. Look up Rolling wave planning for more on this.

And this also comes back to feedback, which I think sums up agile in a word. To align your process in such a way that you get feedback and change your behavior to adapt to the new situation.

If you knew that everything you have decided upon so far could change – how would you set up your project differently?

So – really what is agile?

Agile is a philosophy, a way to approach how you do stuff (system development mostly). There are not steadfast rules on how you should or shouldn’t do things.

So to “become agile” you and your team must apply this thinking and then start adjusting yourself, your process and the people around you in small steps towards that thinking.

I have found simple goal-statements, rephrased into questions, to be to great use to guide me and teams I have coached. For example this statements:

“Will this (meeting, feature, schedule, prioritization) make our customer more happy?”

“If you were forced into delivering new functionality into production every month/week/ day – what would change first to get there?”

“How could we solve the number one hindrance you see to deliver every other week”

Conclusion

This is how I see the question and it’s answer today. I’m sure I will change. Often and a lot. But that’s just part of the whole being agile thing.

I hope you got something out of this. I sure did just by forcing myself to write it down.

Applying the Switch framework to broken builds

2011-12-22T12:54:00.001+01:00

I’m rereading a great book – Switch by the Heath Brothers and it inspires me. The last time I read this book it inspired me to write a post and it’s happening again.

The book is about change, how to accomplish change and especially in situations where you are not in power or control. In short – how to talk to and influence others to understand the need for change and follow through on it.

One very smart thing is that you can download a cheat sheet that contains the ideas in a nutshell – the Switch Framework – for free. Of course it’s not much use if you haven’t read the book, so you better do that first. Could very well be the best book you read.

I thought I would apply the reasoning in the framework to a real life situation and see if I get any ideas on how to get forward. I haven’t implemented this yet but I’ve been reasoning about it with myself and other for quite some time now.

So this is, like many post on this blog, a learning experience for me and I’m sharing it with you. Please supply any feedback to me. I would love to hear your thoughts on the matter.

The Problem – Broken build indifference

To have continuous integration (CI) for your source control nowadays is almost mandatory. But if I think back it was a very novel thing for many developers just a few years back.

CI

In short CI means that you have an automated way to get the latest source code, compile it, run unit test and integration tests (if present) and then deploy it to an environment for testing. You should run this as often you can, at least daily but it’s not uncommon to do a full build on each check-in.

In my experience it’s one of biggest quality improvers you can add to development process or team. It give you a base line for the current quality of the system and a pulse and a sense of going forward, each check-in makes the system just a little better than before. It’s quality guarantor of the source code checked in. You know that it’s good enough to pass the compile and tests in the build script.

An automated build is a guarantor that the build is done in the same way each time. If you get it to work once it will follow the same procedure each time, which gives you much reliable deployments to all environments (to production for example).

Also it give stakeholders around you a way to get hold of the “latest and greatest” whenever they want.

The catch

There’s only one catch… you need to fix broken builds. In fact you should make that the top priority for the team. Always fix broken builds as soon as they happen. But that is of course boring… and irritating … and sometimes even makes you angry as you might not have been the person that was responsible for the check-in that broke the build.

This is the root to a lot of argumentation and inventions; a blame hat, red blinking lights, monitors that shows you when the build broke and who was responsible etc. etc.

In some teams I’ve been in, people after awhile give up. “This build broke… So what – he has broken the build six times in a row”, “Who cares? Broken build, I’ll fix it with my next check-in next Friday”. And if someone in the team starts reasoning like that it doesn’t take long before others tag along. Before long the build process is often side-stepped and stuff is thrown together and into production in an ad-hoc fashion. It all resembles the broken window syndrome, actually

So, back to the topic of the post, how do you CHANGE this behavior? Well – let’s try out the Switch Framework.

The Switch framework – a short introduction

Hey, I cannot start applying the framework without giving a short introduction. In order to understand it you should know this at least;

The authors have a image of the human psyche that they build a lot of their reasoning on; the image of a Rider on an Elephant (borrowed from Jonathan Haidt). The elephant is our emotion and the rider is our rational side. The rider controls the elephant with some reins from his topside position. The only problem is that if the six-ton elephant wants to go into the bushes to have something to eat … those reins will not help much.

In order to accomplish change you will have to address both the rider and the elephant part of people. The rider need clear instructions while the elephant need motivation and some reasons for why this should be done. Finally you need to shape the path on which they are going to be traveling. The Switch framework gives you some guidelines on how to accomplish that.

And there… I have now summed up about 4 pages in the book. Read the rest to get the full picture.

Applying Switch to the Broken build indifference problem

So here is my thinking on what you can do to help a team that’s experiencing the problem I described above.

Direct the rider

The rider probably knows that a solid CI is a good thing, most of the developers do. The problem here though is more that we need to put the attention on the broken builds and fix them, which probably is mostly an elephant-problem (of lazyness). So to the “rider” we need to supply information and arguments on why this is good.

Follow the bright spots.

In the book the authors talk about finding situations where the wanted behavior has been a success. People who already are doing this and succeeds with it.

We might find other teams that can talk to our team about the benefits of fixing a broken build right away. Or find situations when that have helped us find stuff that would have ended up being a problem later on.

Script the critical moves

This is putting emphasis on concrete measures that one can take to make it crystal clear and easy to follow.

In our case that might be a technical solution; for example some tools don’t accept commits or check-ins when the build fails. In Team Foundation Server this is called gated check-ins and simply hinders bad code from entering the system. Mind you that the build we still break… but you will have to fix it before you can continue.

We also need to make sure that everyone understands and can follow the actual build process. With great logging (not too much though) and simplified build scripts that doesn’t tries to do everything. In my experience the build script is often done by one person (and he/she sweats and swears over it). And when it works you “just leave it”.
Maybe we should go through it together and make sure everybody understands what is going on. And even simplify it and the stuff it does. “Does everybody get what happens when we’re running our integration tests”?

Point to the destination

I love this part – it’s like sending a postcard from the future when this have been applied. Look who wonderful stuff is over here. All our problems is just gone. Nice, huh?

Here we can talk about how we can be more responsive and faster in our process. “Due to our rock solid build and deployment process we can now deploy a new version into production, safely, in under 8 minutes. Look ma’. No hands!”

Motivate the elephant

The elephant is all about feeling and getting the emotions on our side. Here are the steps for that.

Find the feeling

This part is about getting a sense of urgency, or feeling that this needs to be fixed. There’s a great example in the book where somebody shows an inventory problem by pouring out all the different gloves that the company is buying on a table. 424 different kinds. That made a feeling of “WOW! How can we go on doing this? This must be fixed!” stick with people seeing the pile of gloves.

For the broken build we could start tracking data on how much time people are spending on untangling bad source code they got when doing a Get. Or how many times outside stakeholders are using a faulty version of our system, or even waiting for us to put another one together.

Make people feel the pain they are inflicting on each other. But here we have to tread carefully and not fall into the blame game, which I think is bad. Jellying and hitting people with treats or humiliations cannot be the answer. But helping them to see that their behavior is hurting others might.

Shrink the change

In this part Switch is asking us to make the first steps easier. Giving us ahead start and a sense of that we’re already are on our way.

In the broken build problem this should not be too hard. We have the process in place. “Some (3/5) builds are working – we just have to try a little harder and we’re almost there”

Grow the people

Here you’re trying to create a team-feel, to get a feeling of belonging and some small peer pressure into the game.

I’ll leave this one with a (tweaked) quote;

“I dream of the day when a broken build is a major thing and a release to production is not”

could be

“On this team a broken build is a major thing and a release to production is not”

Shape the path

Often we think that change is hard because of people, but in reality it’s often the situation and the environment the people are in that is the problem. This part is all about changing the environment to help and support the change.

Tweak the environment

Maybe some simple change in the way we work or the environment we’re acting in can be change to facilitate the change in a big way. A great example of this is the Amazon 1-Click-buy. They simplified their checkout process to be one simple click, which of course resulted in an increase in sales.

I often find that build scripts and the stuff they are doing is very complicated. Maybe we should split the CI build into just doing a compile and running the unit tests. That will leave us in a pretty good state for ensuring that the code quality is good enough to commit into the source repository. We can then have separate build running a couple of times a day, that runs the Integration level test and (if they pass) deploy to a test environment.

This will be much easier to understand and we will get faster builds. The testing environment will still be stable and updated a couple of times a day is still very good.

Build habits

This is about moving your action into the muscle memory, and thereby making them “free”. We do it without thinking about it – reducing the workload of the rider.

A checklist is a very good idea of this. Here you could think of introducing separate steps or checklist on a board that make you go through this. I suggest you read Håkan Forss blog on this matter. It’s a great way to make stuff standard and hence a baseline for quality.

Rally the herd

Here we want to make the behavior contagious so that it’s automatically inflicted on new team members and teams around us. This will help us to grow as well.

I suggest that you make team talk with each other and exchange ideas. Maybe a show-and-tell-session on build automation at the Friday afternoon coffee?

Conclusion

Well – that was a great way of generating ideas and sorting out my tangled mind on the matter.

There are a number of actions that you can take to fight indifference on broken builds. Some are technical and some are just behavioral, just as expected. I haven’t implemented all of these in my current team, but will sure try to start to.

If you read this and think that I have forgotten something or got something wrong – please comment below.

Log4Net, RollingFile named by Date and the staticLogFileName setting

2011-12-13T13:29:00.001+01:00

We have been chasing some strange logging bugs for a while in my current project. We are using log4net to do our logging and it works fine ... until a couple of weeks ago. Some logging didn't occur, in another case we didn't get new files...

The fix is, very simple, but quite surprising and I thought I'll share something on what we did to fix it.

We are using a RollingFileAppender and common strategy for how to handle the log files; we're creating a new file for each new date. In order to achieve this we have set the following configuration:

   1:    <appender name="RollingLogFileAppender" type="log4net.Appender.RollingFileAppender">

   2:       <file value="{a path here}" />

   3:       <appendToFile value="true" />

   4:       <rollingStyle value="Date" />

   5:       <datePattern value="yyyyMMdd" />

   6:       <layout type="log4net.Layout.PatternLayout">

   7:         <conversionPattern value="%date [%thread] %-5level %logger - %message%newline" />

   8:       </layout>

   9:       <filter type="log4net.Filter.LevelRangeFilter">

  10:         <levelMin value="INFO" />

  11:       </filter>

  12:     </appender>

OK – nothing strange here really. And actually it worked fine for a long time. But all of a sudden we have ran into the problems I described above.

The solution is simply to add the “staticLogFileName” element to the configuration above, setting the value to false;

   1:  <staticLogFileName value="false" />

And with that it started to work again. I found some questions around this on StackOverflow that was related but not exactly this.

So if you name your RollingFiles logs with log4net with Dates – be sure to include the staticLogFileName = false setting.

Creating a tools only NuGet package

2011-12-11T21:30:00.001+01:00

I have started to help out in a new OSS project. The last one was SpecFlow and now it’s Pickles. Pickles is a tool that takes .feature-files from SpecFlow or Cucumber and turns it into a nice living documentation. It’s a great addition to SpecFlow.

Ok – one of the things I have been doing in the Pickles project is to create NuGet packages for the project different runners and especially lately the PowerShell runner. This will be the main way to interact with Pickles locally, which basically means that you’ll just write a command in the Package Manager Console to generate the documentation frictionless. Later you will probably run it as part of your Nant or MsBuild build files.

I learned a great deal about PowerShell and NuGet when I put that NuGet package together and since it was quite some time since I blogged I thought I’ll write down my experiences. This post is a about the NuGet package.

The goal of my package is to NOT add any references to any projects. I just want it to drop some assemblies as tools and then use the “Import-Module” PowerShell command to make the commandlet I’ve written available to the Package Manager Console.

NuSpec

NuSpec is a file format in which you specify how your package should be built. It’s actually not necessary in most cases since NuGet can package a Visual Studio project file straight off. But if you want more fine-grained control over how the package is created then you might want to create the NuSpec-file by hand.

The NuSpec-file is just plain XML and here is mine:

   1: <?xml version="1.0"?>

   2: <package xmlns="http://schemas.microsoft.com/packaging/2011/08/nuspec.xsd">

   3:   <metadata>

   4:     <id>Pickles</id>

   5:     <version>$version$</version>

   6:     <title>Pickles - Gherkin Documentation Generator</title>

   7:     <authors>Marcus Hammarberg</authors>

   8:     <owners>Jeffrey Cameron</owners>

   9:     <projectUrl>https://github.com/x97mdr/pickles</projectUrl>

  10:     <requireLicenseAcceptance>false</requireLicenseAcceptance>

  11:     <description>Pickles is an ...</description>

  12:     <summary>A documentation generator ...</summary>

  13:     <copyright>Copyright © Jeffrey Cameron 2010-2011</copyright>

  14:   </metadata>

  15:   <files>

  16:     <file src="bin\Release\*.dll" target="tools" />

  17:     <file src="init.ps1" target="tools\init.ps1" />

  18:   </files>

  19: </package>

Most of the elements speaks clearly for themselves (I’ve shorten the content of some for readability):

At line 5 you can see how you can send in a version number to the NuGet Pack command (see below)

Line 15-18 specifies the files I want to include and where I want them to appear in the deployed package

Noteworthy is that I don’t specify any dependencies here. I don’t want to add any references to the target project. I simply want to copy some files.

Note also that I’m including a powershell file called init.ps1 – see below about that file.

Init.ps1

This is a PowerShell file that runs (if included) when a NuGet package installs. It’s perfect for doing installation stuff. Here is how the file look like for me:

   1: param($installPath, $toolsPath, $package)

2:

   3: Import-Module (Join-Path $toolsPath Pickles.PowerShell.dll)

4:

   5: Write-Host "#############################"

   6: Write-Host "Pickles installed."

   7: Write-Host "Comand usage (switches in brackets are optional):"

   8: Write-Host "Pickle-Features -FeatureDirectory -OutputDirectory [-Language] [-TestResultsFile] [-SystemUnderTestName] [-SystemUnderTestVersion]"

   9: Write-Host "#############################"

On line 1 we have some parameters that NuGet are sending to us. It’s basically some paths to different folder that might be interesting to us

On line 3 is the important stuff; here I do the Import-Module of the PowerShell commandlet I’ve written.
In order to get it to work I have to use another PowerShell command that joins paths to import it from the right folder

The rest is just me writing out some interesting information (?) to inform the user how to use the command

These simple lines will “install” the commandlet into the Package Manger console and it can now be used. With full intellisense for all parameters of the command. Nice!

NuGet.Exe and the Pack command

NuGet also have a command with which you can do a lot of stuff but here I was interested in the the Pack command. It turns a .nuspec or a Visual Studio project file into a .nupkg file that then can be deployed separately.

Here is my command that operate on the .nuspec file I described above:

   1: nuget pack Pickles.nuspec -Version %1

Nothing strange here, I simply point to the .nuspec-file and pass a parameter as version. This allows me to use this command (.cmd-file) from a DOS-prompt to create new packages by simply setting the version number of the package.

So for version 0.3 I got a file called “Pickles.0.3.nupkg”.

Trying out your package locally

A really nice feature of NuGet is that you can very easily set up a local NuGet repository. I’ve blogged about that before and it’s done is just a few minutes.

So to try my package out I simply copied it into my local repository (just a folder on my computer) and then went “Install-Package Pickles” in the Package Manager Console in Visual Studio. I even got intellisense for the package name.

Conclusion and lessons learned

I had some strange problems with the references to dependencies for my package for a while. But when I cleaned my files up in order to write a StackOverflow question about it… the problems went away. Typical! I must have messed up.

But I learned one very important thing when trying your packages out; Increment the version number for each new version! Or start a new project to import your package into. I suspect that the clean up didn’t always work as expected.

All in all this is a very nice way to deploy a tool I think. It have taken me quite some time to put together but 70% of that have been me going back and forth trying to figure out why it didn’t work. Most likely that had to do with me using the same version number all the time.

The Pickles NuGet packages will be available from NuGet.org soon. Very soon.

What i learned from 'From User stories to Acceptance tests' with Gojko Adzic

2011-11-29T20:27:00.001+01:00

I've just returned back home from a course. It's the first course I've attended in over a year and I'm always amazed how much you can learn in a short time; if the course and teacher is good and if you engage yourself in the course.

This time I knew it would be great since Gojko Adzic was the teacher and the subject was something that I find really interesting and useful - Specification by example.
I would be completely impossible to write down even a fraction of all the stuff I picked up during the two days but I thought I stop at the main points and write down what stood out for me.

As I've been following the BDD and Specification by example community intensively for two years I had quite a lot of knowledge on before hand - the main points I wanted to get out of this was practical tips on how to facilitate and handle the early phases of the lifetime of a specification done in this manner. Being a developer I have focused much on the later parts up to now.

Here are the main topics we discussed, but not in this order as I remember:

Communicating with examples
Key process patterns of successful teams
What makes a good specification
Fitting into a development process
Adoption strategies and patterns

Communicating with examples

We started this part by doing a simulation (a Black Jack game) in which we tried out what happens when we try to do "traditional" testing on a short development cycle. And our group went into all the traps that I've been preaching against for so long; developers and testers stopped communicating, we didn't involve the customer, we focused on the implementation rather than on business value. We even got the acceptance test case but returned them as we thought it was a mistake. No group asked the customer (Gojko) what he was expecting.

From that we started to look on how hard it is to understand each other and how the telephone game plays out in a software process. For even such simple stuff as to how many points a simple symbol has there were a lot of disagreements.

We saw how using concrete examples could clarify that a great deal. And when we went back to our simulation and applied examples to the requirements we've got we actually could find a whole lot of inconsistencies in them. Here is was really powerful to find business concepts by using examples such as BlackJack and Busted for example.

Two kind of exercises for creating examples as a group stood out to me:

Diverge and merge - where you intentionally let several groups work on the same example for a while and then merge the groups to compare and learn from each other.
A feedback exercise which reassembled planning poker a bit. Write down a case and then each write the expected outcome. Compare and learn.

The main thing you want here is probably not the examples but rather to learn and get a shared understanding of the problem at hand. The examples is a great way to reach there and also to document them in.

Key process patterns of successful teams

This part I knew the most about during the course, since I've read the book and done presentation on these patterns. I still learned stuff of course:

All the teams that was successful in implementing Specification by example specified collaborative and did that using examples
Key examples is not all examples. It's probably 10-15 not 500
Put up a couple of examples and try to break them (drill a hole in them) to see if all relevant key examples has been found

We then talked about some common collaboration patterns and when they might be applicable:

All-team workshops - when stakeholders are available, you need loads of knowledge transfer (you don't know much about the domain) or you want to explore new ideas. Pretty expensive tough.
Three Amigos - get a Business Analyst, Developer and tester together and do a mini-workshop. Hehe I drew this without knowing a while back. This is useful when you have mature product that your team knows a bit about.
Ad-hoc conversation where you simple skid over and ask the involved people. This requires people being nearby and you knowing a lot about the product.
Write + Review - you write a specification and then have somebody else review it. Good if your are distributed and have a hard time to get hold of people needed to answer questions.

What makes a good specification

I loved this part and learned loads from this. The main part of this was us dissecting and discussing a big load of examples (great idea - examples to understand how to write examples). Together we came up with a big list of stuff that was good or bad with them.

Here are the good part, as we finally summarized them:

A descriptive title (what you would Google for to find this document)
Has context under which the example executes
No technical details such as database ids or web page class names
Short - in fact most of the good stories were considerable shorter than the bad ones
Precis - talks only about the example at hand.
A structure in which the test was separated from the examples (Scenario Outline for Cucumber features for example)
Boundaries included - we triage and try different values to find the boundaries for the example
More than 1 example (see above)
Uses the business language - the domain language
Has the right abstraction level. This is a hard one but you reached this when you cannot remove anything without destroying it. Or another way is to ask someone to summarize the things you have written down. If they can and it's still understandable - use that summary.
Clear and measurable expectations
No new concepts introduced simply due to the fact that we're going to test this with a tool

Other stuff that came up from Gojkos presentation was:

Don't write workflow scripts - write WHAT should be tested not HOW it should be tested. 90% of the team failing with BDD does this. 90%!!!
If you find yourself writing about technical concepts - try to rephrase it into what that technical thing does
Try to find breaking examples to find the boundaries
Show the example to someone else and see if they understand it (given domain knowledge). If so - then you have the right level of self-explanation.
Write the description of the test to be a description on how to read the examples.

Fitting into a development process

This was basically a number of case studies that showed how different teams had fitted the ideas of Specification by example into their process. It was quite interesting as it ranged from fully fledged agile teams to very rigid waterfall-type of processes.

A few tips I picked up:

Set aside time before the planning (be it sprint planning or whatever) to prepare some key examples. That will make the actually planning much smoother not halting on the first question.
The more stuff that is unknown the further ahead the initial team should work. Work a sprint ahead if needed.
Get only bullet points with acceptance criteria from BA if they are very busy
Don't try to write out full specifications at workshops - who writes the specification is not important.
Emphasize collaboration and shared understanding
Define tests as early as possible
Make sure that you get a mindset of collective ownership for the specification

Adoption strategies and patterns

This was something that our group came back to. How do you introduce these concepts? How do you sell it to the team or the stakeholders.

Firstly you can say that this is a general problem that has to do with any change. For that I would recommend the book Switch - how to make change when change is hard that talk about a lot of strategies for change management. A quote that stuck is; "People don't resist change. People resist being changed" - if you get people to think that it's their idea you have gained a lot.

Gojko had three points:

Change the team culture to a culture of collaboration. Collaborate on specifications and test, which will build trust among team members. Focus on delivering business features and not functionality which will build trust with stakeholders
Remove waste from the process - make things precis early to establish a clear definition of done. Validate frequently and strive to get a single source of truth - the examples. This will make other things you do unnecessary (such as writing different documents for specifications and tests for example)
Facilitate change - make sure that the examples becomes the main source of information (send links to them to answer questions), document business process (do not "write tests")

This section also contained a lot of other stuff but it was discussed and is hard to write down here.

Conclusion

This post was my brain dump. I don't expect anyone to get it if you wasn't there. If you still did I'm very happy. If you didn't get it - please ask a question below and I'll answer to the best of my knowledge.

Again - thank you Gojko for a great course. I learned loads from you. Again.

BDD and technical scenarios

2011-11-23T22:34:00.001+01:00

I got a question from Sham (Shamresh Khan) that I thought was interesting and also common. So I thought I post my answer here in the public and maybe we all can learn a bit.

Sham’s question was something like this:

…some of my scenarios/examples, may be very technical (i.e. checking some algorithm for example). If I write these tests under a user story (using the Gherkin syntax), a business analyst will be able to see them which may confuse them as they try to work out what scenarios exist under the user story or am I wrong here? Maybe all tests should be visible under a user story?

TDD and BDD

This is really one of the things we struggle with when coming from using TDD for a while and then starting doing BDD. In my opinion BDD is more than anything a communication tool that helps all parts of the development team to communicate and understand what is to be built – to build the right thing.

By using concrete scenarios and using in plain text we ensure that everybody can collaborate or at least understand and critique them. That is the main point and gain from BDD – communication and a way to quicker find out what we don’t know about the feature in question yet.

That we get a way to test our system (end to end), and in that way can handle regression testing in a very nice way, is just a side effect. A VERY nice one but still just a side effect. Just like in TDD the main benefit not really is the tests but rather the possibility to take baby steps and produce well designed code.

What to do?

So after that long introduction… If you have technical stories or scenarios that your stakeholder cannot understand, I think you have a couple of possibilities:

If the feature in question is a business feature – you should probably discuss it further. If it’s an important business feature it will also be important for the business users. Rephrase it in the language of the business users. Hide the technicalities behind their terms. How it’s actually implemented is NOT important on this level. It’s a TDD-level thing.
If the feature is a truly technical scenario – it shouldn’t be in the BDD-specifications facing the business user at all. It shouldn’t be presented to them and quite frankly – they probably don’t care. (If they do one could wonder if they truly are a business user at all :P)

Actually to combine BDD and TDD is a great way to manage the question you just asked. You first set up the scenarios to get something to discuss around and help you understand and talk about the feature. When that is in place you also have a “definition of done” for the feature. You know how much to implement to fulfill the current requirements. It’s has been referred to as doing outside-in development.

Tips on how to get there

The two groups above could have the headings WHAT and HOW. I suggest that BDD should be about WHAT the system does for the business user – not about HOW it does that. Not about HOW it’s implemented.

A good way to ensure that you don’t miss that target is to work you way backwards;

start with the “So that”-part of the user story – the reason for the user story to exists. If you get that in place everything will be so much easier to understand and reason about.
then for the scenarios start with the “Then”-part – the goal and assertions you are going to make to know that you have fulfilled the goals of the scenarios.

I hope that you got something good out of this Sham. And maybe somebody else too.

OreDev 2011 - day 3

2011-11-11T08:32:00.001+01:00

Last day was great. I didn't go to anything that was really bad and I had some real highlights with Dan North, Gojko Adzic and Mark Rendle on Zen standing out as the best.

After a short and intense Lean Coffee I'm not sitting and waiting for the keynote with Jeff Atwood, who apparently just introduced himself and then went off the stage yesterday. Let's see if he does the keynote himself today

Jeff Atwood on StackOverflow

Jeff did the talk himself (and talked fast, fast, fast!). He talked about how programmers need rules and likes to follow (or even create) rules. Take Facebook for example - who has a "list of their friends" at home. No-one! But a geek could understand social networks by creating rules around them. A list of friend for example.

Huh - I didn't know that Jeff Atwood did invent the game-ification part of StackOverflow by accident. By just being a geek trying to get the most out of a Q&A situation. All of it; numbers, badges etc. was just a programmer analyzing a social situation and modeling it as a game.

Good game design forces you towards the behavior you want from your players. In Counterstrike you cannot win alone. You need to play with others and as a team. And if your anonymous or not doesn't matter.

The same on StackOverflow:

You cannot win without teaching
Individual skill is always inferior to communication skills
Discussion is risky; sharing research and experience.

The ultimate goal is to advance knowledge in the topic for everyone.

Discussions are not promoted because we don't learn from peoples opinions. That's why discussion type questions are downvoted. They will take up place from the questions and answers that StackOverflow wants to be the source of.

Saying No is important to stand out and be pushed into a bland site by the community. "We're playing the StackOverflow game here. If you want to play another game (a.k.a. ask open ended questions) there are other sites". So in a way the community needs to be protected from themselves. With moderation for example.

Tim Huckaby on HTML5 web applications for Windows 8

So I think I will try to pick up some Windows 8 information and skills during this last day of OreDev. So I'll first start with some HTML 5. Tim Huckaby is a guy is heard in a lot of podcasts. Nice to see him live.

Tim started up with stating that we WILL do HTML 5 soon. He started to give a short history of why HTML5 will be big; Apple loves it, Google loves it and Microsoft also loves it.

15 minutes left of this talk and I'm still to see code. I'm learning a lot on the history and fears of HTML5 (only 54% of the internet users today can use it, for example). On with the code Tim!

Nope - no code in Visual Studio here. I'm a bit misled here i think. I learned something but it was more entertainment. Not bad entertainment but not very informative.

Well..... no he didn't have time. It was entertaining but I didn't learn anything of the things I hope for. Sad.

Mattias Karlsson on Java 7

I now popped into the Java track again to see a friend and colleague, Mattias Karlsson, doing a talk on the news in Java 7. I'll brace myself on asking any Java vs .NET questions in here :). Go Mattias!

Mattias is stating that the biggest news of Java 7 is that actually is out. From history we was shown that is big news since Java 6 was released in 2006. So the new release is long awaited.

Hmmm - there's been trouble in Java-land for quite some time as I understand it from Mattias run through history. Politics and business has ruined progress for quite some time.

Leaving that boring stuff out the news are:

new garbage collector called G1 - which is great news for performance tweaking.
Mattias showed some benchmark result for other more general performance. 40% faster than Java 6 - wow! But Mattias thought that it could be closer to 10%
On the concurrency side a new instruction called Fork/Join is introduced. For me it looked like a more low-level implementation of parallel LINQ. That is - PLINQ hides a lot of the things that you need to implement in Java 7. But it must be great to have it if it was missed before. Ha! Mattias recognized that the model still is very low level - I got it before him :). He hinted that a more declarative / implicit model is on its way in Java 8.
Also the JVM that comes with Java 7 has support for dynamic languages. With features that even Java cannot reach! We were shown a way to "invokedynamic" that is a way to via a string call a string.

Other small changes:

You can switch on strings now.
Improved exception - catch multiple exceptions in the same catch-clause. Nice!
Hehe - you can add underscores in numbers. So you can write 1000000000 as 1_000_000_000 if you like.
Some better Resource management, such as external resources such as database, string or files. Nice syntax in which you can state the resources you are using in a block. Looks a bit like using in C#
Java 7 has improved it's generic type inference so that you have to write stuff twice
A new new IO - called NewIO2 :)

Mattias ended with a peek into what will come in Java8. Lamdas - yay! A first-class implementation of JavaScript running on the JVM. Cool!

Jim Benson on Healty projects

After a lunch where I had the time to talk with Andreas Öhlund about NService bus and CQRS. I learned about Amazon that actually has their read data on a CDN! Amazingly cool - distribute read data on the internet. Not even hitting your server! Cool cool cool!

OK - now on to @ourfounder Jim Benson on healthy project. He starts to say that knowledge workers works best when thy are happy.

We got to describe the following to others in the audience:

What do you do at work?
What does your boss do?
What is the goal for the company you work on?

Fewer and fewer persons could answer those questions. It gets harder and harder the further away from us you get. And if you don't know the goal of levels above (or below) you you can't have a healthy project.

We create rules to manage our organizations problem with communication between levels. The problem, however, is that those rules make us more likely to fail in following them. We "mess up" to put it short (sorry Jim, no profanity on this blog).

Jim made an argument that really stuck with me; a Kanban board is not about moving stuff fast - it's about creating quality software. I've fallen into this trap. I want tickets to move fast, but I wouldn't sacrifice quality of course. But I haven't been teaching that - I've been telling people to go fast and that's the purpose of Kanban.

Jim tipped us on something that he called: "Subject well-being". It's simply to write a small smiley on the tickets as you finnish them. Using this technique you is less likely to smooth things over as time has passed from the time you did an awful task to the retrospective. Good tip!

Status meetings was invented by hierarchical organizations to keep them in a tight band. So if you have your status on the board - do you really need a morning meeting to communicate status? No, I say, but maybe to get a team feel and be little better each day.

Conflict avoidance makes conflicts a lot (!) worse. And that doesn't help our project becoming healthy.

What is the definition of success in your project? Is it possible to hit that exact mark? Could it be changed? Don't over define your success criteria.

The indian company TADA hand out award for the best failure last year. The celebrate failures and act on them.

David Evans on Time off for good behavior

David talked about how to extend the useful life of the artifacts we are creating when doing BDD (or ATDD or Specification by example...). The artifacts he talks about is the feature-files, for us Gherkin-geeks.

David state a list of problems or syndromes:

A BDD specification is NOT just a test. AMEN!

Script is not specifications. Very common pattern when writing based on the assumption that a feature is a test.
A BDD artifact is more than just as test; it's a Specification (Now), Test (Soon) and Documentation (Later) all wrapped up in one

Don't skimp on the descriptive parts of your artifacts

You should do BDD artifacts to satisfy humans, not tool.
Tips on User stories

User stories are for Users. A system is not a stakeholder in the story
Don't write the solution - write why you need that
Put the business value first (In order to). Otherwise you risk that the last part of the story is just re-stating the first line
Let the "In order"-clause always talk about improvement (increase, improve). Change something on a scale.
Keep the Why, What and How separated

Tips on Scenarios

Write a first simple positive scenario. Don't only have one scenarios but start with the simple, happy path.
Don't run away and try with every possible edge case. It's important but when we write specifications we only try to find out what we don't know yet
Don't write to much technical details (database-fields for example)
Write your Then-clause first and work your way backwards. Great tip!
Include a keyword from your assertion in the name of the scenario
Make sure that your expectations crisp and the rest will follow to be equally crisp on a as-needed-bases
Look out for Given that are irrelevant to the Thens
Keep the level of abstraction the same
Hour-glass shaped Scenarios; just one line on the WHEN-line!
Bordered edges - examples and counter-examples.

Competing forces of a BDD document

As the document is fulfilling different roles (Specification, Test and Documentation) we have different forces pulling in the document.

Scope

Specification: Specific
Test: Integrated
Documentation: Maintainable

Detail

Specification: Clear and terse
Test: Completeness
Documentation: Self-explanatory

BDD folks seem to be from Great Britain and use Prezi for presentation. David used Prezi to great effect. For example left a final slide that summed up the talk. Thank you, David!

Gojko Adzic on Visualizing Quality

I had a really hard time to pick the last session. But ultimately I went with Gojko on the subject of quality. We started early but calling out examples of quality measures. Hard actually - try for yourself. How do you measure the quality to know when you're ready to ship?

Testers are in the information business! Or rather Inforvate (inform and motivate). Gojko talked about Switch by the Heath brothers. We need to inform the rider of the elephant but also motivate the elephant.

He posed the question; What if you disabled most of your functional tests after implementation? Gojko related a story of a company that disabled their functional tests after development was done. They haven't had major bugs in production for a year. They have decided how to measure quality on business value and visualizing it.

An absence of bugs does not equal a presence of quality. We rather should measure what the people who wants the system, likes and care about. Bug count is not that important.

Try this; get your stakeholders in your room. Write down 3 things you care most about this software. You'll get more results than any developer team can deliver. Most business users don't know what they want - we need to help them.

People who ask for control really wants visibility - Elisabeth Hendrickson. Gojko related another story where they simply showed when the last content (in a content management system) was published. They just showed the waiting time. No fixing the system. The complaints - that had be plentiful - stopped.

Visualize quality

Low tech testing dashboard - show a traffic light kind of status on how you feel about the quality on different areas in the system
Risk/Test heatmaps or charts
Use effect maps
ACC (Attribute component capabilities) Matrix

The main point is - don't measure bug counts, that is not value. That is just measuring failure. Measure whats important for the users of the system. And visualize it - you can it in a concrete and talk-about-state.

Go and give your suggestions - visualisingquality.org

Conclusion

So with that another great stay at OreDev was over. I'm tired but happy as always after these kinds of events. Thank you everybody!

OreDev 2011 - day 2

2011-11-10T17:24:00.000+01:00

As always - after the first day my head was just spinning. I was so tired that I slept before I hit the pillow. But today I'm feeling great again and am ready for another great day of learning.

Dan North on Embracing uncertainty

Dan North is one of my true heroes in our industry. I've learned a lot of the stuff that I'm excited about from stuff he written and spoke about. This is about embracing uncertainty - which seems to sit nice with the whole BDD - Deliberate Discovery body of knowledge. Here's a few things I jotted down:

Patterns of effective delivery (behavior patterns that is)
How we turned the agile manifesto upside down nowadays
The half-time of requirements - how long time before half the requirements needs to be rewritten
"We would rather be wrong than be uncertain!"
Christianity (!) and how faith become religion. Or values becoming rules and structure. And the values disappear in the structure. Jesus was anti-religion of the structures in his time.
The answer don't matter - struggling with question is.
Kent Beck didn't talk about the 12 practices of XP. He focused on values and said: What I want to do is to get business and IT to talk each others. The practices was just ONE way to reach his goal.
Get a handle on uncertainty to be able to manage it. One way to do that is embracing uncertainty:

embracing uncertainty of scope - with rolling wave planning for example
embracing uncertainty of technology - start code by sketching out stuff for example
embracing uncertainty of effort - for example beyond budgeting (measure stuff and handling money as it was your own)
embracing uncertainty of structure - structure only gives you an illusion of control. you can't predict the future

Real options: a model of uncertainty -

Options: making a deal that someone to buy the right to do something in the future. Buy the right to buy a bunch of SEK at a fixed price for example.
Real options - betting against yourself in your everyday decisions.
Create options for every decision
"Never commit early unless you know why"

Deliberate discovery - "Some unexpected bad things will happen" - go and find them. Optimize for finding them. You can actively reduce that ignorance.

Goes without saying - that was great!

Greg Young on When not to use CQRS

Greg is going stone-age on us. No slides, no whiteboard and no sound! :) Nice!
The hall is packed - this promise to be great.

CQRS and Event Sourcing is not suitable for monolithic applications. We need bounded contexts - a huge point of CQRS (and ES) is decoupling. So if you put everything in one big pile anyway - why bother?

CQRS applied in non-core (huge return on investment) applications. Is this an area where our company is where we derive our compatible advantage.

Don't use CQRS for the technical reasons - programmer pornography. It could be considered but should be applied within a single bounded context. It's not done for all bounded contexts.

A common problem is to do CQRS (and ES) and making a lot of decisions upfront. Deciding to use BizTalk before you know what you're actually building.

Using CQRS and becoming use case centric without buy-in from the business. "Only the development team think is that they are core business." Often combined with not being a domain expert. There is a difference in being an expert in how the system currently works and being domain experts.

Nice exercise; sit with the domain expert in just 2 h. Then build the complete system (or at least the domain model). You will fail! But you will raise some important questions. Now go back to the domain expert again and ask those questions. Sit again in 2 h. Delete the code you did the first day. Rinse, lather, repeat.

Don't write a CQRS framework. Hey, don't even USE a CQRS framework. Just do it! You should be able to be up and running with something giving business value in a day. Otherwise you might be focusing on the wrong thing.

Use frameworks for stuff that would take a long time to write it yourself. CQRS could be (and is) written in 200 lines - don't use a framework. MVC would take considerably longer than that. :)

Richard Öberg on Event sourcing explained

Richard will talk about what event sourcing is needed for and what it will give you.

I was probably a bit tired but this didn't grasp me. Richard shows some concise code written in his framework. Ironically Greg Young spent about 40 % of his talk on how you shouldn't write framework.

I liked Richards concept of exposing the events as ATOM feeds. He saves them in JSON format and can then expose them as ATOM feeds for reporting for example. Or send them further to other services.

Storing the events in the first place opens a lot of possibilities; domain events that informs other parts of the system or other systems, reporting, replication, backup and an audit log for free.

By storing the events as the come in gives you a much more fine grained control.

I personally think is that to sell event sourcing to business is a bit scary. But you could probably do ES anyway by simply have a reporting database that is kept in sync with the event source. So that there still is a relation database that most businesses are used to.

Lunch

I had great discussions with colleagues and friends on CQRS and Specification by example. Again struck by the fact that best learning is taking place in the breaks. Hard to sum up in blog posts though. :)

Gojko Adzic on Sleeping with the enemy

Again Gojko is talking about something that really struck a chord with me. So why is testers and developers; us and them? Why does developers "know" more about "developing" and tester more about "testing" than developers?

Independent testing, code freeze, requirements sign-offs and document change management - none of these were done by any of the amazingly successful teams that Gojko interviewed for his book. There were the team with no bugs in production for the last 5 years, doing releases every other weeks, for examples. Or having so few bugs that they don't even use a bug tracking tools - another example.

Disbanded the developers team and the tester team and created a team that is responsible for testing. And developing. Hey - they are responsible for the quality of the product. And they level out the knowledge gap (developing, testing) to better understand and help each other.

In the process they removed a bottleneck (undermanned in the testing department).

We took a quick poll on who should own test automated functional test? The audience argumented for both but Gojko disagreed, arguing that programmers HATE doing automated tests and hence minimizing. Testers should/could spend time their time on designing tests (that testers are good at) and programmers can then automate them (a typical developer skill).

Nice - but I disagree with the incentive for programmers to do it.

Gojko told a story where this kind of co-operation occurred by the developers just sitting next to the tester. Not testing. What happened? They started to suggest what could be automated, they learned about the problems that the testers experienced and where problems and misunderstandings occurred in the system.

Developers and testers need to start working together. Get together as a whole team and ask yourself; what is our number 1 problem and how shall we solve it?

A story on a very productive team that produced excellent quality. But after a while they needed to test a lot of platforms and hired a tester to help them with setting that up. They hired a brilliant tester. But quality took a nose dive. Why? Because all of a sudden they had a tester - who had become responsible for testing. Everybody else stopped to care about test - something they did before!

Testers should be responsible for ensuring that the right things, testing the critical ("hard") stuff and teaching other how to test.

Testers is a common bottleneck in companies today. Why? Because they take on to big a responsibility - move the testing responsibility to everybody.

Mark Rendle on Zen and the art of Software

I moved myself into the excellence-track (hey look at me:)) to see Mark Rendle, the creator of Simple.Data and a general interesting guy. And from the topic of the presentation this promise to be interesting too :).

Mark started with some book tips:

Pragmatic programmer
Design patterns
Code complete
Clean code
Zen & the art of motorcycle maintenance

That last one is a bit surprising and the topic for this talk. It's a book about quality.

Mark is reading from the book and gives comments - novel way for a talk, I must say.

We're talking about what it takes to create quality software; stripping away all the xDD and tools.

Put yourself in the mind of the user. They will use the software. Maybe 8 h a day. So if the software is boring, breaks or hard to use - your are ruining 1/3 of the users life!

Classical understanding (art or code for example) and romantic (art or UI for example). The understanding of the system for the user is actually the UI not the code.

Gumption, a old scottish word; what happens with someone who connects with quality - he is filled with gumption. Drive and enthusiasm. So what is the Gumption traps then? What draws Gumption from us?

External Gumption traps

Getting it wrong - be safe on this by having regression tests for examples.
Intermittent failure - this that is wrong becomes right as you start fixing it. The only way find these kind of failure is to setup a whole lot of profiling.
Parts - third party products in our business. Choose parts from people/companies with Gumption.

Internal Gumption traps

Value rigidity - "my programming language is better than yours". This is a fast moving business - you can't have value rigidity. Expect that you will slow down when you learn new things
Ego - similar to the value rigidity. Approach everything with the attitude as if your were a complete idiot. One day you'll find that you is not an idiot anymore
Anxiety - don't dare do anything of fear of failure. Get control with a Kanban-board or similar
Boredom - flow is the other end of boredom. To get in to flow you can use TDD.
Impatience - give yourself more time. Overestimate. Otherwise you'll end up taking shortcuts

Living right - no job is not "just a job". You do your job to feel good about yourself!

What is quality? How do you know when you delivered quality?

Code is not software by itself. It's software when an user actually starts to use it. Quality cannot be measured until that moment.

The end of the talk was almost a bit emotional... until a phone rang and Mark had too use some profanity to make it stop. So in the same talk we got laughs, tears (well ... almost) and profanity. Pretty amazing for a technical talk :)

Mark Rendle on Phone Apps Unlimited

And then I went out for awhile. My brain was going slower and slower so I need some air. It worked wonders and now I'm back for another, very different, Mark Rendle talk. This time on Phone Applications. Sadly I have to leave early, but hey I'll get some goodness as long as I'm in here.

I talked with Mark before the so I know that the talk will be about moving so much as possible onto a server that your applications can call via HTTP. That means that you can write simple simple applications (views if you want) to call into the server.

65% of the internet traffic in the US is from mobile traffic and climbing. Windows 8 is a not windows anymore, it's mobile platform. With these quotes Mark got our sense and urgency.

All different platforms (Android, IOS, Windows Phone 7) all have different tools and languages. So what could we do?

Well in the cloud we have unlimited resources and can chose our platform and languages. So we try to move as much as possible to the server (in the cloud).

As such Mark has created a Pocket C# - a small IDE in the phone. The actual compiling and all the heavy lifting is done on the server. The writing of code and viewing of the result is done on the phone. Nice!

For example - use DropBox to store files. It's simple and straightforward to build. Using ReactiveOAuth to do authentication.

Use Windows Azure Toolkits for Phones to reach the Azure services such as SQL Azure or Azure storage for example. And some great great scalability "for free" :)

I suggest everybody interested in great ways of building phone applications using your existing skills to find the code from this talk. Some great stuff in here!

Conclusion

And now I'm off meeting an old friend.

OreDev 2011 - day 1

2011-11-09T19:05:00.000+01:00

I was very fortunate to go to OreDev this year. It's just a great show - I described it as a "free bar of knowledge alcohol for learn-o-holics".
I thought I'll take a few moments just to jot down some highlights and stuff I picked up during my first day at OreDev 2011. This is mostly for me... but you can read it if you want.

Jon Skeet on Async in .NET 4.0

I was late to the conference and missed the keynote, that I learned was awesome, and also late to this talk. Jon Skeet on the asynchronous features of .NET 4.0. I picked up some stuff though - most noteable I'll check out the EduAsync blog posts and code that shows of all interesting features.

Aslam Kahn on Functional programming

From this great session I picked up how Alsam had learned functional programming and how that changed how he thought about functional programming; building from simple structures and doing simple transformations on them, to finally abstract concepts on top of that.

Stop think about interactions between objects and start thinking of values of data and transformations on those data instead. Keep it immutable and return something from every function.

That doesn't come out right here but was great.

His slides is here.

Sebastian Lambla on Restful API

This was a great presentation. Loads of fun and a real practical introduction to REST. Thought me to:

Send links back to the client so they now WHERE to go next; Link: ; rel="next"
Send back an empty form so the client knows WHAT to send back
Don't version - but break if data is needed. If not needed use optional values
Send control data in the form so that the client knows HOW to send additional requests

All this will reduce coupling between the client and the server.

Phil Haack on Mobile in ASP.NET 4

Phil struggled with the technology for the presentation but still showed some nice features and ways to make you web application be more mobile friendly.

I'll sure check out JQueryMobile that looks awesome

Christian Johansen on TDD and JavaScript

This was another great session on how to do TDD in JavaScript. Amusing fact was that 100% of the people in the audience did TDD. 0% (!) of us did it in JavaScript.

Christian had some mean skills and the flow of the session was great. I'll check out JsTestDriver and maybe even check out his book.

Felix Geisendörfer on Node.js

And finally I went to learn some more about Node.js. I think it sound very cool and I was blown away; this is huge (#2 watched project on GitHub, 230 contributors).

And the performance was equally impressive due to that everything in Node.js is what's called a "Non-blocking call" or asynchronously if you want.

And then my day came to a full circle with async again.

SpecFlow.Assist.Dynamic–maxing out with Simple.Data

2011-11-02T08:00:00.000+01:00

SpecFlow.Assist.Dynamic came about from my own need when I wrote something that used Simple.Data. I saw the power and simplicity (duh!) in Simple.Data by using dynamics. And I started to thinking on how it could be used in SpecFlow to further simplify and shorten up the code in my steps.
This is the third and final post on SpecFlow.Assist.Dynamic. Again this is not the formal documentation. It can be found here. These are just my personal preferences and thoughts on how to use it. This time with Simple.Data. And it won’t be long.

SpecFlow and Simple.Data

First you should check out Darren Cauthons post on using SpecFlow and simple data together. It shows the great use of a lightweight, simple data access framework in SpecFlow.
As I said in my earlier post test data management is something that you often have to handle in your scenarios. Simple.Data is more than up to the job.

SpecFlow.Assist.Dynamic and Simple.Data

So I thought I’ll take Darrens example from the post and rewrite it using SpecFlow.Assist.Dynamic.
Here is the feature Darren is using, updated to be a bit more readable for a non-technical person (spaces in headers):

   1: Scenario: Submit a valid create account form

   2:  Given the following accounts exist

   3:      | First name | Last name | Email   |

   4:      | Howard     | Roark     | h@r.com |

   5:  When I submit the following Create Account form

   6:      | Field      | Value   |

   7:      | First name | John    |

   8:      | Last name  | Galt    |

   9:      | Email      | j@g.com |

  10:  Then the following accounts should exist

  11:      | First name | Last name | Email   |

  12:      | Howard     | Roark     | h@r.com |

  13:      | John       | Galt      | j@g.com |

SpecFlow.Assist.Dynamic will convert the “First name” header into “FirstName” as you might remember. As will the statically typed SpecFlow.Assist table helpers if I remember correctly.
I first create the Simple.Data dynamic database object in the a method marked with the attribute BeforeScenario. And while I’m there I clear out the Accounts table as well:

   1: private static dynamic DB { get; set; }

2:

   3: [BeforeScenario]

   4: public void Setup()

   5: {

   6:     DB = Database.Open();

   7:     DB.Accounts.DeleteAll();

   8: }

In the Given step I’m using the step argument transformations that comes with SpecFlow.Assist.Dynamic and get a IList<dynamic> from the Table as parameter to the step definition. This can then be sent right on to the Simple.Data database object, like this:
1: [Given(@"the following accounts exist")]

   2: public void x(IList<dynamic> accounts)

   3: {

   4:     DB.Accounts.Insert(accounts);

   5: }

Bom! Table loaded – one line.
In the When-step I have faked up a controller. Here I am using SpecFlow.Assist.Dynamic to get a single instance, again with the use of the step argument transformations; this time it will turn the Table into a dynamic:
1: [When(@"I submit the following Create Account form")]

   2: public void y(dynamic account)

   3: {

   4:     var fakeController = new FakeAccountController(DB);

   5:     fakeController.Create(account);

   6: }

The actual FakeAccountController looks like this:
1: public class FakeAccountController

   2: {

   3:     private readonly dynamic _database;

4:

   5:     public FakeAccountController(dynamic database)

   6:     {

   7:         _database = database;

   8:     }

9:

  10:     public void Create(dynamic account)

  11:     {

  12:         _database.Accounts.Insert(account);

  13:     }

  14: }

Finally the Then-step compares the Table from the scenario to the content of the database. For now I just compare the number of rows, which is super lazy… but it’s ten o’clock and I’m tired…
1: [Then(@"the following accounts should exist")]

   2: public void z(IList<dynamic> expectedAccounts)

   3: {

   4:     IList<dynamic> accountsInDb = DB.Accounts.All().ToList();

5:

   6:     // Assert that they are the same

   7:     expectedAccounts.Count.Should().Equal(accountsInDb.Count);

   8:     // ...

   9: }

Conclusion
So by marrying the great Simple.Data framework with my small efforts in SpecFlow.Assist.Dynamic we can now move clear-text scenario data from and to the database very easily and with just a few lines of code.

I hope you found this helpful and the SpecFlow.Assist.Dynamic could be useful for you.

SpecFlow.Assist.Dynamic–how to use it

2011-11-01T08:00:00.000+01:00

This is the second post about SpecFlow.Assist.Dynamic – a little tool I wrote to help you write less code in your step definitions, and focus on the actual step instead of infrastructure. You can read the first post here – it explains little about what SpecFlow.Assist.Dynamic is.
In this post I’ll show you how some ways I use the dynamic features to and some tricks that you might not know about.
Again – this is not the documentation for SpecFlow.Assist.Dynamic – that can be found here.

Installation

With the power of NuGet it’s super-easy to install SpecFlow.Assist.Dynamic by simply go:

Install-Package SpecFlow.Assist.Dynamic

(Sidenote: Ha! It’s good to try out your own stuff. Found a bug in the 0.2 version on NuGet. Fixed it with the 0.2.1 version )
That will also pull down the latest version of SpecFlow and other dependencies needed.

Test data management

I found out that the most maintainable way to write scenarios is to have them initialize their own test data. Or at least on the feature level. I’ve tried to do it by restoring back ups but that has two drawbacks.
First – it takes too long. And test that takes too long are not run.
Secondly – the features (specifications?) are not easy to follow. You have to know what test data is being read back. It’s hidden for most users.
So why do we still try this approach? Laziness I guess… It’s just to much code to write.

Adding test data – single instances

There are two main features where SpecFlow.Assist.Dynamic helps you; adding test data in the Given-steps and comparing the outcome to the test data in you Then-steps.
For example for this Given step:

   1: Scenario: Searching for books

   2:     Given the following books in the system

   3:         | Title         | Author            | ISBN       |

   4:         | The Goal      | Eliah Goldratt    | 123345678  |

   5:         | Stick         | Ted and Dan Heath | 9955663322 |

   6:         | The Good book | God et. al        | 1          |

I can write the following short statements with SpecFlow.Assist.Dynamic:

   1: [Binding]

   2: public class DemoSteps

   3: {

   4:     [Given(@"the following books in the system")]

   5:     public void GivenTheFollowingBooksInTheSystem(Table table)

   6:     {

   7:         IList<dynamic> books = table.CreateDynamicSet().ToList();

   8:         foreach (var book in books) { DBHelper.AddBook(book); }

   9:     }

  10: }

11:

  12: public class DBHelper

  13: {

  14:     public static void AddBook(dynamic book)

  15:     {

  16:         var insertStatement = string.Format(

  17:             "INSERT INTO BOOKS (Title, Author, ISBN) VALUES ('{0}', '{1}', '{2}')",

  18:             book.Title, book.Author, book.ISBNNumber);

19:

  20:         // Call into the db...

  21:     }

  22: }

There’s a lot to be said about the data access code, for example you probably are using an ORM of sorts and hopefully you can send dynamic objects to it. I’ll show other ways to do that later.

Let’s focus on the SpecFlow.Assist.Dynamic instead. It’s not much to it – I simply create a list of dynamic objects and use my DBHelper to insert each one into the database.
Shorten it up with Step Argument Transformations

You can use a trick that I’ve put into SpecFlow.Assist.Dynamic to shorten the code even further. There is a SpecFlow feature called Step Argument Transformation that I use to shorten up code like the one above. With step argument transformation you can create small conversion methods that is triggered when certain types is used.

I have supplied three such argument transformations that converts Table arguments into:

A dynamic instance
A IEnumerable<dynamic>
A IList<dynamic>

So the example above can be written like this, using these step argument transformations:

   1: [Given(@"the following books in the system")]

   2: public void GivenTheFollowingBooksInTheSystem(IList<dynamic> books)

   3: {

   4:     foreach (var book in books) { DBHelper.AddBook(book); }

   5: }

Pretty slick, huh? This is how it works:

SpecFlow matches my Given statement with the Step definition marked with the Given-attribute above.
From the .feature-file it knows that a table is sent to the step. It send that table.
In the SpecFlow.Assist.Dynamic dll step argument methods are found that matches a Table and the IList<dynamic> that I am using in the method signature
SpecFlow converts my argument using that method and simply returns a IList<dynamic> containing the data from the table.

You can read more about this on my wiki.
Oh yeah – very important! To get this to work you need to add a configuration into your app.config:

   1: <specFlow>

   2:   <stepAssemblies>

   3:     <stepAssembly assembly="SpecFlow.Assist.Dynamic" />

   4:   </stepAssemblies>

   5: </specFlow>

This make sure that the step arguments are picked up from the dll.
Asserting results in the Then-steps

The other place where SpecFlow.Assist.Dynamic shines is when you are asserting that certain result of an action has happened.
If we expand my example from before in to a fully fledged feature, we get this:
1: Scenario: Searching for books

   2:     Given the following books in the system

   3:         | Title         | Author            | ISBN Number |

   4:         | The Goal      | Eliah Goldratt    | 123345678   |

   5:         | Stick         | Ted and Dan Heath | 9955663322  |

   6:         | The Good book | God et. al        | 1           |

   7:     When I search for Titles containing 'Goal'

   8:     Then the results should show the following books

   9:         | Title         | Author            | ISBN Number |

  10:         | The Goal      | Eliah Goldratt    | 123345678   |

I can now write the following steps:

   1: [Binding]

   2: public class DemoSteps

   3: {

   4:     private IList<dynamic> _books;

   5:     private IEnumerable<dynamic> _filteredBooks;

6:

   7:     [Given(@"the following books in the system")]

   8:     public void GivenTheFollowingBooksInTheSystem(IList<dynamic> books)

   9:     {

  10:         foreach (var book in books) { DBHelper.AddBook(book); }

  11:         _books = books;

  12:     }

13:

  14:     [When(@"I search for Titles containing '(.*)'")]

  15:     public void PerformTitleSearch( string value)

  16:     {

  17:         _filteredBooks = from b in _books

  18:                          where b.Title.Contains(value)

  19:                          select b;

  20:     }

21:

  22:     [Then(@"the results should show the following books")]

  23:     public void ResultsContainsBooks(Table table)

  24:     {

  25:         table.CompareToDynamicSet(_filteredBooks.ToList());

  26:     }

  27: }

Don’t worry about the Given and When-steps, it’s just fake. I have save the original list into a private variable called _books and then simply filter that out in the When-step. In a real example the When-step would probably hit a webpage or a controller that in turn reads the database. And the Given-step would put the right data into the database before that.

But the interesting (and short!) stuff is the one liner in the Then-step. Here I can again use the power of SpecFlow.Assist.Dynamic and simply compare the sent-in table to the result of my When-action.

And before you start to think about it. This way of writing it is easier than to use the step argument transformations and compare two IList<dynamic> to each other. That will be a lot more code.

What about single instances?

I have now only showed you the use for Sets. There’s similar methods for creating instances. Typically they are used to fill out forms or find a certain record in the database.
I leave the usage of those methods as a creative exercise for the reader. Let me know if you did anything that helped you.

Conclusion

I have shown you how I use these helpers. Please note that they are inspired to a great deal by the Table helpers in the SpecFlow.Assist namespace that have similar methods but for statically typed objects (that you need to create ^^)

I hope you find this helpful. Please let me know if I can improve.

In the next post I’ll show a final thing that marries this with Simple.Data. See you then.

SpecFlow.Assist.Dynamic–what is it?

2011-10-31T08:00:00.000+01:00

I have created my first ever NuGet package! I am so proud. Actually it’s just a small extension to SpecFlow and the excellent Assist helpers by Darren Cauthon. But I learned quite a lot about dynamics in the process and I thought I write a post or two on SpecFlow.Assist.Dynamic; what it is, the problem it solves and how it’s made. Later post will be around how to use it and to put it together with Simple.Data to create a real sweet testing experience.
Let me say – before I start – that there is documentation for usage on the GitHub wiki. This is a blog posts on how I made SpecFlow.Assist.Dynamic – not the official documentation.

The problem

It’s not a very big problem I’ve solved – but I grew tired of writing small classes that just was for transporting data. For example for the Gherkin-step:

   1: Given the following users exists in the database:

   2:     | Name   | Age | Birth date | Length in meters |

   3:     | Marcus | 39  | 1972-10-09 | 1.96             |

   4:     | Albert | 3   | 2008-01-24 | 1.03             |

   5:     | Gustav | 1   | 2010-03-19 | 0.84             |

   6:     | Arvid  | 1   | 2010-03-19 | 0.85             |

I need to implement the following step-definitions (note that I’m using the SpecFlow.Assists table extension methods here to keep things short):

   1: [Binding]

   2: public class DemoSteps

   3: {

   4:     [Given(@"the following users in the database")]

   5:     public void x(Table table)

   6:     {

   7:         IEnumerable<User> listOfUsers = table.CreateSet<User>();

8:

   9:         // insert in database code goes here

  10:         //foreach (var user in listOfUsers)

  11:         //{

  12:         //    user.Name, user.Age

  13:         //}

  14:     }

  15: }

16:

  17: public class User

  18: {

  19:     public string Name { get; set; }

  20:     public int Age { get; set; }

  21:     public DateTime BirthDate { get; set; }

  22:     public double LengthInMeters { get; set; }

  23: }

So the actual step definition is pretty concise and slick. And actually the only thing that bothers me is that class User. It’s only needed by the step definition and it’s only purpose is to transport some data.
The solution
As I started to use Simple.Data I realized that dynamics could solve this problem for us. So that instead can write:
1: [Binding]

   2: public class DemoSteps

   3: {

   4:     [Given(@"the following users in the database")]

   5:     public void x(Table table)

   6:     {

   7:         IEnumerable<dynamic> listOfUsers = table.CreateDynamicSet();

8:

   9:         // insert in database code goes here

  10:         //foreach (var user in listOfUsers)

  11:         //{

  12:         //    user.Name, user.Age

  13:         //}

  14:     }

  15: }

Just the code we need. No extra data carriers. Of course you’ll sacrifice intellisense, which is a big hurdle for a lot of us. But I got used to it – so can you.
How to get it

The easiest way, by far, is to download it via NuGet.:

Install-Package SpecFlow.Assist.Dynamic

If you want to help out…you are more than welcome. I’m not planning on doing any more work on this right now. But I hope that you see a bug or a improvement possibilities. Let me know via GitHub
How it’s made
The first thing I did was actually to write SpecFlow scenarios for how I wanted the new functionality to work. Even though this was a good thing, as test first is, I would not recommend using SpecFlow to test out unit level stuff. The major benefit with SpecFlow and Cucumber, if you ask me, is that business users can read and understand the specifications.

After that I simply created new extension methods on the SpecFlow Table object:

   1: public static IEnumerable<dynamic> CreateDynamicSet(this Table table)

One for each of the new features I wanted to support, four in all:

CreateDynamicInstance – creates an instance from a table. This can be done either for tables with one row or with two column tables.
In the case of one row tables the header values are used as property names.
In the case of the two column table the first column is presumed to contain the property values and the second the values for those properties (for examples see this)
CreateDynamicSet – creates a IEnumerable<dynamic> for a several rows. The headers are the property names as before.
CompareDynamicInstance (to table) – this compares a instance of a object you send to the method to a table. The table is parsed using the CreateDynamicInstance–method as above.
CompareDynamicSet – and finally you can compare a set of object to a table. The table is parsed using the CreateDynamicSet method as above.

So, I have tried to use the same conventions as the one used in the SpecFlow.Assist helpers;

Spaces can be used in table headers but are removed in property values (“Birth Date” becomes “BirthDate” for example)
I uppercase every new word even though it’s not like that in the table. So “Birth date” in the table header becomes “BirthDate” on the dynamic object

I also added a convention of my own that has to do with types on the dynamic object. I try to parse types from the values in the table in this order:

First try to parse it as a DateTime
Then as a Integer
Then as a Double
And as a fall back I use a string

I wanted to do this to be able to do some more safe comparisons etc.
Challenges and solutions in the implementation
Creating object properties from header strings
One of the first tricks I learned about is the ExpandoObject. By reading up on this I learned a lot about how dynamics actually works.

First – the keyword “dynamic” only means “don’t do compile-time checking for this variable, it will work in runtime”. Nothing more.

Secondly - the base class for a lot of the cool the stuff in System.Dynamics – DynamicObject – is simply a wrapper that gives you the possibility to trap exceptions for when you are accessing members not present on the current object. So if I have written:

   1: dynamic a = new MyDynamicObject();

   2: a.Name = "Marcus";

I’m simply telling the compiler to let me write anything on the “a” object reference. So it swallows the ‘a.Name = “Marcus”’ statement without complaining.

In order to handle that I can write this simple class, inheriting from DynamicObject:

   1: public class MyDynamicObject : DynamicObject

   2: {

   3:     public override bool TryGetMember(GetMemberBinder binder, out object result)

   4:     {

   5:         return base.TryGetMember(binder, out result);

   6:     }

   7: }

In the TryGetMember override I can handle access to any member and do something more useful than to crash with a “Member not defined”.

And that’s exactly what ExpandoObject does. It simply stores all the members, and their value, that you access into an internal IDictionary<string, object>. In fact you can even cast an ExpandoObject into a Dictionary<string, object> and insert your members that way.

And that’s how I insert new dynamic properties and their values from each table row. Like this:

   1: private static ExpandoObject CreateDynamicInstance(TableRow tablerow)

   2: {

   3:     dynamic expando = new ExpandoObject();

   4:     var dicExpando = expando as IDictionary<string, object>;

5:

   6:     foreach (var header in tablerow.Keys)

   7:     {

   8:         var propName = CreatePropertyName(header);

   9:         var propValue = CreateTypedValue(tablerow[header]);

  10:         dicExpando.Add(propName, propValue);

  11:     }

12:

  13:     return expando;

  14: }

Comparing dynamic objects
One problem that almost stumped me was how to compare dynamic instances. There is no way to check which properties a dynamic instance has. … In the .NET framework. … That I know of.

But I found the Impromptu Interface project (and NuGet package). It does a lot of marvelous and strange things but I used it to GetMemberNames. That’s a static method that take an object and returns a list of it’s members.

And there’s also a method that invoke methods on any object by name – InvokeSet(object, memberName).

Combining these two made it a lot easier to write the comparisons methods for dynamic instances and sets.
Reflection
But it was also around this place (having to take a dependency on ImpromtuInterface) that I realized that my solution might be the wrong one. I’m wondering if I might have been closer to a pure “dynamic by the book” solution if I had done a class inheriting from DynamicObject called DynamicSpecFlowTable. In that class I could easier and cleaner have handle how properties are set and comparisions done…

Well it works fine now. I’ll keep it like this.
The future

Actually I would not like to have this NuGet package. I would rather see this included in the SpecFlow main source. But as for now that means that SpecFlow needs to take a dependency on ImpromtuInterface and that’s probably not what they want.

So this has been my training ground for a small adventure in dynamic-land. I’ve enjoyed it. I hope you do too.

Simple.Data–the testing story

2011-10-14T21:51:00.001+02:00

I have fallen for Simple.Data. Big time. It’s so terse and concise that you almost lose the need for data access abstractions altogether. It’s just … there for you. Just see for yourself here. And when have this power at your fingertips it’s easy to forget the testing story.
How should I use Simple.Data so that I still can write unit and acceptance tests with code using it?
In my unit-tests I do not want to call the database, of course, because that would slow my unit-level tests down. Also if I do automated acceptance I want to go end-to-end, testing the full stack of the application. But there the database access will slow me down, often in the form of a network hop as well.
Have no fear! Mr Rendle have set us up with a couple of ways to mock out the actual call to the database on the lowest level, but still go through Simple.Data.
In this blog post I want to show you a couple of ways to write unit-tests that doesn’t hit the database, but still use Simple.Data.

Using the XmlMockAdapter

When you install Simple.Data via NuGet you might be surprised to find a lot of packages there. There one for each adapter (supporting 7 different databases and counting), a core package, a sample and our friend Simple.Data.Mocking. This package installs, according to the description; XML-based mocking for the Simple.Data data access library.
I create a Class Library and NuGet’ed the following packages:

Install-Package NUnit
Install-Package ShouldFluent (excellent assertion library)
Install-Package Simple.Data.Mocking

I can now write some tests. Lets try to simply call a mocked database, in a known state and return some rows for us:

   1: [TestFixture]

   2: public class CallingSimpleDataDirectlyTests

   3: {

   4:     private XmlMockAdapter _mockAdapter;

5:

   6:     [TestFixtureSetUp]

   7:     public void MyTestInitialize()

   8:     {

   9:         _mockAdapter = new XmlMockAdapter(

  10:                 @"<Root>

  11:                     <Users _keys=""Id"" Id=""System.Int32"" Key=""System.Guid"">

  12:                         <User Id=""1"" Email=""marcus@marcusoft.net"" Password=""secretPassword""/>

  13:                         <User Id=""2"" Email=""kalle@marcusoft.net"" Password=""qwerty""/>

  14:                         <User Id=""3"" Email=""john@marcusoft.net"" Password=""1q2s3e4r""/>

  15:                     </Users>

  16:                   </Root>");

17:

  18:         MockHelper.UseMockAdapter(_mockAdapter);

  19:     }

20:

  21:     [Test]

  22:     public void FindByEmail_ShouldFindRecord()

  23:     {

  24:         // Arrange

  25:         var db = Database.Open();

26:

  27:         // Act

  28:         var user = db.Users.FindByEmail("marcus@marcusoft.net");

29:

  30:         // Assert

  31:         Assert.AreEqual(1, user.Id);

  32:         Assert.AreEqual("marcus@marcusoft.net", user.Email);

  33:         Assert.AreEqual("secretPassword", user.Password);

  34:     }

  35: }

So here (and yes, I’ve stolen the sample from the Simple.Data repository’s test project) we create a XmlMockAdapter in the Setup. On row 9 through 16 we load it up with some XML data. This can of course be read from a file if you want to, but then your pushing the envelope on what a unit-test is, in my opinion.

Line 18 is important – that tells Simple.Data to use our _mockAdapter. So that the next time we open a database the Xml mock adapter will be used. In a later version of the code, I’ve peeked, you can write something like this:

   1: Database.UseMockAdapter(_mockAdapter);

which in my opinion is much clearer. However it works in the same way.

On line 25 our test code begins by opening the database for us, which will use the XmlMockAdapter, remember. The rest of the test is simple data access code using Simple.Data to pull back a record and validating against it. Note that we’re using “Users” that is the name of the Xml-element Users as our table name.

Here are some other tests using the same setup method above, and performing insert, updates and deletes:

   1: [Test]

   2: public void Delete_should_delete_a_user()

   3: {

   4:     // Act

   5:     int numberDeleted = Database.Default.Users.Delete(Id: 2);

6:

   7:     // Assert

   8:     numberDeleted.Should().Equal(1);

   9: }

10:

  11: [Test]

  12: public void Update_should_update_the_email()

  13: {

  14:     // Act

  15:     const string email = "marcus@marcusoft.net";

  16:     int updated = Database.Default.Users.UpdateById(Id: 3, Email: email);

17:

  18:     // Assert

  19:     updated.Should().Equal(1);

  20:     string updatedEmail = Database.Default.Users.FindById(3).Email;

  21:     updatedEmail.Should().Equal(email);

  22: }

23:

  24: [Test]

  25: public void Insert_should_insert_new_entry()

  26: {

  27:     // Arrange

  28:     var db = Database.Default;

  29:     var numberOfUsersBefore = db.Users.All().Count;

30:

  31:     dynamic newUser = new ExpandoObject();

  32:     newUser.Id = 40;

  33:     newUser.Email = "newemail@marcusoft.net";

  34:     newUser.Password = "newpassword";

35:

  36:     // Act

  37:     db.Users.Insert(newUser);

38:

  39:     // Assert

  40:     var numberOfUsersAfter = db.Users.All().Count;

  41: }

I used the Database.Default property which simply gives us the default database in the .config, or in our case just the database with the mock-adapter. It’s just a shorthand for:

   1: var db = Database.Open();

   2: db.Users.WhatEver;

Using a repository
In start of this post I said that the Simple.Data helps to write less code. Maybe so that we don’t need to use a separate repository. But maybe you want to use one anyway. To be able to mock out the repository when you’re testing a class using the repository for example.

Simple.Data actually helps us to test the repository as well, using the XmlMockAdapter. Here is a short example.

First my, very simple, repository:

   1: public interface IUserRepository

   2: {

   3:     IList<object> GetUsersNamedBob();

   4: }

5:

   6: public class UserRepository : IUserRepository

   7: {

   8:     private readonly dynamic _database;

9:

  10:     public UserRepository()

  11:     {

  12:         _database = Database.Open();

  13:     }

14:

  15:     public IList<object> GetUsersNamedBob()

  16:     {

  17:         return _database.Users.FindAllByName("Bob").ToList();

  18:     }

  19: }

Worth noting here might be that I am not sending anything in to this repository as a dependency, typically you don’t do that to a repository. I am simply using the Database.Open()-method to create a private field, which I’m then using in my methods.

In my test code I will then use the MockHelper.UseMockAdapter to ensure that the mock adapter will be used when I call the Database-object. Here is my test code:

   1: [TestFixture]

   2: public class RepositoryTests

   3: {

   4:     [TestFixtureSetUp]

   5:     public void SetUp()

   6:     {

   7:         //Create a mock adapter with some suitable XML

   8:         var mockAdapter = new XmlMockAdapter(

   9:                 @"<Root>

  10:                     <Users _keys=""Id"" Id=""System.Int32"" Key=""System.Guid"">

  11:                         <User Id=""1"" Email=""foo"" Name=""Bob"" />

  12:                         <User Id=""2"" Email=""bar"" Name=""Bobelina""/>

  13:                         <User Id=""3"" Email=""baz"" Name=""Bob""/>

  14:                         <User Id=""4"" Email=""baz"" Name=""Bobelina""/>

  15:                      </Users>

  16:                    </Root>");

17:

  18:         // Set up the database to use our mock adapter

  19:         MockHelper.UseMockAdapter(mockAdapter);

  20:     }

21:

  22:     [Test]

  23:     public void using_mock_adapter_in_a_repository()

  24:     {

  25:         // Arrange

  26:         var repo = new UserRepository();

27:

  28:         // Act

  29:         var result = repo.GetUsersNamedBob();

30:

  31:         // Assert

  32:         Assert.AreEqual(2, result.Count);

  33:     }

  34: }

As you can see this gives me the ability to test out the code in the repository.
The future of mocking in Simple.Data

In the GitHub repository there is a branch (dict-query) that contains something really interesting; an in-memory adapter for Simple.Data.

The things I’ve show so far is nice but they require you to write (boring) Xml code. It might not be too bad but if we have an in-memory adapter you can simply insert the data you need using the standard Insert-methods of Simple.Data.

A test would then look something like this (peeking the source again):

   1: [Test]

   2: public void InsertAndFindWithTwoColumnsShouldWork()

   3: {

   4:     // Arrange

   5:     MockHelper.UseMockAdapter(new InMemoryAdapter());

   6:     var db = Database.Open();

7:

   8:     // insert some users

   9:     db.Users.Insert(Id: 1, Name: "Albert");

  10:     db.Users.Insert(Id: 2, Name: "Marcus");

  11:     db.Users.Insert(Id: 3, Name: "Gustav");

  12:     db.Users.Insert(Id: 4, Name: "Arvid");

13:

  14:     // Act

  15:     var numberOfRecords = db.Users.All().Count;

  16:     Assert.AreEqual(4, numberOfRecords);

17:

  18:     var albert = db.Users.FindByIdAndName(1, "Albert");

  19:     Assert.IsNotNull(albert);

  20:     Assert.AreEqual(1, albert.Id);

  21:     Assert.AreEqual("Albert", albert.Name);

  22: }

From the name of the branch (dict-query) you can deduce how that’s implemented. It’s a Dictionary holding the tables and it’s rows. I actually took a stab of implementing it myself but when I saw that Mark had it in place already I thought I help him writing some documentation instead . This blog post is a start.
Conclusion
I am very impressed that Mark Rendle has supplied a mocking possibility with Simple.Data. It gives us a way to mock out the absolutely lowest level of database interaction, giving us the possibilities to test the full stack of the application without having to pay the network or database interaction toll.

I hope you found this helpful.

Is Scrum a –ism that doesn’t work for real?

2011-10-07T10:42:00.001+02:00

The other day I got my hands on the Scrum Guide 2011. It’s a updated version of how Ken Schwaber and Jeff Sutherland looks at Scrum, it’s practices and state today. It’s well worth a read – especially if you haven’t read up on Scrum in a while.

And let me here in the outset of this post also state that I have use Scrum a lot and it helped me and my teams a lot as well. I like Scrum to be short – but (had to be one right) I think that some situations doesn’t fit perfectly with Scrum.

After that short side note, let’s get back to the real thing. After I read the article I had an opportunity to sit down with Morgan Ahlström, a fellow lean / agile coach here at Avega Group. We started to discuss about the things we read in the document and finally arrived on the question; can Scrum, as described in the document, actually be done?

The rest of this blog post are inspired by Morgan and written by me. Any strange stuff and faults are just my own and any great stuff is probably Morgan.

It’s Scrum ™ or it’s not

The conclusion section of the Scrum Guide 2011 states (my highlighting):

“Scrum is free and offered in this guide. Scrum’s roles, artifacts, events, and rules are immutable and although implementing only parts of Scrum is possible, the result is not Scrum. Scrum exists
only in its entirety and functions well as a container for other techniques, methodologies, and practices.”

So if you call it Scrum ™ – it has to be done exactly as described in the Scrum Guide 2011, or at least with all the practices described therein, additions are welcome. If you are not – then it’s not Scrum. Simple and to the point.

The guide starts with stating that Scrum is lightweight, easy to understand but extremely difficult to master.

When Morgan and I discussed this we asked ourselves what was the best Scrum we’ve seen was like? Between us we have about 14-15 years of Scrum experience. Sadly the best Scrum at least I’ve seen didn’t do all of the prescribed components of the Scrum framework, and the same was for Morgan.

Common Scrum Pitfalls

We went back and forth and started to list the most common deviations from true Scrum ™ we’ve seen. Not that this only based of our experience and I would be happy to be corrected on this.

This is our short list:

The product owner is an unicorn
Deliver business value each iteration is hard
Not doing retros and not following up on actions

“The product owner is an unicorn”

This heading is actually a quote from Torbjörn “DrunkCod” Gyllebring said at LeanKaffeSump gathering in Stockholm. The reasoning behind it is one that I supports to 100% – a product owner (as described in the document) doesn’t exists. And for that matter – do we really want one?

The product owner should be “The Product Owner is one person, not a committee” that in order to succeed “… the entire organization must respect his or her decisions”. Among the product owner responsibilities are:

Clearly expressing Product Backlog items;
Ordering the items in the Product Backlog to best achieve goals and missions;
Ensuring the value of the work the Development Team performs;
Ensuring that the Product Backlog is visible, transparent, and clear to all, and shows what the Scrum Team will work on next;
Ensuring the Development Team understands items in the Product Backlog to the level needed

Man – that’s one steep order indeed. So she is to understand the business well enough to be the Development Team’s (new term in Scrum 2011, to my knowledge) single source of truth in questions about the product. And then to be able to prioritize the backlog according to business value on such a fine grained level that the items fit in a Sprint or less.

I have never met a Product Owner ™ to that definition. I have meet proxies in front of a group of people that together constitute those skills – but then you loose the speedy communication needed in short sprints and an agile projects.

I’ve also met PO that hold a lot of these properties but doesn’t care about the low level stuff that the team cares about.

I have never been in a project where the PO was close to the team physically. One floor above is the closest I ever got, but then it was an IT-person that proxied some business users.

I cannot help but making yet another connection here. I am very much into Specification by example and one of the early questions or remarks that Gojko Adzic makes is that many teams expect their customer/product owner to create the backlog for them. That is bad since you in essence ask the customer to design the system for you. It’s much better to harvest the whole teams collective knowledge and write, prioritize and create the backlog items together. The customer can then input and guard that the business values not get lost but leave the discussions of solutions to better suited people in the team.

Not doing retros and not following up on actions

The second most common pitfall or inability to live up to true Scrum is to do retrospectives. This is probably the most important meeting but is so often down-prioritized and skipped. I don’t know why it’s skipped actually, but I think that teams fail to see the possibilities to change and improve their process.

And even if the team are doing retrospectives the actions that comes out of them are seldom followed up or implemented. Common causes for this, that I’ve seen, are the brick wall of the organization in which the Scrum team are acting.
This is not Scrum ™ fault per se, but Scrum doesn’t give much advice and help on how to be better in doing this either.

Deliver business value each iteration is hard

Another thing that is very common is that the deliverables is not of business value each sprint.

You have to do upgrade a database
a new version of the ORM-mapper has been released and need to be integrated in our code base
in order to squeeze the user story into a single sprint we mock out the external dependencies (and do it “for real” in the next sprint)

These are all example of things that is not of business value but still needs to be done. And after a while the requirement of the stories being of business values easily fades a way, leaving the product owner behind. Maybe even the team start to decide that “a new database” will give us business value.

This is not at all what Scrum intended but I’ve seen it happen in many of my teams. Again and again. Probably reflect my coaching skills – or is it just to hard ro do?

Scrum as a –ism

Putting all of these together and I must ask if Scrum has become an “-ism”?
To clarify my thinking I will take you back to a project I was on about 5 years back. On that particular project we had a Swedish girl that was very left wing indeed and a guy from Cuba. The Swedish girl was convinced that communism could work and you can guess what the Cuban guy thought about that… having fled the country as just one example why he hated communism.

That discussion had me thinking – the basic ideas with communism is not bad, right? It just cannot be implemented right. Or to put it another way; show me the best implementation of communism in the world – and I’ll show you a dysfunctional country, a ruthless leader or other horrible things. It’s an idea that’s good but it doesn’t work. In reality. On humans.

Ok – quickly out of the politics (I hate politics) and back to Scrum. Is Scrum an –ism? A great idea that simply cannot be implemented (correctly, Scrum ™ style). In reality. On humans.

And does that matter?

Is ScrumBut bad?

Scrum-but is a term that is used to describe Scrum implementation that does some of Scrum BUT not all.

“Yes, we do Scrum … but we don’t have PO”
“Yes, we do Scrum … but we don’t do retrospectives”
“Yes, we do Scrum … but we don’t deliver business value each sprint”

The term is, of course, frowned upon by the Scrum community. But is it really bad?

Take 100 projects. Say that 80 of them failed. Some are probably not using Scrum, some are probably using Scrum-but (parts of Scrum). And some are probably using Scrum ™. Yes – good people, you can fail using Scrum.

And the 20 that succeeds. I bet you that not all of them are using Scrum. I’ll bet that some of them are using Scrum-but. And some are probably using Scrum ™. So where does that leave us? It’s seems like it doesn’t matter if you use Scrum ™ or not. It seems like you can fail and succeed either way.

Yes – but the important thing is what you do with that failure. I have from time to time called Scrum and other agile methods “problem finders”. They are used to find problems in your organization. But from there you have to intervene and DO SOMETHING ABOUT THE PROBLEM. You (the team) are the only one that can act and improve.

That’s the important thing and the way I sum up agile; get fast feedback and act upon it. Scrum ™, Scrum-but, Non-Scrum or Kanban doesn’t matter – what you do with the feedback does.

Conclusion

So if you cannot do proper Scrum ™ as described in the Scrum Guide 2011 – why is it important? It isn’t! Not to us that is – it probably is if you have invented Scrum and are making a living of it.

Scrum ™ can also be used as a nirvana state. A goal we’re striving for but never reach. And on the way Scrum-but can be used.

That’s my two cents on the subject. Hope you got something out of this discussion.

Kanban-inzg the Avega Group Office–reflections

2011-09-26T22:05:00.001+02:00

This is the third post in a short series on how Morgan and me went about to introduce Kanban and some Lean thinking to two support teams of the Avega Group office. You can read post 1 and post 2 before this to get the complete picture.

With this post I wanted to do some reflections on how the introduction of Kanban took hold for the teams and also what Morgan and me learned in the process. We are used to do this for IT-projects and teams working with system development or maintenance.

Reflection on the result for the teams

First week

The immediate and positive responses from the group was reactions like:

“This is very clear and shows us what we are doing”
“I can see dependencies between our tasks. And on the ones where we wait for others”
“We have greater understanding for what we do. And other can see that (what we do) also”
“I think our effectiveness went up 25 %”

This is all related to visualization and the simple act of putting stuff on the wall (or the board) if you ask me. It’s also very common for teams starting with agile practices to get this kind of boost just by showing what they are doing.

There was also also a buzz around these new different rituals ( morning meeting, moving cards, creating avatars) that made the first week a true revelation for the teams.

One of the more interesting signs of that was that they spontaneously started to evolve the board:

Both teams used different colored stickies to further communicate information.
- One team used a color for each big project they were in. The question was what that helped them with. It was easy to see if one project was taking over the board but did it help them to prioritize? Not so much probably
- The other team used a more common Classes of Service concept grouping their tasks in Urgent, Normal and Low priorities
One team considered adding a new "To Do”-column for stuff that goes on during the year. So they could prioritize the week from that backlog.

Also it was great to see them starting to collect statistics for what had been completed each day. One team left that on the board to “feel good about themselves” and then put it into a binder for later processing. We have suggested a simple SPC diagram for starters to track progress and deviations.

Second week

The second week the focus has not been great. We tried to urge both teams to do a retrospective on how to improve their process and work but that was down-prioritized. To which Morgan said (love this): “If you gained 25% effectiveness – why not re-invest 5% of that time in improving your further work”

As it seems the week has been very busy and people have been in and out of the office. This has led to skipping of morning meetings and not updating the board, tasks not on the board.

Me and Morgan have to take some of this responsibility on us. We gave them very few instructions and tools and just told them to get going. And they did. But without guidance and continuous coaching to get them further the first inspiration seems to fade quickly.

I think this is also very common – a lot of teams “pick up agile” and just get starting but soon run into problems or don’t know how to go further. That’s the whole idea of a coach. Using the Shu-Ha-Ri thinking they early on need constant guidance and firm advice to follow the rules strictly. So we left them on their own a bit to much maybe. We have promised to ask them if we may coach them on a more daily basis.

Coaches reflection

So that’s how our teams have coped. But what (if anything) did Morgan and I get out of this?

First and foremost we had to up our game. We are used to talk to teams in the IT-business. Many of the have heard of agile or even Scrum and Kanban. We know the common problems and pitfalls – so we are ahead before we start.

But here … we had to go in with a humbleness and use a different language. And, very important, we had to understand what the problem really was. This is a very common mistake that I fall into from time to time – just coming with solution to problems that don’t exist.

So we couldn’t use buzzwords that everyone would smile and nod to. We couldn’t use explanations like Littles Law and “WIP is bad" but rather had to argument using other words. Really hard, but also very good for our own understanding.

We cannot emphasize the use of simple exercises enough (see the first post). That made a lot of difference and gave us a common platform to refer back to. We also introduced the exercises in terms of their business (Imagine 3 projects, one is this important one, the other one …).

We tried that particular exercise on an other colleague that just stop in her tracks and said: “Point taken!” She just GOT IT. Those are the exercises you want to use.

With great powers comes great responsibilities

Something that Morgan and I have discussed is that we left our teams hanging a bit. We gave them tools and told them that “this is a simple process that you can apply to whatever way your are working today”. But without proper coaching and a helpful hand it quickly degenerates.

I think that this is what has happened to Scrum in many cases. People gets ga-ga over the power of Scrum and starts to implement it with the knowledge at hand. That leads to that the core values of Scrum and agile easy get misused or misunderstood. And then you end up in implementations like:

Yes – we are doing Scrum here. Well we have no board, or Scrum master or product owner. But we have daily meetings and don’t much documentation.

Kanban – of course. We have been running it a year! Well we don’t have any WIP-limits and not all tasks are on the board…

Implementing a process change will be hard. Problems are bound to occur and those will help you to improve your process. But inexperienced teams will need guidance (firm at the outset) through that process.

To just inspire them and not follow through with proper coaching was not very nice or professional of us – we will do something about this. Soon.

Conclusion

This has been an interesting journey to follow so far. It’s just beginning of course but already we have seen progress and decay (our bad to some extent). I’ve learned a lot by trying to teach some simple practices in a domain I knew little about.

To do this together with Morgan has been great. It really good to have a speaking partner as you coach a team. Try it for yourselves.

And please – do try to implement Kanban in non-IT related business. You will learn a lot from it. But don’t leave them alone – they will need your help.

Executable specification - the whole stack or not

2011-09-26T14:45:00.001+02:00

This is really an age-old BDD question that pops up from time to time. Just now I got a question from an ex-colleague about it. A bit rephrased it something like this:

Should I test through the GUI all the way down to the database? How do you handle test-data in and test executing speed in those cases?

Many of us first got exposed to BDD tests through web applications and the way that we could test through the actual web page. We use tools like Watin, Selenium and other automation tools to accomplish that. We structure our automation code with page wrappers to get manageable automation code that can be re-used in our step definitions. For some BDD even implies automation (which is not at all the case).

On the "other" side we try to go through the whole stack and go through all the of the application and hit the database as it's supposed to be used by the application.
And that's when it strikes us - this thing with test-data is really hard. And a bit slow also.

Through the GUI

Up to this summer I was all for the whole-stack approach since if you don’t let your acceptance test run through the whole stack you might leave stuff out. Just imagine how much of a web application today that is made up by JavaScript for example, that would get tested if you didn’t test your views. Or for mocked data-access you wouldn’t really know if that integration actually worked until you tested it manually.

When I read the Specification by example book this summer I realized that many teams that are successful in implementing automated acceptance tests didn’t test the whole stack with all of their tests. On the key examples was implemented as full stack tests – while other test for edge cases or test completeness was executed against the “controller level” (as in Model-View-Controller) and mocked data.

I liked that approach very much. In my last project we ran in to major problems with the test execution time. We had 300 acceptance tests that just took too long to execute and was also quite brittle due to some problems with out GUI automation tool of choice.

So if you don’t have to – don’t go through the GUI.

Mind your language

Please note that even if you test through the GUI you should not write the test in terms of the GUI. Don’t write test scripts (“Click this”, “Fill out those”, “Choose that” etc.) but rather what focus on the behavior of the system ("Given I am logged in”, “When I add an item to the shopping cart”)

How about the database

I would argue in the same manner for the database – if you don’t have to, then don’t hit the real database. But let the key examples go through the whole stack and hit the database directly.

If you go against a real database you have a few options for managing test-data:

Have setup/teardown method that creates (and deletes) the test data the tests need. This can in Cucumber-like settings be accomplished with hooks and tags. I would prefer this approach if there’s not too extensive test data to set up.
Create a test database with known test-data and restore it before the tests. Use this approach by selecting out some of your real data too be used in tests. You have test data in a known state.

In the cases where you don’t go against real data (using mocks and stubs) you’ll have more fine grained control over the data your testing on and can create the edge cases (“should not happen”-stuff) easier. Also for these test the execution time will be much lower.

Conclusion

If you don’t have to run through the GUI and the complete application stack – don’t! Your test will run much faster and be less brittle.

Kanban-inizing the Avega Group office–getting the show on the road

2011-09-19T09:00:00.000+02:00

This is the second post in a small series on how we helped some parts of the Avega Group office to use Kanban to manage their workflow.
In the first post we introduced the teams that we coached to a foundations for why you want to limit your work-in-process, what benefits that could give them and some key elements of Kanban and Lean.
As a final exercise we helped them to create a simple Kanban board for each team. We stressed the fact that the boards not is complete (at any time!) but should change and this is just a suggestion.
Together with the teams we created a simple board, containing the following columns and policies:

Inbox – their to do list for a horizon that feel comfortable. We didn’t limit the number of items here, but suggested that they may think about that.
Also we told them that the size of the items here could vary. When the move it into the next column the work items is broken down into smaller tasks
Today – the items that is due today. We suggested that they should be between 0-8 hours in size and a work in process limit of max 6 per person.
Doing – the current work going on. We told them to create 2 avatars each and used that as WIP-limit for this column.
- This column has a sub column of Waiting for stuff that is waiting for someone or something that hinder the progress
Done – stuff they finished. We suggested that they keep the stickies around until the end of each week , grouped by day. And then to hold a retrospective based on the outcome of the week. What worked? What felt great/bad?
We offered our services for the first retrospectives.

And with that we let them loose…

First steps and hits

The next morning we found this in one room:

and saw this meeting taking place in another place at the office:

They were up and running. The first few days we got comment like these:

“Hmmm – why is 4 stickies in waiting? Maybe I can take one of them”
“Creating avatars was to much fun – we created 4 each “(Breaking the WIP-limit on doing with 100% (writers note))
“So nice to don’t have to hold all these task in my head”
“This made communication around the work easier and more concrete”
“Others will see how much we do. And have done – Ha ha!”

And finally a small episode that Morgan oversaw/heard:
As one of the teams was hold a morning meeting two “clients” to the team approached them. One of them saw that they were holding a morning meeting and went away, saying “I’ll be back later then”. The other one didn’t catch that and stood around. Stomping his feet. The team persevered and continued through their meeting.
Well done - team!

The road forward

As for now – we have asked the teams to grab any of the Lean/Agile coaches for any questions and at least invite one of us to their Friday retrospectives. That’s not much coaching but we trust them to learn by doing.
Also other teams on the Avega Group office has shown interest in us helping them in a similar fashion… Lean for the win!
I’ll try to post the rest of our story here.

Conclusion

Although we might have helped these teams and their working environment, I think the greatest learning experience was for myself and Morgan. If you are doing Kanban at your software project – take a look around and find other areas that can use the same techniques. You’ll get a deeper understanding of the underlying principles I promise

Kanban-inizing the Avega Group office–setting the stage

2011-09-16T09:00:00.000+02:00

I have recently returned from parental leave and got to experience office tempo first-hand. Was not prepared for that, but it also had me look around and I saw two teams at Avega Group that was under a lot of stress (they said) and seemed to handle a lot of task simultaneously.

Being dunked deeply in the Kanban-pool it made the hair on my neck stand up and I shivered to my bones. WIP is bad you know…

Being a Avega Coach I have some time to spend at the office, so I teamed up with Morgan, a fellow Lean/Agile coach here at Avega Group – and we decided to see if we could do anything about it.

In doing so we had to sharpen our arguments and think a bit different. They are not creating systems you know. Both Morgan and I have experiences mostly with software development teams, of course. So we saw a great learning opportunity.

This has just started but I thought I’ll try to blog about the experience here.

The teams and some background

The teams we approached were the Support team and the Marketing team. The support team is a small team (3 persons) that handles a lot of the short term activities that makes and office work. But they are also involved in some bigger projects such as an upcoming conference for the whole Avega Group (350 people), and getting activities there to work. Typical task is managing travel arrangements, answering phone and the door, booking events for Elevate and even refurnishing the office.

The marketing group is also small team with 3 persons. Their work is made up of a lot of short marketing project combined with some long term stuff, such as the annual report. They have a higher degree of specialization within the group and the work is creative in character. Typical task range from creating presentations, layouting invitations and programs for Elevate events, creating a visual presence at conferences such as OreDev. Through all their work the Avega Group brand is the most important thing.

Both team deliver with great quality and is very (!) accessible for anyone who need them.

Humble approach

First and foremost we needed to check with them if they also saw that they have problem. I had picked up some indices that they were under a lot of stress and I knew that they (both teams) handled LOADS of simultaneous work. Up to 30 small and large projects are some time in progress. I saw them listing it on the whiteboard, then items are delegate and then removed it from the whiteboard.

Secondly – we don’t know much about the work they’re doing. So we need to communicate to them that we don’t want to change what they do – merely the order and when.

So – we simply went up to the team leaders and asked them if they could spare us a minute or five. In character they both answered Yes! and more or less promptly left what they were doing and followed us.

When we asked if we could help – they both were very happy. Arm-waving took place in one instance .

So we set a date for a workshop – “Monday would be great. All the MD’s are away then – which means that we get a quieter day”. Hmmm – maybe something to talk about?

The goal

Our goal for them was not to increase throughput – they are already answering up to the demands from the organization both in throughput and quality. Amazingly and probably not indefinitely since the stress level will take it’s toll on quality at least.

Instead we focus on a steady pace and a more acceptable stress level. And a way to handle the interruptions that will and probably should occur in their normal work.

A practical exercise – learn to count

We kicked off the workshop with a practical exercise that we hope would give them understanding in that the more projects I focus on at the same time the longer all of them takes.

So we asked them to write down three columns (representing 3 projects they work on) with roman number I – X, A – J and 1 –10. To show how context switching has a bad result on performance we asked them to write each column with a different pen.

The first time around we asked them to write line by line, changing pen and focus constantly. Here is my own result.

The second time (you guess it) we asked them to write column by column finishing the first project before moving on to the next. Here is my result doing that:

Findings

This game was real eye-opener for the group (although one pair did the exercises in the reversed order ).

The first time the complete exercise took (with my numbers) roughly 2.5 time longer time (1.12 versus 0.28)
The first column – the most important project – was delivered after 001.10 minutes the first time around and in 8 seconds (!!) the second
The quality suffered in many cases (writing the wrong letters, writing uglier and forgetting rows) in the first exercise – while the second time around almost no-one messed up (I did – if you look closely )
The general feeling was “calmer and more focused” for the second iteration.

Morgan and I was very happy with the outcome that we thought created an awareness of the problems with the “old way” of doing things.

Lean and Kanban basics

From there it was quite easy to drive the main points of Kanban (Visual workflow, Limit Work-in-process and Help work to flow) and Morgan did a great presentation of it. But we were careful not to mention to much keywords and concepts focusing instead on the task at hand for them and how they could handle different situations.

End of part I

This blog post is getting too long already. I have posted the second part for Monday 0900. So you’ll have to wait until then to get the rest of our work.

Who writes the specification, now again?

2011-09-13T19:30:00.001+02:00

There’s been a lot of buzz around Specification by example lately. At least in my networks and close to me. I hear it almost everyday at the office. A couple of days ago I was asked to do an introduction to the subject to a group of interested people. Striking also is that most people I hear talking about it is not developers (anymore) but rather business/requirements people and testers.

One question that I often get asked and that puzzles me is:

But who writes the specifications? Business people, developers or testers?

Maybe not exactly like that but something close too that. Actually I think the underlying intent is something else and has to do with ownership…

I have also discussed the topic with a few of my Twitter contacts that are involved in a project using Specification by example. In this post I’ll give my view on this – based on experience from a project last winter and from the book by Gojko Adzic.

Touches 4 of 7 key process patterns

In the Specification by example book Gojko Adzic lists seven “key process patterns” – stuff that teams are doing that makes them successful. And that in essence is Specification by example. He doesn’t call it a method or methodology (it can be combined with anything you use to day), but rather refers to these 7 key process patterns.

The patterns are, very briefly (BUY THE BOOK):

Deriving scope from goals – where the scope for a feature or iteration is derived from the business goals
Specifying collaboratively – the specification and details surrounding a feature is discussed and talked about in a collaborative forum
Illustrating using examples – a practice where you clarifies your intent using concrete examples instead of natural language.
Refining the specification – this steps ensures that the specification is concise, conforms to the language in other specifications and adhere to best practices that the teams has agreed upon
Automating validation without changing specifications – with this practice teams uses tools (such as SpecFlow, FitNesse or Concordion) that can run the examples as executable tests against the system in question and validate that the system fulfills the specification
Validating frequently – the executable specifications are run often – such as part of a daily build – to shorten the feedback loop and get quick feedback on the current status of the system.
Evolving a documentation system – where the executable specifications are organized, easy to find and concise so that they can use it when taking the product forward, change features and maintaining the product

I think that the question above (Who writes the spec) touches on 4 of the patterns – at least – namely:

Specifying collaborative
Illustrate using examples
Refine the specification
Automate without changing the specification

Let’s look closer to what the question on who should write the specification means for these different process patterns.

“Who writes the spec?” and Specifying collaborative

That the specification is done collaborative is in my opinion a very important pattern. This means that you can do it with the whole team or in smaller groups but don’t let one person alone write the specification.

You could do this as a workshop with the whole team, writing a skeleton specification (with examples) on a white board. Or you can have small feature teams with just a few people representing the different discipline in the team, maybe even writing the specification directly into a tool for automation.

The thing with this practices is that you want to harvest the knowledge of the whole team; so that the business user can focus on how well the example fulfills the business needs, the developer can focus on how this can be implemented and suggest alternative solutions and the tester on how this can be tested and test coverage.

In doing this collaborative the team will (easier) get a collective ownership of the specification. Stuff that you have been a part in creating is much easier to take responsibility for. Compared to a specification that is handed over to you – written by someone else.

So in this aspect the answer to the question is: The Team does!

“Who writes the spec?” and Illustrate using examples

This is basically the same as above. If only one person is coming up with examples they tend to reflect that single person views and be limited by his creativity. Compare that to examples that has been created by a team with their collective knowledge and creativity.

Also here to the question is: The Team does!

“Who writes the spec?” and Refining the specification

Ok – in this process pattern we refine the specification and examples we have created together to conform with the best practices and uses the same language in the rest of the specifications.

So, if you like, this is where you create the actual specification document. It’s of paramount importance that the intent and examples are kept intact even if the language is tweaked into the one used in the other specifications.

If this was code this step would have been called Refactoring. Refactoring has been described as "disciplined technique for restructuring an existing body of code, altering its internal structure without changing its external behavior".

Often this is the first step where a computer based tool is needed. So if you’re using SpecFlow (of course you do, right?) you’ll use Visual Studio 2010 to get the rich experience with formatting, syntax highlighting and auto complete for already implemented steps.

My experience is that this is way over the head of any normal business user. Just installing Visual Studio and explaining what it is was more than some of our users managed in my latest project. And those were people in the IT-department, supporting the users of the system we were building.

So I would suggest that a technical person performs this step. But of course validating and checking with the other people in the team that any changes didn’t change the intent of the specification.

So here the answer would be: Someone who has enough technical knowledge for the tool does. But the Team has last say!

“Who writes the spec?” and Automate without changing the specification

In this step it’s up to a developer or a technical tester to connect the examples in the specification to the actual system in question. This is done by “automating” the application, that is writing code that runs the application. This can be tricky and messy if not done right – but it’s also way off the topic of this scope. To avoid the mess – read this.

So make no mistake – this is coding! A developer automate the steps in the specification and starts to build the actual system to fulfill them – one by one. The progress through the specification can be viewed as the steps are implemented and passes validation.

For this pattern the answer is: A developer or a technical tester does.

Conclusion

So there are a few answers to a question that I almost always get asked as I present Specification by example. I think it might be because that this is where there is a big difference from how we traditionally writes specifications or tests. They are done by a specialist and then handed over to another specialist to be fulfilled.

But that it also where the biggest improvements and benefits can be achieved.

When you have done this many times I suspect that all the 4 patterns above (or at least the 3 first I have listed) can be done all at once. Sitting around a computer and simply entering the specification and it’s example directly into the tool of choice.

I hope you found this helpful.

Twitter for beginners

2011-09-13T18:49:00.001+02:00

I got a question from 3 colleagues on what Twitter is and how we should use it. I don’t claim to be an expert but I use it quite a lot everyday. So here is my introduction to Twitter – with my own opinions on how to use it. There are other introductions, like this by Scott Hanselmann, of course – but since they asked I might as well put it up here.

What is it?

From Wikipedia we learn that:

Twitter is an online social networking and microblogging service that enables its users to send and read text-based posts of up to 140 characters, informally known as "tweets."

So you send short messages (tweets) that is picked up by anyone who follows you. You can follow a number of people to subscribe to their updates. Everything is public (almost see DM below) and can be viewed by anyone by going to your Twitter-page – that’s basically it.

But as with many simple things there are much more than meets the eyes.

What does X means?

Since the text messages only allow for 140 characters there are a few conventions and tricks that are used to communicate intent and more information in that short space. Here are few of them:

# – hashtag (or tags for short)

By writing a tag (#avegaConference for example) in your tweet other people can pick up what you have to say in a subject. Also you can easy follow the discussion going on for that tag. An example might clarify the use a bit:

Say that you are at a big conference with hundreds of other people. Of course you don’t follow all of them. That’s very impractical. But the conference might use a certain tag (#oredev, #javaZone2011 etc) and by including that tag in your message this can be picked up by other that search for that tag in all tweets.

You can find the most popular tags by searching for trending topics actually that’s a great way to keep up-to-date with important topics. The revolution in Egypt for example used #tahir (if I remember correctly) to collect all tweets about the event at Tahir Square.

@[a Twitter user name]

When you mention a Twitter user you write the @-sign followed by the twitter name (aka Twitter handle) for that user; @marcusoftnet to mention me for example. Most Twitter client applications can inform you on all the times somebody has mentioned you.

As special case of mentioning is to write the user name first in your tweet. That means that you direct the message to that user. It’s visible to that person and all the persons following both you and the person you mention.

So for example if I tweet:

“Saw @hugohaggmark eating at Vapiano today. Again!”

Everybody who follows me will see that tweet and know about Hugos lunch business. But if I instead write:

“@hugohaggmark do you want to tag along to Vapiano today? Again.”

Hugo will see this, but also the people who follow both me and Hugo.

Using this technique you can direct a message to anyone with a twitter account (@barackObama for example).

RT - retweet

A retweet is simply repeating what somebody else has said. This is the reason that news and other information can travel very, very fast on Twitter .

You simple start your tweet with RT and then a copy of the tweet you are re-tweeting. Most client application has built-in functionality to do this for you.

For example, if @hakanforss tweets this noteworthy information:

“Don’t forget Lean Coffe tomorrow at Le Café”

Everybody that follows Håkan will see this. Including me, since I follow Håkan. So I can simply re-tweet that information to all the people who are following me, by tweeting:

“RT @hakanforss - Don’t forget Lean Coffe tomorrow at Le Café”

With this simple mechanism we have now shared this information with hundreds of users, who in turn can retweet it. It’s an information sharing avalanche that cannot be stopped.

Smiles, abbreviations etc

Smiles and other abbreviations are plentiful. I would stick to a few since not many are well known. Here are a few that I use:

:) – I’m happy, that was fun
:( – I’m sad, sadly etc
;) – pun intended, irony
FTW – for the win and exclamation of triumph (#specflow #ftw)

Replies

If you reply to a tweet be sure to the Reply-function. Doing so give Twitter a chance to keep a nice thread going for you so that you can see the conversation in chronological order. Here is one example from the Twitter website:

DM – direct message

You can send private messages to someone you follow and who follows you. Start your tweet with “DM” and only you two will see the message. For example:

“DM @attefall Isn’t this one boring meeting? And it stinks from the person next to me :(”

That will not be picked up by anyone else. I would still use this function with care. It’s all too easy to forget the DM…. And if other people in the meeting, in my example follows you… You get the picture.

Integration with other social networks

Most other social networks can import your tweets. Mine show up on on LinkedIn for example.

I have also hooked up a nice service called SelectiveTweets that publish any tweets I do which include the tag “#fb” to Facebook.

There are numerous applications that can use the tweets you send (if you grant permissions of course) to do all kind of fun stuff.

More and more site allow you to sign in using your Twitter-account as well.

How do I use it?

Finally my own opinions on how to use Twitter.

First and foremost – let it wash over you. Don’t try to read every single tweet. Twitter has been called “The River of Crap” :). So if you don’t read for a day or two – no sweat. Pick up where you are now.
Follow people that YOU think are interesting. You are putting your information feed together for you. Most twitter users don’t get upset if you don’t follow them back – as has been know to happen on Facebook.
Get started by checking out some interesting tag and start to add your opinion on them. Soon you’ll be hooked too.
I have few hashtags and saved searches that I use to keep myself informed on certain areas. Great for keeping up to date – you can’t get information faster.
TURN IT OFF – when doing work that requires your concentration.
Be polite – remember that most of the things you write are publically accessible. I don’t tweet stuff that I cannot say to someone’s face or in public.

Conclusion

There you have my short introduction to Twitter and how to use it. Used right it can be a great source to news and information.

I hope you found useful. At least you Ulrica and friends

The dynamic keyword, ExpandoObject–a short intro for me

2011-09-05T15:08:00.001+02:00

Right away – this blog post is mostly for me. I have not been dabbling enough with the “dynamic” keyword to say that I know it. This is what and how I understand it now.

So, if you care to read this… please be gentle with your comments. I love to see them and learn that I was wrong – I’m most certainly am.

dynamic keyword

Since .NET 4.0 we have a new keyword – dynamic. If you read the MSDN documentation you’ll learn that:

The dynamic type enables the operations in which it occurs to bypass compile-time type checking. Instead, these operations are resolved at run time.

Ok, but what useful is that? I mean you *could* write this, you probably shouldn’t but you could:

   1: // Any object can be typed dynamic   2: // The dynamic keyword just means that    3: // the operations will be evaluated at runtime   4: dynamic s = "Marcus again";   5: Assert.AreEqual(s.Length, 12);

We now have a “dynamic” string. So the “s.Length” part compiles fine, but that’s actually just because the compile time checking is disabled. If I had written “s.StringLength” instead – that would also compile, but fail in runtime – since the string-class doesn’t implement a “StringLength” property.

Eeeh – so it’s a way to bypass the compiler and mess up?

I must admit that up to recently I hadn’t tried the dynamic keyword out and hence my thinking about it was here somewhere.

ExpandoObject using dynamic to add stuff at runtime

The first many of us (me that is) hear about is the mystically named ExpandoObject. You can add property or method you like to the ExpandoObject – and it just works. So, yes it’s a bit mystical. Let’s see it in action:

   1: dynamic de = new ExpandoObject();   2: de.Name = "Marcus";   3: de.Age = 39;   4: de.GetBirthCertificate =    5:     new Func<DateTime, string>(now =>   6:                      de.Name + " was born " + (now.Year - de.Age));   7:     8: Assert.AreEqual("Marcus was born 1972", 
                                de.GetBirthCertificate(DateTime.Now));

Yes – it’s super easy to add properties for Name and Age and then use them in a dynamic function that is defined by a lambda expression and use the dynamic variables. Even though those properties doesn’t exists on the ExpandoObject it compiles (since the variable is dynamic the compiler checking is disabled, remember) and works fine when we run it.

But how on earth can that work?

DynamicObject

Also included in the 4.0 release were a couple of classes that take full advantage of the dynamic keyword. They inherit from DynamicObject (some way or another) and it’s used as a base class for specifying dynamic behavior at runtime.

On the DynamicObject is a couple of interesting methods that you can override, for example TryGetMember, TrySetMember and TryInvokeMember. These method are run when a dynamic property is accessed and you can gracefully handle the non-existing member.

In the case of ExpandoObject it internally holds a IDictionary<string, object> that simply stores the name of the property as the key and the value as ... well … the value.

Actually it’s not too hard to implement. Here’s a rudimentary implementation for properties only:

   1: [TestFixture]

   2: public class MarcusExpandoTests

   3: {

   4:     [Test]

   5:     public void should_be_able_to_use_marcusexpando()

   6:     {

   7:         // My own Expando...

   8:         // Oh how close i was to name it MarcusSpandex

   9:         dynamic m = new MarcusExpando();

  10:         m.Name = "Marcus";

  11:         m.Age = 39;

  12:         Assert.AreEqual("Marcus is 39 old. My God!",

  13:             string.Format("{0} is {1} old. My God!", m.Name, m.Age));

  14:     }

  15: }

16:

  17: public class MarcusExpando : DynamicObject

  18: {

  19:     private readonly IDictionary<string, object> _dic = 
                      new Dictionary<string, object>();

20:

  21:     public override bool TrySetMember(SetMemberBinder binder, 
                      object value)

  22:     {

  23:         if (!_dic.ContainsKey(binder.Name))

  24:             _dic.Add(binder.Name, value);

  25:         else

  26:             _dic[binder.Name] = value;

27:

  28:         return true;

  29:     }

30:

  31:     public override bool TryGetMember(GetMemberBinder binder, 
                                 out object result)

  32:     {

  33:         result = _dic[binder.Name];

  34:         return true;

  35:     }

  36: }

If you check row 21 you’ll see how I override the methods TrySetMember. Then I can pickup the name of the dynamic property the developer used and store it, with the value, in a dictionary.

On row 31 it’s easy to override TryGetMember and retrieve the value from the dictionary.

So how can this be flexed

Well, there’s a million ways to use this, but to water your appetite check out Simple.Data. It’s a data access tool that uses dynamic to the full. Here’s an example:

   1: IEnumerable<User> u = Database   2:         .Users   3:         .FindByAllByName("Bob")   4:         .Cast<User>();

It looks like magic but now we can understand it. Ok – this is how I understand it without reading the source.

Database is the base object in Simple.Data. It inherits from DynamicObject and hence can use TryGet/SetMember and other methods for doing dynamic operations

When the user writes “.Users” Simple.Data stores that name to be used as the table name in the query we’re about to construct. So there is a table in the database called “Users”…

“.FindAllByName(“Bob”)” – this will again be picked up by TryGetMember and we can parse out the “FindAllBy” and “Name” part of the method name and the “Bob” argument.

Finally we cast the result to a IEnumerable of the concrete User class. So that’s simply going to a concrete implementation, filling out the User class with the columns returned by the query.

All in all we now have the parts to construct this query:

   1: SELECT * FROM Users    2: WHERE Name = 'Bob'

Neat – right?

Conclusion

The right use of the dynamic keyword can significantly shorten your code. It take some time to get used to and fully understand (I’m not there yet) and I think that care need to be taken on where you use it. But in the right places – and with the right introduction to the rest of the team – it’s very useful.

I hope that I wasn’t the only one that got anything from this. But if so … I’m now happy that I understand and can use the dynamic keyword.