This project is read-only.

TDD / Mocks Framework Decision Needed

Nov 30, 2007 at 8:12 PM
We need to make a decision on the testing framework to be used by the project.

I have worked with NUnit and the VS Unit Testing framework. I find both of them to be acceptable.

If everyone has access to the VS Testing Framework I would say use that because of the ability to integrate check-in policy with this (you cannot check in unless unit tests pass). However I recognize that not everyone agrees with this so I don't have a problem using some other unit test framework.

We do need a Test Lead who will drive the decision and write up Test Coding Standards for the team as well as review the tests that are being written looking for Test Smells.

Mock Framework

I have used both NMock and Rhino.Mock. I found NMock easier but Rhino.Mock more powerful. I have heard of another one called "Type Mock"(?) that is supposed to be really great but have no idea. Be careful about overuse of mocks and behavioral testing because these type of tests become very brittle quickly. I prefer state based testing for this reason. If you are interested in a good resource on this and other testing topics check out xUnit Test Patterns.
Dec 2, 2007 at 2:35 AM
Having the coding and testing standards in place is a great idea. Standards will help people get up to speed and involved quickly by having a consistent feel to the code base and tests.

My initial feeling is that due to this being an open source project, it should be approachable by anyone who wants to contribute.

Taking this further it seems appropriate to ensure that all the tools needed to participate were freely available; we would not want to exclude people who did not have access to an MSDN subscription, or a copy of VS2008 with testing capabilities built in.

I suggest we start out with NUnit for the testing framework as it is widely known and used throughout the community. If we do need to switch to MbUnit in the future it is an easy process. MbUnit supports all of NUnits attributes and Assert syntax.

I know I probably shouldn't do, but I am going to assume that we will be splitting Doppler into a set of components or services that will interact to perform the necessary functions of the application. This is where I think that the mocking frameworks will help to isolate the system under test and help keep the tests clean and simple. I agree that too much mocking and interaction based tests can be brittle and sometimes indicate a problem with the design, I have felt this pain in the past.

I suggest that we keep it simple to start with and see how much mileage we get from NMock2. Again if we find that we need something more powerful we can look towards Rhino.Mocks.
Dec 3, 2007 at 3:14 PM
I agree - we should make this possible with freely available tools.

I'm fine with NMock except that the version I worked with had little or no support for Events. When it comes to the MVP testing this really hurts.
Dec 4, 2007 at 9:00 AM
I think that we should seriously consider Rhino Mocks right from the start then.

Documentation on how to check that the object under tets subscribes to an event on a mock object:
http://www.ayende.com/Wiki/Rhino+Mocks+Events

Documentation on how to raise an event from within a mock object:
http://www.ayende.com/Wiki/Rhino+Mocks+IEventRaiser

J.P. Boodhoo has a good post on how to work with Rhino Mocks for raising events:
http://codebetter.com/blogs/jean-paul_boodhoo/archive/2007/05/07/raising-events-from-a-mock-using-rhino-mocks.aspx

Screen cast on how to use Rhino Mocks to help TDD
http://www.ayende.com/97/section.aspx/download/194

Other interesting screen casts
http://www.ayende.com/hibernating-rhinos.aspx
Dec 4, 2007 at 6:18 PM
My pal Scott Hanselman is in love with TypeMock - http://www.typemock.com/. I have worked with Rhino Mocks a bit and I would have to say I did find it more difficult to use than NMock but I did it because of the event support (which did work).

I think we ought to investigate TypeMock - they do have a free community version.
Dec 4, 2007 at 9:58 PM
while i have little to no *Mock info, I do know that typemock has a plugin available for cruisecontrol .net and i will be adding support for that plugin in ccnetconfig on the next check in of the code. I am not aware of any other plugins for the mock frameworks.
Dec 4, 2007 at 10:44 PM
We ought to give it a try. Anyone want to take some time to experiment with TypeMock? I'm going to give it a go.
Dec 5, 2007 at 5:32 PM
I have downloaded and installed TypeMock. I'll have a play tonight and see how I get on.
Dec 6, 2007 at 11:43 AM
Edited Dec 6, 2007 at 11:48 AM
Here are my initial thoughts on TypeMock (after only a short time playing around): -

It is a shame that the eventing features are part of the professional version and not available in the community version. If the lack of eventing in NMock2 is the only reason not to use it, then that would be a reason not to use TypeMock. This link has some interesting information on adding event support to NMock2 http://becomingagile.blogspot.com/2007/03/raising-events-in-nmock-20.html
edit Found this one as well http://manicprogrammer.com/cs/blogs/willeke/archive/2007/10/17/testing-delegates-with-nunit-and-nmock.aspx/edit

TypeMock does look like a very powerful way to do mocking. It may well become something that I use in my day job, with the professional or enterprise edition.

I would worry that people who are new to mocking might find the fact that some calls to an object instance are mocked and others are not mocked to be very confusing.
I got the hang of it pretty quickly but mixing the two up may be confusing.

Did anyone else have a play around with it?
Dec 6, 2007 at 5:31 PM
Here is the link to the Documentation on Rhino Mocks.
http://www.ayende.com/Wiki/Default.aspx?Page=Rhino+Mocks+Documentation

Here is the link to the Documentation on Rhino Mocks API
http://www.ayende.com/projects/rhino-mocks/api/files/MockRepository-cs.html

Some screen casts a couple of which are about Mocking.
http://www.ayende.com/hibernating-rhinos.aspx
Dec 7, 2007 at 3:19 PM
Unless there are compelling reasons not to I think I'm inclined to use Rhino Mocks because of its event support. As Test Lead Aiden you get to make the call though.
Dec 7, 2007 at 4:40 PM
I will put together a small application over the weekend to demonstrate the use of Rhino Mocks and BDD with Passive View.

At least that is what I will try to accomplish. It all depends on how long my girlfriend drags me around the shops for tomorrow.
Dec 11, 2007 at 1:59 PM
This is just a quick update, as things seem to have been quite.

I am working on a sample application using Rhino Mocks and NUnit. I didn't get much time at the weekend as I ended up spending 8 hours shopping on Saturday...

What I have found so far.

Passive View

Testing a passive view implementation is extrememly simple with Rhino Mocks. It took a few minutes to work out the syntax but here is a taster of what I am doing.
1	[TestFixture]
	public class DVDListPresenterTests
	{
		private IDVDCollectionModel _mockModel;
5               private IDVDCollectionView _mockView;
 
		private MockRepository _mocks;
 
		[SetUp]
10	        public void Setup()
		{
                        // Mock repository manages the mocks
			_mocks = new MockRepository();
                        // Mock up the interface for my view
15		        _mockView = _mocks.CreateMock<IDVDCollectionView>();
                        // Mock up the interface for my model
			_mockModel = _mocks.CreateMock<IDVDCollectionModel>();
		}
 
20		[TearDown]
		public void TearDown()
		{
                        // Placing the verify in the teardown ensures that I don't forget to call it.
			_mocks.VerifyAll();
25		}
 
                [Test]
		public void ShouldSetupViewWhenModelInitialises()
		{       
30                      // Setup some test data.
  		        DVD[] dvds = new DVD[]{new DVD("Testing")};
                        // This sets an expectation on the view that DVDs will be set to the array dvds
			_mockView.DVDs = dvds;
                        // Setup the expectation that someone will assign an event handler to the Updated event on the model
35			_mockModel.Updated += null;
			LastCall.IgnoreArguments();
                        // Setup the expectation that someone will assign an event handler to the Intialise event on the model
			_mockModel.Initialise += null;
			LastCall.IgnoreArguments();
40                       // Creates an event raiser that will actually fire the Initialise event
			IEventRaiser raiser = LastCall.GetEventRaiser();
                        // Switch all of the Mocks into replay mode
			_mocks.ReplayAll();
                        //  Create an instance of my presenter, passing in the view and the model
45			DVDCollectionPresenter presenter = new DVDCollectionPresenter(_mockView, _mockModel);
                        // Use the event raiser to fire the Initialise event from the mock model
			raiser.Raise(this, EventArgs.Empty);
		}
        }
Apologies for the mass of code. As you can see in this example it is quite easy to fire the event from the mock instance of the model and ensure that the property is set on the view.

I think that I should probably test the view in this case using a state based test by asserting the value of the property DVDs after raising the event.

To accomplish this I have to alter line 15
15		        _mockView = _mocks.CreateMock<IDVDCollectionView>();
To look like this:
15		        _mockView = _mocks.Stub<IDVDCollectionView>();
Remove line 33
                        _mockView.DVDs = dvds;
Then add another line after the event is raised (last line of the test)
			Assert.That(stubView.DVDs, NUnit.Framework.SyntaxHelpers.Is.EqualTo(dvds), "The DVD property was not set correctly");
There is a slight problem here with the NUnit.Framework.SyntaxHelpers.Is syntax bumping up against the Rhino.Mocks.Constraints.Is

BDD

I am not sure if I am doing this correctly or not. I have changed the names of the test methods to describe the desired behaviour of the system under test. I have a set of user stories that go along with work that I am doing, gathered from myself and my girlfriend as customers. If anyone is interested in what they look like I can post them on here somewhere.

Once I have implemented the view and the model for this I will check in the code in one of the spike folders.
Dec 14, 2007 at 5:41 PM
@Aiden you could also change the syntax to this

{
Test
public void ShouldSetupViewWhenModelInitialises()
{
// Setup some test data.
DVD[] dvds = new DVD[] { new DVD("Testing") };

using (_mockView.Record())
{
// This sets an expectation on the view that DVDs will be set to the array dvds
_mockView.DVDs = dvds;
// Setup the expectation that someone will assign an event handler to the Updated event on the model
_mockModel.Updated += null;
LastCall.IgnoreArguments();
// Setup the expectation that someone will assign an event handler to the Intialise event on the model
_mockModel.Initialise += null;
LastCall.IgnoreArguments();
// Creates an event raiser that will actually fire the Initialise event
IEventRaiser raiser = LastCall.GetEventRaiser();

}

using (_mockView.Playback())
{
// Create an instance of my presenter, passing in the view and the model
DVDCollectionPresenter presenter = new DVDCollectionPresenter(_mockView, _mockModel);
// Use the event raiser to fire the Initialise event from the mock model
raiser.Raise(this, EventArgs.Empty);
}
}
}

you probably worked this out already but just in case you didn't i thought i would post it up.

The benefit of the using blocks is that you can clearly see the record/replay phase of the unit test.

Also on the typemock versus rhino argument google "typemock to powerfull" to get a different perspective on typemock.

Sorry to intrude, just saw your debate and though I would add my $0.02.

Dermot
Dec 15, 2007 at 10:59 PM

dkil1972 wrote:

The benefit of the using blocks is that you can clearly see the record/replay phase of the unit test.


To be honest with you I have not looked into the "using" ability in Rhino Mocks. You are right, this is one way to help separate the phases of the testing code.
I think I need to go have a look at this to see if there are any side effects to the using statement other than the extra separation.


Also on the typemock versus rhino argument google "typemock to powerfull" to get a different perspective on typemock.

Sorry to intrude, just saw your debate and though I would add my $0.02.

No need to apologize, this is an open conversation and you brought up good points. Thanks for chipping in.

Aiden
Dec 27, 2007 at 3:23 PM
Edited Dec 27, 2007 at 3:24 PM
We used the Passive View pattern in our last project and it was easy to figure out, how to use it with Events:
    [TestFixture]
    public class TestSignOnPresenter
    {
        private Mockery m_mockery;
        private ISignOnView m_signOnViewMock;
        private IClientSession m_clientSessionMock;
        private IServerConsoleModel m_serverConsoleModelMock;
 
        private const BindingFlags DEFAULT_BINDING_FLAGS = BindingFlags.IgnoreCase | BindingFlags.Instance | BindingFlags.NonPublic;
 
        [SetUp]
        public void SetUp()
        {
            Locator locator = new Locator();
            PmMda.Net.Application.Locator.Register(locator);
 
            m_mockery = new Mockery();
            m_signOnViewMock = m_mockery.NewMock<ISignOnView>();
            m_clientSessionMock = m_mockery.NewMock<IClientSession>();
            m_serverConsoleModelMock = m_mockery.NewMock<IServerConsoleModel>();
            Locator.Instance.Register<IClientSession>(m_clientSessionMock);
            Locator.Instance.Register<Navigator>(new Navigator());
            Locator.Instance.Register<IServerConsoleModel>(m_serverConsoleModelMock);
        }
 
        [TearDown]
        public void TearDown()
        {
            foreach (PmMda.Net.Application.IComponent component in Locator.Instance.RegisteredComponents)
            {
                Locator.Instance.Deregister(component);
            }
 
            // This call will set the internal member to null:
            PmMda.Net.Application.Locator.Deregister();
 
            m_signOnViewMock = null;
            m_clientSessionMock = null;
            m_serverConsoleModelMock = null;
            m_mockery.Dispose();
        }
 
        [Test]
        public void TestPasswordChangeCancelled()
        {
            Expect.Once.On(m_signOnViewMock).EventAdd("DataChanged", Is.Anything);
            Expect.Once.On(m_signOnViewMock).EventAdd("SignOnRequested", Is.Anything);
            Expect.Once.On(m_signOnViewMock).EventRemove("SignOnRequested", Is.Anything);
            Expect.AtMost(1).On(m_signOnViewMock).Method("ShowResultOfSignOn");
            Expect.Once.On(m_clientSessionMock).Method("SignOff");
 
            using (SignOnPresenter presenter = new SignOnPresenter())
            {
                // sequence is crucial
                presenter.Navigator = Locator.Instance.Get<Navigator>();
                Locator.Instance.Get<Navigator>().State[NavStates.UnitTest] = true;
                Locator.Instance.Get<Navigator>().State[NavStates.UserHasChangedPassword] = false;
                presenter.View = m_signOnViewMock;
 
                // We call the Method on the Presenter same as when the user clicked the button in the view:
                presenter.GetType().GetMethod("BackFromChangePassword", DEFAULT_BINDING_FLAGS).Invoke(presenter, null);
            }
            m_mockery.VerifyAllExpectationsHaveBeenMet();
        }

I'm currently starting to continue development on NMock2. One of the first features will be better support for Events.

Thomas