It’s best best best.
Amidst all the excitement around the thumbprint scanner and plastic bling of the new iPhones, no one’s paying much attention to the newly minted Gold Standard release of Xcode 5.0. This release brings a whole suite of changes that we here at Xtreme Labs have been tinkering with all summer and we would like to share with you some of the more interesting features that jump out at us.
Read more »
“Everything that can be automated must be automated.”
I spent almost 20 years in Manufacturing following that principle, because it works on so many levels. Old habits die hard, so even after I started doing more software / less hardware I still found myself following that deeply ingrained directive.
Read more »
Xcode has a little known feature that can save you time, frustration, and ultimately can even improve code quality. Templates are what Xcode uses to create new projects and files — when you create an “Objective-C Class” Xcode is just populating the Objective-C class.xctemplate with your options.
Read more »
So if you’re anything like me you get a lump in your throat and a cold sweat every time you are about to hit the “SUBMIT” button in iTunes Connect and release your latest creation upon the world.
Usually about five minutes later the voice in the back of my head starts whining “What if we missed something?
Read more »
tl;dr Latest version of Cedar can nicely integrate with Xcode just like OCUnit.
When creating new Xcode project you might have noticed that New Project dialog has 'Include Unit Tests' checkbox at the bottom. When that option is selected in addition to creating an app target Xcode will also create a separate target that is used for testing your application. This target most likely will be named [app]Tests. Convenient, huh! Since Xcode is such a nice IDE it will also create an example test file that uses Xcode's built-in unit testing framework -- OCUnit (also known as SenTestingKit). If you never heard of it below is a screenshot of how that example OCUnit test file looks.
Because Apple ships OCUnit with Xcode it is tightly integrated into the IDE and that gives it somewhat of an advantage over other testing frameworks. Example of such unfair treatment is relatively nice looking red error markers (shown above) that show up next to failed test cases when application tests are run. If you have ever used Cedar before you would know that when Cedar specs fail we do not get those markers but rather we are presented with boring gray console output (we have dots though!). Unfortunately there is no documented way for custom testing frameworks to mimic OCUnit behavior without building on top of it. But even if do decide to use OCUnit as a base for your custom testing framework you will run into several problems. One of those problems, at least a year ago, was that you could not reliably use debugger with OCUnit tests. That was one of the main reasons why Cedar was not build on top of it. Recently, however, Xcode 4 was released with a lot of enhancements including better support for running OCUnit tests (with or without debugger) or so it seems. Red markers being a driving force, once again it was worth to investigate on how take advantage of OCUnit goodies from Cedar. Latest version of Cedar succeeds at that. What follows are the steps on how to setup Cedar for maximum Xcode integration.
Check 'Include Unit Tests' checkbox in New Project dialog. If you want to add application tests to an existing project here is a very nice tutorial: http://twobitlabs.com/2011/06/adding-ocunit-to-an-existing-ios-project-with-xcode-4/
After Xcode is done creating a new project file go to Build Settings for application tests target (in this case ExampleTests) and find Other Linker Flags settings. Add
-lstdc++ to that setting and click Done.
Next step is to add Cedar static library to application tests target.
Now you are ready to create a Cedar spec file. Create a new empty file and end the filename with 'Spec.mm'. 'mm' file extension is needed to be able to use built-in Cedar matcher library. Also since this file should only be run as part of application tests and not the actual application make sure that you only include it in application tests target.
Add a test or two (see above for example) to be able to test that Cedar is working correctly. Example above tries to assert that 2+2 equals 5 and that should hopefully fail. Notice that here I'm using recently added Cedar matchers should syntax. If you do not like that you can always rewrite that expectation as
Larger image. And finally to run specs you just wrote select scheme for your app (in this case Example) from the drop down in the upper left corner and press 'Cmd+U'. 'Cmd+U' in Xcode by default runs tests associated with selected scheme. That should first build your app (because application tests target lists the app as a target dependency), then build application tests target and then run the tests. Since 2+2 does not equal 5 you will see that Xcode will highlight that line in red and show that as an error in the errors panel on the left side. Fixing that expectation should make the test pass.
Running from the command line
If you want to run your application tests from the command line you can use
rake ocunit:application to do that. Before doing so make sure you have updated
OCUNIT_APPLICATION_SPECS_TARGET_NAME in the Rakefile. Also you'll need to use Debug for
CONFIGURATION so that your tests are able to include classes from your application. What's nice about this rake task is that it is ready to be used on your CI machine since it returns exit code 1 when any tests fail and 0 if they all succeed. It also prints in color.
How it actually works
If you are wondering how Cedar pretends to be OCUnit here is a short summary:
- When Cedar is loaded it overrides runTests class method on
SenTestProbeclass (related file to look at is CDROTestIPhoneRunner.m)
SenTestProbe +runTestsgets called by otest -- an executable that Xcode uses to run tests after it injected the application tests bundle into TEST_HOST app
SenTestProbe +runTestswas overridden with Cedar's run method Cedar executes all of its specs following all the usual rules: focused, pended, etc.
- When specs are running
CDROTestReporter(source) gets notified which tests passed, failed, got skipped, etc. and prints out appropriate information which you see in console log
CDROTestReporterprints out failed tests in a way that is recognized by Xcode (OCUnit spits out same output for failed examples) e.g.:
Test Case '-[ExampleTests testExample]' started. /Users/work/workspace/Example/ExampleTests/ExampleTests.m:16: error: -[ExampleTests testExample] : '4' should be equal to '5': 2+2 != 5 Test Case '-[ExampleTests testExample]' failed (0.000 seconds).
- Xcode notices that pattern in the console output, parses it, gets filename and line number and marks that file location with a red marker to indicate a test failure
Drawbacks/Not implemented features
- Debugger support is still a bit flaky but this is a problem with Xcode/OCUnit integration
- I have not tried running it on the device but it should work
- If you are used to specifying
-SenTestoption to run specific test case it will be ignored; however, you can use Cedar's fit, fdescribe, and fcontext to focus on specific tests
- Some third party OCUnit add-ons might not work (let us know about them)
- Cedar source code
- Apple Unit Testing Overview
- Official but outdated OCUnit and otest source code
- More recent version of OCUnit source code
- Public Pivotal Tracker project for Cedar to submit bugs and feature requests
 It's not.