Glad You're Ready. Let's Get Started!

Let us know how we can contact you.

Thank you!

We'll respond shortly.

Running Your Platform—Upgrade Much?

A common question for operations when considering running a platform for the first time is, what do you do to the platform when you are running all your apps on it? Be it for patches, upgrades, routine maintenance, or the ever-present security upgrades—how do you do this easily and without downtime? In this podcast, host Simon Elisha explains how Pivotal Cloud Foundry handles most of the heavy lifting for its users.

Shunting Ethernet Guests to a "Safe" Network


On occasion a non-employee will need to connect their laptop to our ethernet network, which begs the question, "How do we allow customers to access our network while protecting our workstations?"

The short answer is that we use a combination of VMPS-capable switches, VMPS software, VLANs, DNS, and DHCP. And, of course, reasonably stringent firewall rules.

We are a Software Services company, and at any given moment 40% of the 200-odd people in our San Francisco office are not employees. Of those 80 people, 98% of them can access the guest WiFi network without a problem. There are, however, the remaining 2% who, for whatever reason (their WiFi chipset doesn't interoperate well with our WiFi Access Points, their wireless is broken, they've accidentally deleted their drivers, etc...) cannot connect to the WiFi. They need to access the Internet, and they can only use ethernet.

We want to give our guests ethernet connectivity when needed, but not in such a way that it jeopardizes the security of our workstations.

Accessing the Packages that underlie Apple's App Store

Apple uses its App Store as a mechanism to distribute software, and it works quite well when a human operator is available to interact with it.

Unfortunately, many configuration management tools (e.g. chef, puppet) can't interact with the App Store, but they can interact with MAC OS X installer packages (.pkg, .mpkg files). We'll show you how to extract the underlying installer package file from the App Store.

This was tested under OS X 10.8.1 installing the OS X server package. It may not work for other packages.

Recovering OS X Open Directory from Backup

Lost Open Directory Database

You've lost your Open Directory server database. You need to recover it, but you don't have an Open Directory Archive, and you don't have a replica that you can promote. And you don't want to restore the entire server, either.

This blog post covers how to restore an Open Directory database from backup.


This blog post is directed towards system administrators

  • who have an Open Directory Server that is running OS X Snow Leopard 10.6.8 (this procedure would probably run under other versions of Snow Leopard, but we haven't tested it)
  • who do not have a replica that they can promote
  • who do not have a conventional Open Directory backup (i.e. Server Admin → Open Directory → Archive)
  • who want to do a surgical restore of just the Open Directory; who do not want to touch the other parts of the system
  • who have backed up their files.


This procedure worked for us; it may not work for you. YMMV. There is no warranty, express or implied. This is by no means an Apple-approved procedure.

Making Printers and Common Resources Available to Separate Network Segments via Bonjour and DNS-SD


When we moved to a new office, we faced a problem: how do we give printer access to everyone even though we had segregated machines to different networks? And how do we make it transparent to the user?

The solution we found was to add a new VLAN (i.e. network segment) for the printers (and other common resources, e.g. license servers), and use DNS Service Discovery (dns-sd) and a handful of crafted records to our DNS server.

This blog post is directed at Operations staff at companies which have the following characteristics:

  • primarily use Apple workstations
  • have network-attached printers
  • use Bonjour for printer discovery
  • need to print from multiple VLANs

The Problem

The short version: people couldn't print from the WiFi network.

Got Lion? Now Get Everything Else.

So you have a nice new Apple machine running Lion, but you don't want to spend the next few hours installing software.

What to do?

The short answer: use chef/soloist in conjunction with a slew of recipes developed at Pivotal Labs to help install the most common set of components.

Here's what to do:

  1. make sure you have Xcode installed; if not, you can install it through the App Store. Don't worry--it's a free application.
  2. Do the following:

    sudo gem install soloist
    mkdir -p ~/workspace
    cd ~/workspace
    git clone
    cat > ~/soloistrc <<EOF
    - $HOME/workspace
    - pivotal_workstation::meta_osx_base
    - pivotal_workstation::meta_osx_development
    - pivotal_workstation::meta_ruby_development
  3. It typically takes an hour for the chef run to complete. You can do other things while it's running (but if you reboot or logout you'll need to restart the chef run).

[Chef is a framework written by OpsCode to help configure in maintain one or more machines using 'recipes' (ruby scripts, more or less).]

[Soloist is an application which makes running chef-solo (the version of chef which runs without any centralized server) easier. It was written by my co-worker Matthew Kocher.]

What Is Installed by Default??

At the end of the chef run [as of this writing, and if all goes well] the following software will be installed:

  • java
  • Skype
  • SizeUp
  • Firefox
  • DropBox
  • Chrome
  • Mouse Locator
  • Homebrew
  • Pivotal Git scripts
  • Gitx
  • Vim
  • CCMenu
  • TextMate
  • KeyCastr
  • rvm
  • MySQL
  • postgres
  • RubyMine
  • ImageMagick
  • node.js

The following services will be enabled:

  • sshd
  • screen sharing
  • locate

The following preferences will be set:

  • better bash (history, PS1, inputrc)
  • faster key repeat
  • git (settings, a few scripts)
  • TextMate preferences (e.g. soft tabs)
  • RubyMine preferences (e.g. keymaps)

What if I don't want all that software? I just want TextMate and node.js!

Want to change the software that is installed? It's simple: just change your ~/soloistrc file. Here's a soloistrc that will only install TextMate & node.js:

- ./workspace
- pivotal_workstation::textmate
- pivotal_workstation::node_js

If you're interested in seeing all the recipes available (and there are quite a few), just browse the recipes in the pivotal_workstation repo.

Why Did You Choose That Set of Software?

Early in June, several pivots (Sean Beckett, Matthew Kocher, and David Goudreau, and I) met to decide on the bare minimum set of software and features that our developers would need to function on a new Lion Machine.

This set of chef recipes is the result of that meeting. There have been some changes (we have had great difficulty writing recipes to install firefox addons, so we iceboxed the story; some of our developers contributed recipes for things they wanted, so we added those).

Why Did You Choose Chef?

We chose chef/soloist partly because felt that our previous process had reached the end of its usefulness and were familiar with chef from our work automating server configuration.

Here's how our previous process worked:

  1. For minor releases (e.g. 10.6.7 → 10.6.8), we would take the previous golden image (a golden image is a snapshot of the disk drive of a machine with the applications, preferences, and settings that we wanted), install it on a workstation, upgrade the OS and possibly upgrade some of the applications. We would then use DeployStudio to take an image of the workstation, and that image would become the new golden image.
  2. For major releases (e.g. 10.5 → 10.6), we would re-create the golden image by hand, manually installing & configuring the individual software. For the Leopard/Snow Leopard transition, my co-worker Kevin Fitzpatrick spent a week painstakingly configuring the new machine. We then took an image using DeployStudio, and that image became the golden image.

This approach had several shortcomings:

  • It was monolithic: if you were a developer, there was no choice: there was only one image. This wasn't so bad when we were strictly a ruby shop, but when we expanded into android and iphone development, the monolithic approach began to show some shortcomings.
  • There was cruft in the image: the golden image had been built up over years.
  • It wasn't clear that the Golden Image would make the jump to Lion: Lion introduced some big changes (e.g. no PPC executables).
  • We were hesitant to approach the developers to ask them what they would like to see on the new image; we worried of re-igniting a Holy War.

We looked for alternatives. We wanted the following features:

  • We wanted to be able to install all (or almost all) of the features automatically (with minimal user intervention)
  • We wanted to be able to pick-and-choose which features were installed; the needs of an Android developer were different than those of an iPhone developer.
  • We want our developers to be happy; sure, they could install the features that they wanted or fix a problem with their workstation, but we want to go a step further: we want to provide the resources they need to write a recipe to minimize the effort the next developer has to go through. [This has been a success: several of our recipes were written in conjunction with developers.]


Integration tests for the cookbook took several days to set up. We use Faronic's Deep Freeze on a fairly pristine mac mini to ensure that we have a clean machine when we run our chef scripts. Continuous integration has proven invaluable for collaboration, for we quickly learn if a commit has unintended consequences.

In the more complex chef recipes, we attempt to write tests to test that they [the recipes] have succeeded; sshd_on.rb is a good example of testing that a service (sshd) was correctly started.


The chef runs, especially the initial one, are flaky. Our current chef run must download software from over 40 different servers, any one of which being down or having changed the download location can cause a failure. For example, Little CMS, a dependency of ImageMagick, resided on, which was down for a few days. Our integration tests failed during that period.

If you encounter a server being down or a file that has moved, please send us a pull request with an updated download location, or just comment-out the broken recipe.

Target Audience

Our target audience is developers, which is great: they understand errors, and often contribute code fixes. Our goal is for Pivotal Ops to provide a framework for Pivotal Engineers to write the recipes that build the workstations they want.


I am grateful to Matthew Kocher, who more than anyone helped me write the bulk of the ruby scripts. Also to Sean Beckett, without whose support this would never have happened. And to the many pivots who offered suggestions & help.