Glad You're Ready. Let's Get Started!

Let us know how we can contact you.

Thank you!

We'll respond shortly.

Tagged in 's3'

Tech How To: Using Pivotal Greenplum Hybrid Queries on Amazon S3 Data

In this post, Pivotal data engineers Amey Banarse and Qi Shao follow the recent announcement of Pivotal Greenplum’s new ability to query both on-premise and AWS S3 data. They cover a high level overview of the key technical scenarios, a deeper technical explanation of how it works, and a hands-on set of steps for getting started.

Read More

Backing up VCSA 5.5 DBs to S3

The Cloud Foundry Development Teams use a heavily-customized VMware vCenter Server Appliance (VCSA) 5.5. We needed to architect an offsite backup solution of the VCSA’s databases to avoid days of lost developer time in the event of a catastrophic failure.…

Read more

Read More

Mocking Fog when using it with Carrierwave

There is a new kid on the block when it comes to file attachments for Rails and it is called Carrierwave.

Carrierwave gives you the ability to easily store attachments on S3 using another great gem called Fog.

Uploading files to S3 is great for many reasons but it can slow down your testing environment because it takes a while to send stuff up to S3. The Carrierwave documentation tells you how to switch the storage location over to file storage during testing but that wasn't enough for me. I wanted to use the same storage mechanism for dev, test and production so I sought out a way to do so.

I had heard about Fog's ability to mock itself to pretend that it was interacting with S3 so I decided to see if I could get it working with Carrierwave. This allowed me to use the same storage mechanism in test mode without slowing my tests down waiting for images to go to S3.

After a bunch of tinkering and a message on the Fog mailing list(thanks for the quick response Wesley), this is what I came up with:

The key is that you have to tell the mocked Fog that an S3 bucket exists before it will let Carrierwave put an image there. I wasn't doing this at first and Carrierwave kept showing me a 404 error from Fog.

Drop this in a file in your spec/support and/or features/support directories and you will have your tests thinking they are sending things to S3 without actually sending them to S3.

Now I don't have to mess around with having a bunch of test images laying around my hard drive and I can make sure I'm using the same storage mechanism across all environments without slowing my tests down.

Your mileage may vary but I'd love to hear how this works for people and if there are any limitations. I haven't found any yet.

UPDATE: Check out my guest post on the Engine Yard Blog for updated details:

Read More

Interacting with AWS S3 in ruby…

If you are looking for a ruby gem to interact with S3, you may want to choose the S3 gem over the AWS::S3 gem.

The S3 gem seems to be under active development while the AWS::S3 gem gem seems to have gotten a little stale.

I mistakenly chose the AWS::S3 gem gem based on the number of downloads on but it was missing the ability to copy from one S3 bucket to another without having to move the files through my local machine. That will teach me for not reading the source code...

Read More