Close
Glad You're Ready. Let's Get Started!

Let us know how we can contact you.

Thank you!

We'll respond shortly.

LABS
Avro's Reflect Datum Writer

As some of you may know, I’ve been writing a bit of Java in Boulder recently. Overall, it’s pretty exciting and a nice change from Ruby.

That being said, I’ve somewhat isolated our Java development to server-side components. I still consider Ruby/Rails to be the best solution for web application development, although recently had a need for Java.

Anyway, I’ve been playing around with Avro and I thought I’d share a small example that marshals a domain object to bytes. The bytes are later passed around several queues within application.


package com.barinek.devourer.avro;

import com.barinek.devourer.rest.resources.Activity;
import org.apache.avro.Schema;
import org.apache.avro.file.DataFileWriter;
import org.apache.avro.reflect.ReflectData;
import org.apache.avro.reflect.ReflectDatumWriter;

import java.io.ByteArrayOutputStream;
import java.io.IOException;

public class ReflectMarshaller {

    private final Schema schema = ReflectData.get().getSchema(Activity.class);

    public byte[] marshal(Activity activity) throws IOException {
        ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
        ReflectDatumWriter< Activity > reflectDatumWriter = new ReflectDatumWriter< Activity >(schema);
        DataFileWriter< Activity > writer = new DataFileWriter< Activity >(reflectDatumWriter).create(schema, outputStream);
        writer.append(activity);
        writer.close();
        return outputStream.toByteArray();
    }
}

The interesting bit is that I don’t have a .proto file or .json representation of the type. The Activity class is a Plain Old Java Object.

The above snippet is a candidate to replace some proto files, assuming the MicroBenchmark test suite passes.

Look for more Java posts in the near future.

Here’s an update with unmarshal and T params.


package com.barinek.devourer.avro;

import org.apache.avro.Schema;
import org.apache.avro.file.DataFileStream;
import org.apache.avro.file.DataFileWriter;
import org.apache.avro.reflect.ReflectData;
import org.apache.avro.reflect.ReflectDatumReader;
import org.apache.avro.reflect.ReflectDatumWriter;

import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;

public class ReflectMarshaller {

    public byte[] marshal(Object activity) throws IOException {
        Schema schema = ReflectData.get().getSchema(activity.getClass());
        ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
        ReflectDatumWriter< Object > reflectDatumWriter = new ReflectDatumWriter< Object >(schema);
        DataFileWriter< Object > writer = new DataFileWriter< Object >(reflectDatumWriter).create(schema, outputStream);
        writer.append(activity);
        writer.close();
        return outputStream.toByteArray();
    }

    public < T > T unmarshal(Class< T > returnType, byte[] bytes) throws IOException {
        Schema schema = ReflectData.get().getSchema(returnType);
        ByteArrayInputStream inputStream = new ByteArrayInputStream(bytes);
        ReflectDatumReader< T > reflectDatumReader = new ReflectDatumReader< T >(schema);
        DataFileStream< T > reader = new DataFileStream< T >(inputStream, reflectDatumReader);
        Object activity = reader.next();
        reader.close();
        inputStream.close();
        return ( T ) activity;
    }
}

Comments
Post a Comment

Your Information (Name required. Email address will not be displayed with comment.)

* Copy This Password *

* Type Or Paste Password Here *