#summary Defining and registering entities

First define your domain model in Java using Objectify's annotations, then register these classes with the {{{ObjectifyFactory}}}.

<wiki:toc max_depth="3" />

= Defining Entities =

Entities are simple Java POJOs with a handful of special annotations.  Objectify has its own annotations and *does NOT use JPA or JDO annotations*.

Note that throughout this documentation we will leave off getter and setter methods for brevity.

== The Basics ==

Here is a minimal entity class:

{{{
import com.googlecode.objectify.annotation.Entity;
import com.googlecode.objectify.annotation.Id;
import com.googlecode.objectify.annotation.Ignore;

@Entity
public class Car {
    @Id Long id;
    String vin;
    int color;
    byte[] rawData;
    @Ignore irrelevant;

    private Car() {}
    
    public Car(String vin, int color) {
        this.vin = vin;
        this.color = color;
    }
}
}}}

Observe:

  * Entity classes must be annotated with {{{@Entity}}}.

  * Objectify persists fields and only fields.  It does not arbitrarily map fields to the datastore; if you want to change the way a field is stored, rename the field.  Getters and setters are ignored so you can isolate the public interface of your class (eg, {{{public String getVehicleIdentificationNumber() { return vin; }}}}).

 * Objectify will not persist {{{static}}} fields, {{{final}}} fields, or fields annotated with {{{@Ignore}}}.  It *will* persist fields with the {{{transient}}} keyword, which only affects serialization.

  * Entities must have have one field annotated with {{{@Id}}}.  The actual name of the field is irrelevant and can be renamed at any time, even after data is persisted.  This value (along with the kind 'Car') becomes part of the {{{Key}}} which identifies an entity.

  * The {{{@Id}}} field can be of type {{{Long}}}, {{{long}}}, or {{{String}}}.  If you use {{{Long}}} and save an entity with a null id, a numeric value will be generated for you using the standard GAE allocator for this kind.  If you use {{{String}}} or the primitive {{{long}}} type, values will never be autogenerated.

  * You can persist any of the [http://code.google.com/appengine/docs/java/datastore/dataclasses.html#Core_Value_Types core value types], Collections (ie Lists and Sets) of the core value types, or arrays of the core value types.  You can also persist properties of type {{{Key}}} (the native datastore type) and {{{Key<?>}}}) (the Objectify generified version).  There are more advanced types that can be persisted which will be discussed later.

  * There must be a no-arg constructor (or no constructors - Java creates a default no-arg constructor).  The no-arg constructor can have any protection level (private, public, etc).

  * {{{String}}} fields which store more than 500 characters (the GAE limit) are automatically converted to {{{Text}}} internally.  Note that {{{Text}}} fields, like {{{Blob}}} fields, are never indexed (see [Queries]).

  * {{{byte[]}}} fields are automatically converted to {{{Blob}}} internally.  However, {{{Byte[]}}} is persisted "normally" as an array of (potentially indexed) {{{Byte}}} objects.  Note that GAE natively stores all integer values as a 64-bit long.

== Polymorphism ==

Objectify lets you define a polymorphic hierarchy of related entity classes, and then load and query them without knowing the specific subtype.  Here are some examples:

{{{
import com.googlecode.objectify.annotation.Entity;
import com.googlecode.objectify.annotation.EntitySubclass;
import com.googlecode.objectify.annotation.Id;

@Entity
public class Animal {
    @Id Long id;
    String name;
}
	
@EntitySubclass(index=true)
public class Mammal extends Animal {
    boolean longHair;
}
	
@EntitySubclass(index=true)
public class Cat extends Mammal {
    boolean hypoallergenic;
}
}}}

Things to note:

  * The root of your polymorphic hierarchy *must* be annotated with {{{@Entity}}}.
  * All polymorphic subclasses must be annotated with {{{@EntitySubclass}}}.
  * You can skip {{{@EntitySubclass}}} on intermediate classes which will never be materialized or queried for.
  * You should register all classes in the hierarchy separately, but order is not important.
  * Polymorphism applies only to entities, not to @Embed classes.

In a polymorphic hierarchy, you can {{{load()}}} without knowing the actual type:

{{{
Animal annie = new Animal();
annie.name = "Annie";
ofy().save().entity(annie).now();

Mammal mam = new Mammal();
mam.name = "Mam";
m.longHair = true;
ofy().save().entity(mam).now();

Cat nyan = new Cat();
nyan.name = "Nyan";
nyan.longHair = true;
nyan.hypoallergenic = true;
ofy().save().entity(nyan).now();

// This will return the Cat
Animal fetched = ofy().load().type(Animal.class).id(nyan.id).now();

// This query will produce three objects, the Animal, Mammal, and Cat
Query<Animal> all = ofy().load().type(Animal.class);

// This query will produce the Mammal and Cat
Query<Mammal> mammals = ofy().load().type(Mammal.class);
}}}

=== Polymorphism Native Representation ===

When you store a polymorphic entity subclass your entity is stored with two additional, hidden synthetic properties:

  * _^d_ holds a discriminator value for the concrete class type.  This defaults to the class shortname but can be modified with the {{{@EntitySubclass(name="alternate")}}} annotation.
  * _^i_ holds an indexed list of all the discriminators relavant to a class; for example a Cat would have [["Mammal", "Cat]].  Note that *subclasses are not indexed by default*.  See below.

The indexed property is what allows polymorphic queries to work.  It also means that you cannot simply change your hierarchy arbitrarily and expect queries to continue to work as expected - you may need to re-save() all affected entities to rewrite the indexed field.

There are two ways you can affect this:

  # You can control indexing of subclasses by specifying {{{@EntitySubclass(index=true)}}}.  *Subclasses are not indexed by default*, you must explicitly enable this for each subclass.  Note that this only affects queries; you can always get-by-key or get-by-id and receive the proper typed object irrespective of indexing.
  # You can use {{{@EntitySubclass(alsoLoad="OldDiscriminator")}}} to "reclaim" old discriminator values when changing class names.  Note that this will not help with query indexes, which must be re-saved().

== Embedding ==

Objectify supports embedded classes and collections of embedded classes.  This allows you to store structured data within a single POJO entity in a way that remains queryable.  With a few limitations, this can be an excellent replacement for storing JSON data.

*IMPORTANT!* Read the note about changes in the [Entities#Embedding_Native_Representation native storage representation].

=== Embedded Classes ===

You can nest objects to any arbitrary level.

{{{
import com.googlecode.objectify.annotation.Embed;
import com.googlecode.objectify.annotation.Entity;
import com.googlecode.objectify.annotation.Id;

@Embed
class LevelTwo {
    String bar;
}

@Embed
class LevelOne {
    String foo;
    LevelTwo two
}

@Entity
class EntityWithEmbedded {
    @Id Long id;
    LevelOne one;
}
}}}

=== Embedded Collections and Arrays ===

You can place {{{@Embed}}} objects in collections and arrays:

{{{
@Entity
class EntityWithEmbeddedCollection {
    @Id Long id;
    List<LevelOne> ones = new ArrayList<LevelOne>();
}
}}}

Some things to keep in mind:

  * This does not support two-dimensional structures of any kind.
    * You cannot nest {{{@Embed}}} arrays/collections inside other {{{@Embed}}} arrays/collections.
    * You cannot put arrays/collections of native types inside {{{@Embed}}} arrays/collections.
  * You can, however, nest {{{@Embed}}} arrays/collections inside any number of {{{@Embed}}} classes.
  * You should initialize collections.  Null or empty collections are not written to the datastore and therefore get ignored during load.  Furthermore, the concrete instance will be used as-is, allowing you to initialize collections with Comparators or other state.

=== Embedding Native Representation ===

*VERY IMPORTANT!*

The native storage format for @Embed structures is changing between Objectify 4.X and 5.X. Objectify 4.1 is a transitional version which can read and write both formats. To avoid surprises, it defaults to writing the old format.

You are _strongly_ encouraged to migrate to the new format now. Objectify 4.1 will automatically detect the correct format when loading entities. To migrate your data, call `ObjectifyFactory.setSaveWithNewEmbedFormat(true)` and load+save every entity which has an @Embed structure. 

Migration to the new storage format _cannot be reverted_. If you migrate and then revert to a previous version of Objectify which does not recognize the new format, *you will lose data*.

Starting with Objectify 5.0, the legacy storage format will be dropped; it is incompatible with many new features (such as embedded object polymorphism). Start this migration now!

Here's the nitty gritty detail of how {{{@Embed}}} objects are persisted so that you an access them through the Low-Level API.

==== New Format For 4.1+ ====

For Objectify 4.1 you must explicitly enable save in the new format by calling `ObjectifyFactory.setSaveWithNewEmbedFormat(true)`. Objectify 5.0 will only support this format.

The new format is very simple. Objects are stored as `EmbeddedEntity`, a new map-like container that was recently added to the Low-Level API. Collections of objects are simply collections of `EmbeddedEntity` objects.

However, GAE does not natively support indexing of `EmbeddedEntity` fields. To allow you to query on embedded data, Objectify creates synthetic index properties in the top-level Entity objects. These are dot-separated property names that have the exact same structure as the old legacy format, so your query code need not change when upgrading to the new format. However, these indexes are only used for queries and are not used to load your POJO objects.

The appearance of your entities in the GAE console Datastore Viewer will change. Unfortunately, the Datastore Viewer currently shows `EmbeddedEntity` objects and collections of `EmbeddedEntity` objects as opaque blobs. Please [https://code.google.com/p/googleappengine/issues/detail?id=10737 star this issue].
 
==== Legacy Format For 4.0 And Previous ====

This is the default format for Objectify 4.X. You are _strongly_ encouraged to migrate to the new format by upgrading to Objectify 4.1, calling `ObjectifyFactory.setSaveWithNewEmbedFormat(true)` and load+saving every entity which has an @Embed structure.

Here is an example of the native storage format, given the example data structures we have discussed already:

{{{
EntityWithEmbedded ent = new EntityWithEmbedded();
ent.one = new LevelOne();
ent.one.foo = "Foo Value";
ent.one.two = new LevelTwo();
ent.one.two.bar = "Bar Value";

ofy().save().entity(ent);
}}}

This will produce an entity that contains:

|| one.foo || "Foo Value" ||
|| one.two.bar || "Bar Value" ||

For collections and arrays of {{{@Embed}}} classes, the storage mechanism is more complicated:

{{{
EntityWithEmbeddedCollection ent = new EntityWithEmbeddedCollection();
for (int i=1; i<=4; i++) {
    LevelOne one = new LevelOne();
    one.foo = "foo" + i;
    one.two = new LevelTwo();
    one.two.bar = "bar" + i;

    ent.ones.add(one);
}

ofy().save().entity(ent);
}}}

This will produce an entity that contains:

|| ones.foo || {{{ ["foo1", "foo2", "foo3", "foo4"] }}} ||
|| ones.two.bar || {{{ ["bar1", "bar2", "bar3", "bar4"] }}} ||

This is what the entity would look like if the second and third values in the {{{ones}}} collection were {{{null}}}:

|| ones.foo^null || {{{ [1, 2] }}} ||
|| ones.foo || {{{ ["foo1", "foo4"] }}} ||
|| ones.two.bar || {{{ ["bar1", "bar4"] }}} ||

The synthetic ^null property only exists if the collection contains nulls.  It is never indexed.

== Serializing ==

An alternative to {{{@Embed}}} is to {{{@Serialize}}}, which will let you store nearly any Java object graph.

{{{
@Entity
class EntityWithSerialized {
    @Id Long id;
    @Serialize Map<Object, Object> stuff;
}
}}}

There are some limitations:

  * All objects stored in the graph must follow Java serialization rules, including implement {{{java.io.Serializable}}}.
  * The total size of an entity cannot exceed 1 megabyte.  If your serialized data exceeds this size, you will get an exception when you try to {{{save()}}} it.
  * You will not be able to use the field or any child fields in queries.
  * As per serializaton rules, {{{transient}}} (the java keyword) fields will not be stored.
  * All Objectify annotations will be ignored within your serialized data structure.  This means {{{@Ignore}}} fields within your serialized structure will be stored!
  * Java serialization data is opaque to the datastore viewer and other languages (ie GAE/Python).  You will only be able to retrieve your data from Java.

However, there are significant benefits to storing data this way:

  * You can store nearly any object graph - nested collections, circular object references, etc.  If Java can serialize it, you can store it.
  * Your field need not be statically typed.  Declare {{{Object}}} if you want.
  * Collections can be stored in their full state; for example, a SortedSet will remember its Comparator implementation.
  * {{{@Serializ}}} collections can be nested inside {{{@Embed}}} collections.

You are *strongly* advised to place {{{serialVersionUID}}} on all classes that you intend to store as {{{@Serialize}}}.  Without this, *any* change to your classes will prevent stored objects from being deserialized on fetch.  Example:

{{{
class SomeStuff implements Serializable {
    /** start with 1 for all classes */
    private static final long serialVersionUID = 1L;

    String foo;
    Object bar;
}

@Entity
class EntityWithSerialized {
    @Id Long id;
    @Serialize SomeStuff stuff;
}
}}}

=== Compression ===

You can automatically zip the serialized blob:

{{{
@Entity
class EntityWithSerialized {
    @Id Long id;
    @Serialize(zip=true) Map<Object, Object> stuff;
}
}}}

You can add or remove zipping at any time.  Irrespective of the {{{zip}}} value, Objectify will read the field correctly whether it was zipped or not in the datastore.  Of course, when the entity is saved, the {{{zip}}} value will determine whether or not the data in that instance is compressed.

== Relationships ==

A relationship is simply a key stored as a field in an entity.  Objectify provides several facilities to make it easier to manage relationships using keys.

=== Keys ===

There are two different Key classes in Objectify:

  # {{{com.google.appengine.api.datastore.Key}}}, usually referred to as {{{Key}}}
  # {{{com.googlecode.objectify.Key}}}, usually referred to as {{{Key<?>}}}

#1 is the native datastore {{{Key}}} class used by the low-level API.  It is the "one true Key" because all datastore operations use this type; Objectify always converts to (or from) the native {{{Key}}}.  However, because this structure lacks generic type information, it should almost never appear in your code.

#2 is Objectify's generified version of {{{Key}}}.  It's much less errorprone to use {{{Key<Person>}}} or {{{Key<Car>}}}, so you should always use {{{Key<?>}}} when you use keys directly.

For all intents and purposes, these structures are equivalent.  For example, these two classes are *identical* in the datastore:

{{{
import com.google.appengine.api.datastore.Key;

@Entity
class Thing {
    @Id Long id;
    Key other;
}
}}}

{{{
import com.googlecode.objectify.Key;

@Entity
class Thing {
    @Id Long id;
    Key<Other> other;
}
}}}

You can change the type of the {{{other}}} field at any time, even after data is stored.  Remember, the datastore contains the native {{{Key}}}.

=== {{{Ref<?>}}}s ===

Even {{{Key<?>}}}s are not very convenient when you are working with graphs of entities.  Objectify provides {{{Ref<?>}}}, which works just like a {{{Key<?>}}} but allows you to directly access the actual entity object as well:

{{{
import com.googlecode.objectify.Ref;

@Entity
class Car {
    @Id Long id;
    Ref<Person> driver;    // Person is an @Entity
}

Car car = new Car();
car.driver = Ref.create(driverKey);
ofy().save().entity(car).now();

Car fetched = ofy().load().entity(car).now();
Person driver = fetched.driver.get();
}}}

{{{Ref<?>}}} fields are stored as native {{{Key}}} fields in the datastore.  You can freely swap between {{{Key}}}, {{{Key<?>}}}, and {{{Ref<?>}}} in your Java data model without modifying stored data.

This provides a nice consistent structure for your object graph, but it's still clunky.  {{{Ref<?>}}} works best in concert with the {{{@Load}}} annotation.

==== {{{@Load}}} ====

In the above example, the {{{fetched.driver.get()}}} caused an immediate fetch from the datastore. This could be a performance problem if you are iterating through lots of Cars loading their drivers; each driver fetch would be a separate request. By using the {{{@Load}}} annotation, you can load drivers into the session efficiently:

{{{
import com.googlecode.objectify.annotation.Load;

@Entity
class Car {
    @Id Long id;
    @Load Ref<Person> driver;    // Person is an @Entity
}

Car car = new Car();
car.driver = Ref.create(driverKey);
ofy().save().entity(car).now();

Car fetched = ofy().load().entity(car).get();
Person driver = fetched.driver.get();
}}}

Objectify will translate this into an _optimal_ number of batch fetches.  For example:

{{{
ofy().load().keys(car1, car2, car3);
}}}

This will produce one round of batch fetching for the cars, plus one round of batch fetching for all the drivers, plus one round of fetching for any {{{@Load}}} fields in all the drivers... and on.  Objectify is also smart about not re-fetching anything that has already been fetched in an earlier cycle.  This optimized behavior is very hard to write by hand.

Often you will not wish to {{{@Load}}} every entity relationship every time.  {{{@Load}}} provides the ability to specify conditional _load groups_.  See [https://code.google.com/p/objectify-appengine/wiki/BasicOperations#Load_Groups Load Groups] for more.

==== Hiding Ref<?> ====

You will generally find it convenient to encapsulate {{{Ref<?>}}} behind getter and setter methods:

{{{
@Entity
class Car {
    @Id Long id;
    @Load Ref<Person> driver;    // Person is an @Entity

    public Person getDriver() { return driver.get(); }
    public void setDriver(Person value) { driver = Ref.create(value); }
}
}}}

==== Additional Ref<?> Considerations ====

{{{Ref<?>}}} is a reference to a key, but does not itself hold an instance to an entity. {{{Ref<?>}}}s link to the current session; calling {{{get()}}} loads the referenced entity out of the session (loading from the datastore if necessary). Think of them as "live" connections to the datastore; if you load a {{{Ref<?>}}}, then {{{ofy().delete()}}} the referenced entity, the next call to {{{Ref<?>.get()}}} will return null.

There is an exception to this: If you serialize a {{{Ref<?>}}} (either with Java serialization or GWT-RPC), it becomes a "dead" holder of the referenced entity.

=== @Parent Relationships ===

Parent relationships are described in [Concepts].  Objectify models them with the {{{@Parent}}} annotation:

{{{
import com.googlecode.objectify.annotation.Entity;
import com.googlecode.objectify.annotation.Id;
import com.googlecode.objectify.annotation.Parent;

@Entity
public class Person {
    @Id Long id;
    String name;
}

@Entity
public class Car {
    @Parent Key<Person> owner;
    @Id Long id;
    String color;
}
}}}

Each Car entity is part of the parent owner's entity group.  Important things to note:

  * There can only be one {{{@Parent}}} field in a class!
  * Along with the {{{@Id}}} and kind, the {{{@Parent}}} field defines the key (identity) of the entity.
    * Construct a key to the car like this:  {{{Key.create(ownerKey, Car.class, carId)}}}
  * The name of the field is irrelevant and can be changed at any time without modifying stored data.
  * The {{{@Parent}}} field can be {{{Key}}}, {{{Key<?>}}}, or {{{Ref<?>}}}.
  * @Load annotations work with {{{Ref<?>}}}s.  As a bonus, the parent will be loaded in the same batch as the initial load.

Note that this example is an inappropriate use of the @Parent entity; if a car were to be sold to a new owner, you would need to delete the Car and create a new one.  It is often better to use simple nonparent key references even when there is a conceptual parent-child or owner-object relationship; in that case you could simply change the owner.

*VERY IMPORTANT*: *If you load() an entity, change the {{{@Parent}}} field, and save() the entity, you will create a new entity*.  The old entity (with the old parent) will still exist.  You cannot simply change the value of a {{{@Parent}}} field.  This is a fundamental aspect of the appengine datastore; {{{@Parent}}} values form part of an entity's identity.

=== One-To-Many Relationships ===

The datastore can natively persist collections of simple types, including keys.  This creates one approach for defining one-to-many (and many-to-many) relationships.

{{{
@Entity
public class Employee {
    @Id String name;
    List<Key<Employee>> subordinates = new ArrayList<Key<Employee>>();
}
}}}

This works with {{{Key}}}s, {{{Key<?>}}}s, and {{{Ref<?>}}}s.  You can use {{{@Load}}} annotations with {{{Ref<?>}}}s.

There are some considerations:

  * Every time you {{{load()}}} or {{{save()}}} an Employee, it will fetch and store the entire list of _subordinates_ keys.  Large numbers of subordinates will bloat the entity and impact performance.
  * App Engine limits you to 5,000 entries in a multi-valued property.
  * Because fetches are performed by key, you have a strongly-consistent view of the a manager and subordinates.

If you put {{{@Index}}} on _subordinates_, you can issue a query like this:

{{{
Iterable<Employee> managers = ofy().load().type(Employee.class).filter("subordinates", fred);
}}}


==== One-To-Many Alternative ====

Storing keys in a collection field is a very "appenginey" way of modeling data, and faithfully works with the key/value nature of the datastore.  However, this is not the traditional way of modeling one-to-many relationships in a relational database.  You can still use the RDBMS approach, but there are caveats:

{{{
@Entity
public class Employee {
    @Id String name;
    @Index Key<Employee> manager;
}

Iterable<Employee> subordinates = ofy().load().type(Employee.class).filter("manager", fred);
}}}

  * This is a query and therefore weakly consistent.
  * You cannot use this query inside of a transaction.
  * Queries are less cache-friendly than direct key references.

The decision of which approach to take will depend heavily on the shape of your data and what queries you need to perform.

= Registering Entities =

Before you use Objectify to load or save data, you must register all entity classes for your application.  Objectify will introspect these classes and their annotations to build a metamodel which is used to efficiently manipulate entities at runtime.

{{{
ObjectifyService.register(Car.class);
ObjectifyService.register(Motorcycle.class);
}}}

  * Registration must be done at application startup, before Objectify is used.
  * Registration must be single-threaded.  Do not register() from multiple threads.
  * All entity classes must be registered, including polymorphic subclasses.
  * {{{@Embed}}} classes do not need to be registered.

We recommend that you register classes in a static initializer for something that is used at application startup.  For example:

{{{
public class MyStartupServlet {
    static {
        ObjectifyService.register(Car.class);
        ObjectifyService.register(Motorcycle.class);
    }

    ...the rest of your servlet
}
}}}

Because this registration process requires loading and introspecting classes, it adds a small delay to your application start time which is directly proportional to the number of classes you register.  While Objectify tries to minimize this delay by eschewing classpath-scanning, the delay is not entirely avoidable (classloading on appengine is very slow).  The best that can be said is that this delay is significantly shorter than the equivalent JDO/JPA delay.