Custom-Partition Your Region Data
By default, Geode partitions each data entry into a bucket using a hashing policy on the key. Additionally, the physical location of the key-value pair is abstracted away from the application. You can change these policies for a partitioned region. You can provide your own data partitioning resolver and you can additionally specify which members host which data buckets.
Note: If you are colocating data between regions and custom partitioning the data in the regions, all colocated regions must use the same custom partitioning mechanism. See Colocate Data from Different Partitioned Regions.
For the default implementation of string-based partitioning, use org.apache.geode.cache.util.StringPrefixPartitionResolver.
For standard partitioning, use org.apache.geode.cache.PartitionResolver. To implement fixed partitioning, use org.apache.geode.cache.FixedPartitionResolver.
Prerequisites
- Create partitioned regions. See Understanding Partitioning and Configuring Partitioned Regions.
- Decide whether to use the default implementation of the string-based partitioning, standard custom partitioning, or fixed custom partitioning. See Understanding Custom Partitioning and Data Colocation.
- If you also want to colocate data from multiple regions, understand how to colocate. See Colocate Data from Different Partitioned Regions.
Procedure
If using
org.apache.geode.cache.PartitionResolver(standard partitioning) ororg.apache.geode.cache.FixedPartitionResolver(fixed partitioning), implement the standard partitioning resolver or the fixed partitioning resolver in one of the following locations, listed here in the search order used by Geode:- Custom class. You provide this class as the partition resolver to the region creation.
- Entry key. You use the implementing key object for every operation on the region entries.
- Cache callback argument. This implementation restricts you to using methods that accept a cache callback argument to manage the region entries. For a full list of the methods that take a callback argument, see the
RegionJavadocs.
If using the default implementation of the string-based partition resolver,
org.apache.geode.cache.util.StringPrefixPartitionResolver, specify all keys with the required syntax. Keys are strings, and contain the '|' character as a delimiter. The substring that precedes the '|' delimiter will used in the hash function that partitions the entry.- If you need the resolver's
getNamemethod, program that. If not using the default implementation of the string-based partition resolver, program the resolver's
getRoutingObjectmethod to return the routing object for each entry, based on how you want to group the entries. Give the same routing object to entries you want to group together. Geode will place the entries in the same bucket.Note: Only fields on the key should be used when creating the routing object. Do not use the value or additional metadata for this purpose.
For example, here is an implementation on a region key object that groups the entries by month and year:
Public class TradeKey implements PartitionResolver { private String tradeID; private Month month; private Year year; public TradingKey(){ } public TradingKey(Month month, Year year) { this.month = month; this.year = year; } public Serializable getRoutingObject(EntryOperation opDetails) { return this.month + this.year; } }For fixed partitioning only, program and configure additional fixed partitioning pieces:
Set the fixed partition attributes for each member.
These attributes define the data stored for the region by the member and must be different for different members. See
org.apache.geode.cache.FixedPartitionAttributesfor definitions of the attributes. Define eachpartition-namein your data host members for the region. For each partition name, in the member you want to host the primary copy, define it withis-primaryset totrue. In every member you want to host the secondary copy, define it withis-primaryset tofalse(the default). The number of secondaries must match the number of redundant copies you have defined for the region. See Configure High Availability for a Partitioned Region.Note: Buckets for a partition are hosted only by the members that have defined the partition name in their
FixedPartitionAttributes.These examples set the partition attributes for a member to be the primary host for the "Q1" partition data and a secondary host for "Q3" partition data.
XML:
<cache> <region name="Trades"> <region-attributes> <partition-attributes redundant-copies="1"> <partition-resolver name="QuarterFixedPartitionResolver"> <class-name>myPackage.QuarterFixedPartitionResolver</class-name> </partition-resolver> <fixed-partition-attributes partition-name="Q1" is-primary="true"/> <fixed-partition-attributes partition-name="Q3" is-primary="false" num-buckets="6"/> </partition-attributes> </region-attributes> </region> </cache>- Java:
``` pre
FixedPartitionAttribute fpa1 = FixedPartitionAttributes.createFixedPartition("Q1", true);
FixedPartitionAttribute fpa3 = FixedPartitionAttributes.createFixedPartition("Q3", false, 6);
PartitionAttributesFactory paf = new PartitionAttributesFactory()
.setPartitionResolver(new QuarterFixedPartitionResolver())
.setTotalNumBuckets(12)
.setRedundantCopies(2)
.addFixedPartitionAttribute(fpa1)
.addFixedPartitionAttribute(fpa3);
Cache c = new CacheFactory().create();
Region r = c.createRegionFactory()
.setPartitionAttributes(paf.create())
.create("Trades");
```
- gfsh:
You cannot specify a partition resolver using gfsh.
2. Program the `FixedPartitionResolver` `getPartitionName` method to return the name of the partition for each entry, based on where you want the entries to reside. Geode uses `getPartitionName` and `getRoutingObject` to determine where an entry is placed.
**Note:**
To group entries, assign every entry in the group the same routing object and the same partition name.
This example places the data based on date, with a different partition name for each quarter-year and a different routing object for each month.
``` pre
/**
* Returns one of four different partition names
* (Q1, Q2, Q3, Q4) depending on the entry's date
*/
class QuarterFixedPartitionResolver implements
FixedPartitionResolver<String, String> {
@Override
public String getPartitionName(EntryOperation<String, String> opDetails,
Set<String> targetPartitions) {
Date date = (Date)opDetails.getKey();
Calendar cal = Calendar.getInstance();
cal.setTime(date);
int month = cal.get(Calendar.MONTH);
if (month >= 0 && month < 3) {
if (targetPartitions.contains("Q1")) return "Q1";
}
else if (month >= 3 && month < 6) {
if (targetPartitions.contains("Q2")) return "Q2";
}
else if (month >= 6 && month < 9) {
if (targetPartitions.contains("Q3")) return "Q3";
}
else if (month >= 9 && month < 12) {
if (targetPartitions.contains("Q4")) return "Q4";
}
return "Invalid Quarter";
}
@Override
public String getName() {
return "QuarterFixedPartitionResolver";
}
@Override
public Serializable getRoutingObject(EntryOperation<String, String> opDetails) {
Date date = (Date)opDetails.getKey();
Calendar cal = Calendar.getInstance();
cal.setTime(date);
int month = cal.get(Calendar.MONTH);
return month;
}
@Override
public void close() {
}
}
```
Configure or program the region so Geode finds your resolver for every operation that you perform on the region's entries. How you do this depends on where you chose to program your custom partitioning implementation (step 1).
Custom class. Define the class for the region at creation. The resolver will be used for every entry operation. Use one of these methods:
XML:
<region name="trades"> <region-attributes> <partition-attributes> <partition-resolver name="TradesPartitionResolver"> <class-name>myPackage.TradesPartitionResolver </class-name> </partition-resolver> <partition-attributes> </region-attributes> </region>Java API:
``` pre
PartitionResolver resolver = new TradesPartitionResolver();
PartitionAttributes attrs =
new PartitionAttributesFactory()
.setPartitionResolver(resolver).create();
Cache c = new CacheFactory().create();
Region r = c.createRegionFactory()
.setPartitionAttributes(attrs)
.create("trades");
```
**gfsh:**
Add the option `--partition-resolver` to the `gfsh create region` command, specifying the package and class name of the custom partition resolver.
- **Entry key**. Use the key object with the resolver implementation for every entry operation.
- **Cache callback argument**. Provide the argument to every call that accesses an entry. This restricts you to calls that take a callback argument.
If using the default implementation of the string-based partition resolver, configure with one of:
**XML:**
``` pre
<region name="customers">
<region-attributes>
<partition-attributes>
<partition-resolver name="StringPrefixPartitionResolver">
<class-name>org.apache.geode.cache.util.StringPrefixPartitionResolver
</class-name>
</partition-resolver>
<partition-attributes>
</region-attributes>
</region>
```
**Java API:**
``` pre
PartitionAttributes attrs =
new PartitionAttributesFactory()
.setPartitionResolver("StringPrefixPartitionResolver").create();
Cache c = new CacheFactory().create();
Region r = c.createRegionFactory()
.setPartitionAttributes(attrs)
.create("customers");
```
**gfsh:**
Add the option `--partition-resolver=org.apache.geode.cache.util.StringPrefixPartitionResolver` to the `gfsh create region` command.
- If your colocated data is in a server system, add the
PartitionResolverimplementation class to theCLASSPATHof your Java clients. The resolver is used for single hop access to partitioned region data in the servers.