Primitives provided by ObjLayer are broadly classified into creation, modification, read access, validation and destruction. Informal descriptions of each, with sample C code follow.
An object space is created by calling the C function:
void ObjLayer_spaceInit ( int numProc );where processors numbered 0 through numProc - 1 participate in the global operations performed by ObjLayer. Currently only one object space is supported, but this is not an inherent feature. Nodes have to synchronize after this call to make sure ObjLayer's internal data structures are initialized on all participating nodes before operations begin.
ObjLayer_spaceInit ( numActiveProcs );
An object is created by allocating memory, filling it up with (the first version of) data, and registering it into the object space. The function
int ObjLayer_createInit ( int size, void *data, Id_t idOut );initiates the creation of an object size bytes large, pointed to by data, and returns a unique ID in idOut. The function
int ObjLayer_createTest ( Id_t IdIn );checks to see if creation is complete, returns OBJ_TEST_Y if so, and OBJ_TEST_N otherwise.
Id_t newId; ObjLayer_createInit ( size, data, newId ); while ( ObjLayer_createTest ( newId ) != OBJ_TEST_Y ) MultipolPoll(); /* insert useful work */
An object, identified by idIn, is modified by calling
ObjStatus_t ObjLayer_modifyInit ( Id_t idIn, int newSize, void *newData );where newSize is the number of bytes in the new version of the object's value, pointed to by newData. The old buffer is eventually garbage collected by the system. Completion of the modify operation is tested by
ObjStatus_t ObjLayer_modifyTest ( Id_t idIn );returning OBJ_TEST_Y or OBJ_TEST_N. Overlapped modifications to the same object by different processors is an error, and the application using ObjLayer is responsible for achieving such excluded access.
if ( ObjLayer_modifyInit ( oldId, newSize, newData ) != OBJ_MAPPED ) HandleError ( oldId ); while ( ObjLayer_modifyTest ( oldId ) != OBJ_TEST_Y ) MultipolPoll(); /* insert useful work */Waiting only for OBJ_TEST_Y is not recommended; there could be recoverable error conditions which will be ignored and the program will hang at the while loop.
After a processor gets exclusive ownership of an object, it can initiate the destruction of the object, and test for completion, using the functions
int ObjLayer_destroyInit ( Id_t oldId ); int ObjLayer_destroyTest ( Id_t oldId );Usage is identical to the modification primitives. After ObjLayer_destroyTest succeeds, all processors will get return status OBJ_DELETED when they try to call any ObjLayer function on oldId. This is a stable property. Note that this implies a permanent record of the deletion; thus the object space is not recycled.
Reading an object is a two phase process. First the caller acquires a handle to the local version (if any). This also notifies ObjLayer that if any other versions arrive, they will have to pile up; this version cannot be discarded until the reading transaction completes. Next the application can access the data buffer. A write is not prevented, but is an error according to the consistency semantics. Finally, when the application is done, it releases the handle. The functions are listed below.
ObjStatus_t ObjLayer_readAcquire ( Id_t idIn, Wrap_t **handleOut ); void *ObjLayer_mapData ( Wrap_t *handleIn ); void ObjLayer_readRelease ( Wrap_t *handleIn );In principle, the ObjLayer_mapData step is superfluous. The Wrap_t structure keeps bookkeeping information about the attached version, but need not have been visible to the user. This is only an implementation idiosyncracy.
int status;
Wrap_t *w;
UserData_t *p;
if ( (status = ObjLayer_readAcquire(oldId,&w)) == OBJ_MAPPED ) {
p = ObjLayer_mapData ( w );
AccessUserData ( p );
ObjLayer_readRelease ( w );
}
else
HandleError ( oldId, status );
The usual reasons why the ObjLayer_readAcquire may not succeed is that
either the object has been deleted, or no version of it has yet been cached.
HandleError should deal with these conditions.
When an object is read on a processor, any version cached in local memory, even if outdated, may be returned. To ensure that the value a processor reads is the most recent one requires a validation protocol at the application level. The function
ObjStatus_t ObjLayer_validTest ( Id_t idIn )returns OBJ_TEST_Y if at the time of calling, the calling processor knows of no later version of the object than is already cached. Note that there is no critical region being defined here: immediately after ObjValidTest returns successfully, another processor may destroy this property. If, for a processor to continue, the property must be preserved, a lock must be associated with the object at the application level so that by the time the test succeeds, no processor can modify or delete the object.
The second primitive for consistency control is to upgrade the version of an object, or to get a version if none currently exists.
ObjStatus_t ObjLayer_validInit ( Id_t oldId );The call returns without waiting for the validation to complete. As mentioned earlier, validation continues asynchronously in the background with ongoing computation of the main thread of control, and can complete at an arbitrary later point in time.
Validation, once initiated by a processor, proceeds asynchronously in the background, and can complete at an arbitrary later point in time. A hook is provided that is fired when validation of an object completes: the user writes a function ObjLayer_validHook_h which is called. This function is executed without interruptions from the network. ObjLayer_validHook_h must not access the network, must access only data local to the calling processor, and run to completion without waiting for any event.
extern void ObjLayer_validHook_h ( Id_t id, void *sendBase, void *recvBase )The hook is called with the ID of the object whose validation has just completed, the start offset of the object on the source processor's local memory and the start offset of the copy made on the receiving (calling) processor. The most usual use of this hook is to relocate objects by modifying local pointer fields in the destination buffer. Also, the ID may be entered into task bags for further processing.
The application needs to allocate a block of memory when creating an object. This must be done using the function OBJ_MALLOC, which has the same semantics as malloc. OBJ_FREE also exists, but the application never needs to use it explicitly. Garbage collection is managed internally based on funstion calls to modify, validate, and delete objects.
The set of return status returned by the ObjLayer operations are as follows.
OBJ_NOMAP: The calling processor has no knowledge of the existence of
the argument object ID.
OBJ_MAPPED: The object is known to exist by the calling processor, but
the value may not be the most recent one.
OBJ_ABSENT: The object is known, but has never been locally cached in
the calling processor.
OBJ_DELETED: The object is known, but has been deleted by the last
owner. Detection of deletion is guaranteed within finite but arbitrary time
from the actual action of deletion on the last owner.
OBJ_INVALID: The object is known, but the version is out of date.
OBJ_ACKWAIT: There is a pending create, modify or destroy operation to
be completed on the argument object.
OBJ_TEST_Y: Success status to all ObjLayer test primitives.
OBJ_TEST_N: Failure status to all ObjLayer test primitives.