org.tensorflow.ndarray.buffer.layout

Interface DataLayout<S extends DataBuffer<?>,T>

Type Parameters:

S - type of buffer this layout can be applied to

T - user data type of this layout

All Known Subinterfaces:

BooleanDataLayout<S>, ByteDataLayout<S>, DoubleDataLayout<S>, FloatDataLayout<S>, IntDataLayout<S>, LongDataLayout<S>, ShortDataLayout<S>

All Known Implementing Classes:

Bfloat16Layout, BoolLayout, Float16Layout, StringLayout

public interface DataLayout<S extends DataBuffer<?>,T>

Converts data stored in a buffer to a given type.

DataLayout instances are used to define a custom format for storing and reading data of a DataBuffer. They provide a segregation layer between the type of data stored in the buffer (the buffer type) and the type of data manipulated by the end user (the user type).

Since the conversion methods are invoked for every value that is written or read, working with data layouts may have a negative impact on the performances so using primitive types directly should be preferred whenever possible.

It is also recommended to implement immutable data layouts so they can be reapplied to multiple buffers without reallocating a new instance for each of them. For example:

 class BigIntegerBufferAllocator {

     public DataBuffer<BigInteger> allocate(long size) {
         return LAYOUT.applyTo(DataBuffers.ofLongs(size * LAYOUT.scale()));  // scale is 1 by default
     }

     private static final DataLayout<LongDataBuffer, BigInteger> LAYOUT = new DataLayout<LongDataBuffer, BigInteger>() {

         @Override
         public void writeObject(LongDataBuffer buffer, BigInteger value, long index) {
             buffer.setLong(value.longValue(), index);
         }

         @Override
         public BigInteger readObject(LongDataBuffer buffer, long index) {
             return BigInteger.valueOf(buffer.getLong(index));
         }
     };
 }
 

Method Summary

Modifier and Type	Method and Description
default DataBuffer<T>	applyTo(S buffer) Apply this layout to the provided buffer.
T	readObject(S buffer, long index) Reads n = scale() values from the buffer at the given index and return them as a single value in the user type.
default int	scale() Indicates the number of buffer values are required to represent a single user value, default is 1.
void	writeObject(S buffer, T value, long index) Writes a user value into the buffer at the given index after converting it to the buffer type.

Method Detail

applyTo

default DataBuffer<T> applyTo(S buffer)

Apply this layout to the provided buffer.

The returned DataBuffer instance is simply a wrapper to the original buffer and does not have a backing storage of his own.

Parameters:

buffer - the target buffer to apply this layout to

Returns:

a buffer with this layout

writeObject

void writeObject(S buffer,
                 T value,
                 long index)

Writes a user value into the buffer at the given index after converting it to the buffer type.

It is the responsibility of the implementors of this interface to write the converted value to the given buffer before this call returns, using the most appropriate method. For example, for a layout converting a BigInteger to a single long,

 @Override
 public void writeObject(LongDataBuffer buffer, BigInteger value, long index) {
   buffer.setLong(value.longValue(), index);
 }
 

If a single user value scales over more than one buffer values, index indicates the starting position of the sequence to be written to the buffer.

Parameters:

buffer - the buffer to write to

value - the value in the user type to convert and write

index - index in the buffer where the converted value should be written

readObject

T readObject(S buffer,
             long index)

Reads n = scale() values from the buffer at the given index and return them as a single value in the user type.

It is the responsibility of the implementors of this interface to read the value to be converted from the given buffer, using the most appropriate method. For example, for a layout that converting a single long to a BigInteger,

 @Override
 public BigInteger readObject(LongDataBuffer buffer, long index) {
   return BigInteger.valueOf(buffer.getLong(index));
 }
 

If a single user value scales over more than one buffer values, index indicates the starting position of the sequence to be read from the buffer.

Parameters:

buffer - the buffer to read from

index - position of the buffer to read in the buffer

Returns:

the converted value

scale

default int scale()

Indicates the number of buffer values are required to represent a single user value, default is 1.

Scale must be positive and must be an integer, meaning that a single buffer value in a buffer cannot be used to represent more than one user value.