public final class DataLayouts extends Object
DataLayout instances of data formats frequently used in linear algebra computation.
Example of usage:
// Storing boolean values in a ByteDataBuffer
BooleanDataBuffer boolBuffer = DataLayouts.BOOL.applyTo(byteDataBuffer);
// Allocating a new buffer of 256 half floats
FloatDataBuffer halfBuffer = DataLayouts.FLOAT16.applyTo(DataBuffers.ofShorts(256 * DataLayouts.FLOAT16.scale());
| Modifier and Type | Field and Description |
|---|---|
static FloatDataLayout<ShortDataBuffer> |
BFLOAT16
Data layout for converting 16-bit bfloats to/from short values.
|
static BooleanDataLayout<ByteDataBuffer> |
BOOL
Data layout for converting booleans to/from byte values.
|
static FloatDataLayout<ShortDataBuffer> |
FLOAT16
Data layout for converting 16-bit half floats to/from short values.
|
| Constructor and Description |
|---|
DataLayouts() |
| Modifier and Type | Method and Description |
|---|---|
static DataLayout<DataBuffer<byte[]>,String> |
ofStrings(Charset charset)
Creates a data layout for converting strings to/from byte sequences.
|
public static final FloatDataLayout<ShortDataBuffer> BFLOAT16
This format used to be specific to TensorFlow but has now been adopted more broadly in the machine learning field. It is optimized for fast conversion with single-precision 32-bit floating points by simply shifting their value and truncating the mantissa to only 7 bits.
Therefore, this is a lost of precision in the fraction part compared to the IEEE-754
half-precision floating point specification (see FLOAT16 but it has a larger range of
possible values in the whole part as it preserves the 8-bit exponent and uses the same bias,
(i.e. an absolute range above 0 of approximately [10-40, 3.39 × 1038]
Some CPUs support the bfloat16 format natively for better performances.
public static final FloatDataLayout<ShortDataBuffer> FLOAT16
Half floats are stored in memory accordingly to the IEEE-754 half-precision floating point specification, and are converted to/from 32-bit floats in the user space.
There is a potential loss of precision when converting a single float (32-bit) to a half float (16-bit). Absolute range of values above 0 for a half float is approximately [5.96 × 10-8, 6.55 × 104] and their decimal part is rounded up to a 10 bits mantissa.
In general, half float computation perform better on GPUs since, in general, CPUs do not support this format natively.
public static final BooleanDataLayout<ByteDataBuffer> BOOL
Since there is no Java NIO boolean buffer, this layout is particularly useful for mapping booleans values to standard byte buffers. The conversion between a boolean and a byte requires explicit type casting.
public static DataLayout<DataBuffer<byte[]>,String> ofStrings(Charset charset)
This layout requires a charset in parameter to specify how the strings must be
encoded/decoded as byte sequences. So a new layout instance is always returned.
charset - charset to useCopyright © 2015–2020. All rights reserved.