Package io.delta.kernel.internal.util

Class SchemaIterable

Object
io.delta.kernel.internal.util.SchemaIterable
All Implemented Interfaces:
Iterable<SchemaIterable.SchemaElement>
public class SchemaIterable extends Object implements Iterable<SchemaIterable.SchemaElement>
Utility class for iterating over schema structures and modifying them.

Sample usage for iterating schemas: 


 StructType schema = ...;
 for (SchemaIterable.SchemaElement element : new SchemaIterable(schema)) {
    StructField field = element.getField();
    // Get info from field in some way.
 }

Sample usage for mutating schemas: 


 StructType schema = ...;
 SchemaIterable schemaIterable = new SchemaIterable(schema);
 Iterator<SchemaIterable.MutableSchemaElement> iterator = schemaIterable.newMutableIterator();
 while (iterator.hasNext()) {
    SchemaIterable.MutableSchemaElement element = iterator.next();
    // Calculate a new field in some way then call updateField.
    element.updateField(...)
 }
 updatedSchema = schemaIterable.getSchema();

  • Constructor Details

    • SchemaIterable

      public SchemaIterable(StructType schema)
      Construct a new Iterable for the schema.

  • Method Details

    • newSchemaIterableWithIgnoredRecursion

      public static SchemaIterable newSchemaIterableWithIgnoredRecursion(StructType schema, Class<?>[] typesToSkipRecursion)
      Constructs a new iterable that skips recursion for some data types.

      No recursion will be done on any type that is an instance of a class in typesToSkipRecursion.

      For example with schema: 

      
 a: Map<String, Int>
 b: Array<Int>

      If typesToSkipRecursion = {MapType.class} is provided then the iterator will only visit "a", "b.element", and "b".

      If typesToSkipRecursion = {ArrayType.class} is provided then the iterator wil only visit "a.key", "a.value", "a" and "b".

    • getSchema

      public StructType getSchema()
      Gets the latest schema (either the initial schema or the one set after a mutable iterator is fully consumed).

    • iterator

      public Iterator<SchemaIterable.SchemaElement> iterator()
      Specified by:
      iterator in interface Iterable<SchemaIterable.SchemaElement>

    • stream

      public Stream<SchemaIterable.SchemaElement> stream()

    • mutableStream

      public Stream<SchemaIterable.MutableSchemaElement> mutableStream()

    • newMutableIterator

      public Iterator<SchemaIterable.MutableSchemaElement> newMutableIterator()
      Returns a new iterator that can be used to iterate and update elements in the schema. The current schema on this iterable is updated once the returned iterator is fully consumed.

      Consuming multiple iterators across different threads concurrently is not thread safe.

      Example usage: 

      
 Iterator<SchemaIterable.MutableSchemaElement> iterator = schemaIterable.newMutableIterator();
 while (iterator.hasNext()) {
    SchemaIterable.MutableSchemaElement element = iterator.next();
    element.updateField(...)
 }
 updatedSchema = schemaIterable.getSchema();