io.delta.standalone.util

Class ParquetSchemaConverter

Object

io.delta.standalone.util.ParquetSchemaConverter

public final class ParquetSchemaConverter
extends Object

:: DeveloperApi ::

Converter class to convert StructType to Parquet MessageType.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	ParquetSchemaConverter.ParquetOutputTimestampType :: DeveloperApi ::

Field Summary

Fields

Modifier and Type	Field and Description
static ParquetSchemaConverter.ParquetOutputTimestampType	outputTimestampTypeDefault
static Boolean	writeLegacyParquetFormatDefault

Method Summary

Modifier and Type	Method and Description
static org.apache.parquet.schema.MessageType	deltaToParquet(StructType schema) :: DeveloperApi ::
static org.apache.parquet.schema.MessageType	deltaToParquet(StructType schema, Boolean writeLegacyParquetFormat) :: DeveloperApi ::
static org.apache.parquet.schema.MessageType	deltaToParquet(StructType schema, Boolean writeLegacyParquetFormat, ParquetSchemaConverter.ParquetOutputTimestampType outputTimestampType) :: DeveloperApi ::
static org.apache.parquet.schema.MessageType	deltaToParquet(StructType schema, ParquetSchemaConverter.ParquetOutputTimestampType outputTimestampType) :: DeveloperApi ::

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

writeLegacyParquetFormatDefault

public static final Boolean writeLegacyParquetFormatDefault

outputTimestampTypeDefault

public static final ParquetSchemaConverter.ParquetOutputTimestampType outputTimestampTypeDefault

Method Detail

deltaToParquet

public static org.apache.parquet.schema.MessageType deltaToParquet(StructType schema)

:: DeveloperApi ::

Convert a StructType to Parquet MessageType.

Parameters:

schema - the schema to convert

Returns:

schema as a Parquet MessageType

Throws:

IllegalArgumentException - if a StructField name contains invalid character(s)

deltaToParquet

public static org.apache.parquet.schema.MessageType deltaToParquet(StructType schema,
                                                                   Boolean writeLegacyParquetFormat)

:: DeveloperApi ::

Convert a StructType to Parquet MessageType.

Parameters:

schema - the schema to convert

writeLegacyParquetFormat - Whether to use legacy Parquet format compatible with Spark 1.4 and prior versions when converting a StructType to a Parquet MessageType. When set to false, use standard format defined in parquet-format spec.

Returns:

schema as a Parquet MessageType

Throws:

IllegalArgumentException - if a StructField name contains invalid character(s)

deltaToParquet

public static org.apache.parquet.schema.MessageType deltaToParquet(StructType schema,
                                                                   ParquetSchemaConverter.ParquetOutputTimestampType outputTimestampType)

:: DeveloperApi ::

Convert a StructType to Parquet MessageType.

Parameters:

schema - the schema to convert

outputTimestampType - which parquet timestamp type to use when writing

Returns:

schema as a Parquet MessageType

Throws:

IllegalArgumentException - if a StructField name contains invalid character(s)

deltaToParquet

public static org.apache.parquet.schema.MessageType deltaToParquet(StructType schema,
                                                                   Boolean writeLegacyParquetFormat,
                                                                   ParquetSchemaConverter.ParquetOutputTimestampType outputTimestampType)

:: DeveloperApi ::

Convert a StructType to Parquet MessageType.

Parameters:

schema - the schema to convert

writeLegacyParquetFormat - Whether to use legacy Parquet format compatible with Spark 1.4 and prior versions when converting a StructType to a Parquet MessageType. When set to false, use standard format defined in parquet-format spec.

outputTimestampType - which parquet timestamp type to use when writing

Returns:

schema as a Parquet MessageType

Throws:

IllegalArgumentException - if a StructField name contains invalid character(s)