io.delta.kernel.defaults.engine

Class DefaultJsonHandler

Object

io.delta.kernel.defaults.engine.DefaultJsonHandler

All Implemented Interfaces:

io.delta.kernel.engine.JsonHandler

public class DefaultJsonHandler
extends Object
implements io.delta.kernel.engine.JsonHandler

Default implementation of JsonHandler based on Hadoop APIs.

Constructor Summary

Constructors

Constructor and Description
DefaultJsonHandler(org.apache.hadoop.conf.Configuration hadoopConf)

Method Summary

Modifier and Type	Method and Description
io.delta.kernel.types.StructType	deserializeStructType(String structTypeJson)
io.delta.kernel.data.ColumnarBatch	parseJson(io.delta.kernel.data.ColumnVector jsonStringVector, io.delta.kernel.types.StructType outputSchema, java.util.Optional<io.delta.kernel.data.ColumnVector> selectionVector)
io.delta.kernel.utils.CloseableIterator<io.delta.kernel.data.ColumnarBatch>	readJsonFiles(io.delta.kernel.utils.CloseableIterator<io.delta.kernel.utils.FileStatus> scanFileIter, io.delta.kernel.types.StructType physicalSchema, java.util.Optional<io.delta.kernel.expressions.Predicate> predicate)
void	writeJsonFileAtomically(String filePath, io.delta.kernel.utils.CloseableIterator<io.delta.kernel.data.Row> data, boolean overwrite) Makes use of LogStore implementations in `delta-storage` to atomically write the data to a file depending upon the destination filesystem.

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DefaultJsonHandler

public DefaultJsonHandler(org.apache.hadoop.conf.Configuration hadoopConf)

Method Detail

parseJson

public io.delta.kernel.data.ColumnarBatch parseJson(io.delta.kernel.data.ColumnVector jsonStringVector,
                                                    io.delta.kernel.types.StructType outputSchema,
                                                    java.util.Optional<io.delta.kernel.data.ColumnVector> selectionVector)

Specified by:

parseJson in interface io.delta.kernel.engine.JsonHandler

deserializeStructType

public io.delta.kernel.types.StructType deserializeStructType(String structTypeJson)

Specified by:

deserializeStructType in interface io.delta.kernel.engine.JsonHandler

readJsonFiles

public io.delta.kernel.utils.CloseableIterator<io.delta.kernel.data.ColumnarBatch> readJsonFiles(io.delta.kernel.utils.CloseableIterator<io.delta.kernel.utils.FileStatus> scanFileIter,
                                                                                                 io.delta.kernel.types.StructType physicalSchema,
                                                                                                 java.util.Optional<io.delta.kernel.expressions.Predicate> predicate)
                                                                                          throws java.io.IOException

Specified by:

readJsonFiles in interface io.delta.kernel.engine.JsonHandler

Throws:

java.io.IOException

writeJsonFileAtomically

public void writeJsonFileAtomically(String filePath,
                                    io.delta.kernel.utils.CloseableIterator<io.delta.kernel.data.Row> data,
                                    boolean overwrite)
                             throws java.io.IOException

Makes use of LogStore implementations in `delta-storage` to atomically write the data to a file depending upon the destination filesystem.

Specified by:

writeJsonFileAtomically in interface io.delta.kernel.engine.JsonHandler

Parameters:

filePath - Destination file path

data - Data to write as Json

Throws:

java.io.IOException