DeltaLog (Delta Standalone 0.6.0 JavaDoc)

public interface DeltaLog

Represents the transaction logs of a Delta table. It provides APIs to access the states of a Delta table.

You can use the following code to create a DeltaLog instance.


   Configuration conf = ... // Create your own Hadoop Configuration instance
   DeltaLog deltaLog = DeltaLog.forTable(conf, "/the/delta/table/path");

Method Summary

All Methods Static Methods Instance Methods Abstract Methods Default Methods
Modifier and Type	Method and Description
`static DeltaLog`	`forTable(org.apache.hadoop.conf.Configuration hadoopConf, org.apache.hadoop.fs.Path path)` Create a `DeltaLog` instance representing the table located at the provided `path`.
`static DeltaLog`	`forTable(org.apache.hadoop.conf.Configuration hadoopConf, String path)` Create a `DeltaLog` instance representing the table located at the provided `path`.
`java.util.Iterator<VersionLog>`	`getChanges(long startVersion, boolean failOnDataLoss)` Get all actions starting from `startVersion` (inclusive) in increasing order of committed version.
`CommitInfo`	`getCommitInfoAt(long version)`
`org.apache.hadoop.fs.Path`	`getPath()`
`Snapshot`	`getSnapshotForTimestampAsOf(long timestamp)` Travel back in time to the latest `Snapshot` that was generated at or before `timestamp`.
`Snapshot`	`getSnapshotForVersionAsOf(long version)` Travel back in time to the `Snapshot` with the provided `version` number.
`long`	`getVersionAtOrAfterTimestamp(long timestamp)` Returns the latest version that was committed at or after `timestamp`.
`long`	`getVersionBeforeOrAtTimestamp(long timestamp)` Returns the latest version that was committed before or at `timestamp`.
`Snapshot`	`snapshot()`
`OptimisticTransaction`	`startTransaction()` Returns a new `OptimisticTransaction` that can be used to read the current state of the log and then commit updates.
`boolean`	`tableExists()`
`Snapshot`	`update()` Bring `DeltaLog`'s current `Snapshot` to the latest state if there are any new transaction logs.

- Method Detail
  - snapshot
```
Snapshot snapshot()
```
    Returns:
    
    the current Snapshot of the Delta table. You may need to call update() to access the latest snapshot if the current snapshot is stale.
  - update
```
Snapshot update()
```
    Bring DeltaLog's current Snapshot to the latest state if there are any new transaction logs.
    
    Returns:
    
    the latest snapshot after applying the new transaction logs.
  - getSnapshotForVersionAsOf
```
Snapshot getSnapshotForVersionAsOf(long version)
```
    Travel back in time to the Snapshot with the provided version number.
    
    Parameters:
    
    version - the snapshot version to generate
    
    Returns:
    
    the snapshot at the provided version
    
    Throws:
    
    IllegalArgumentException - if the version is outside the range of available versions
  - getSnapshotForTimestampAsOf
```
Snapshot getSnapshotForTimestampAsOf(long timestamp)
```
    Travel back in time to the latest Snapshot that was generated at or before timestamp.
    
    Parameters:
    
    timestamp - the number of milliseconds since midnight, January 1, 1970 UTC
    
    Returns:
    
    the snapshot nearest to, but not after, the provided timestamp
    
    Throws:
    
    RuntimeException - if the snapshot is unable to be recreated
    
    IllegalArgumentException - if the timestamp is before the earliest possible snapshot or after the latest possible snapshot
  - startTransaction
```
OptimisticTransaction startTransaction()
```
    Returns a new OptimisticTransaction that can be used to read the current state of the log and then commit updates. The reads and updates will be checked for logical conflicts with any concurrent writes to the log.
    Note that all reads in a transaction must go through the returned transaction object, and not directly to the DeltaLog otherwise they will not be checked for conflicts.
    
    Returns:
    
    a new OptimisticTransaction.
  - getCommitInfoAt
```
CommitInfo getCommitInfoAt(long version)
```
    Parameters:
    
    version - the commit version to retrieve CommitInfo
    
    Returns:
    
    the CommitInfo of the commit at the provided version.
  - getPath
```
org.apache.hadoop.fs.Path getPath()
```
    Returns:
    
    the path of the Delta table.
  - getChanges
```
java.util.Iterator<VersionLog> getChanges(long startVersion,
                                          boolean failOnDataLoss)
```
    Get all actions starting from startVersion (inclusive) in increasing order of committed version.
    If startVersion doesn't exist, return an empty Iterator.
    
    Parameters:
    
    startVersion - the table version to begin retrieving actions from (inclusive)
    
    failOnDataLoss - whether to throw when data loss detected
    
    Returns:
    
    an Iterator of VersionLogs starting from startVersion
    
    Throws:
    
    IllegalArgumentException - if startVersion is negative
    
    IllegalStateException - if data loss detected and failOnDataLoss is true
  - getVersionBeforeOrAtTimestamp
```
long getVersionBeforeOrAtTimestamp(long timestamp)
```
    Returns the latest version that was committed before or at timestamp. If no version exists, returns -1. Specifically:
    - if a commit version exactly matches the provided timestamp, we return it
    - else, we return the latest commit version with a timestamp less than the provided one
    - If the provided timestamp is less than the timestamp of any committed version, we throw an error.
    .
    Parameters:
    
    timestamp - the number of milliseconds since midnight, January 1, 1970 UTC
    
    Returns:
    
    latest commit that happened before or at timestamp.
    
    Throws:
    
    IllegalArgumentException - if the timestamp is less than the timestamp of any committed version
  - getVersionAtOrAfterTimestamp
```
long getVersionAtOrAfterTimestamp(long timestamp)
```
    Returns the latest version that was committed at or after timestamp. If no version exists, returns -1. Specifically:
    - if a commit version exactly matches the provided timestamp, we return it
    - else, we return the earliest commit version with a timestamp greater than the provided one
    - If the provided timestamp is larger than the timestamp of any committed version, we throw an error.
    .
    Parameters:
    
    timestamp - the number of milliseconds since midnight, January 1, 1970 UTC
    
    Returns:
    
    latest commit that happened at or before timestamp.
    
    Throws:
    
    IllegalArgumentException - if the timestamp is more than the timestamp of any committed version
  - tableExists
```
boolean tableExists()
```
    Returns:
    
    Whether a Delta table exists at this directory.
  - forTable
```
static DeltaLog forTable(org.apache.hadoop.conf.Configuration hadoopConf,
                         String path)
```
    Create a DeltaLog instance representing the table located at the provided path.
    
    Parameters:
    
    hadoopConf - Hadoop Configuration to use when accessing the Delta table
    
    path - the path to the Delta table
    
    Returns:
    
    the DeltaLog for the provided path
  - forTable
```
static DeltaLog forTable(org.apache.hadoop.conf.Configuration hadoopConf,
                         org.apache.hadoop.fs.Path path)
```
    Create a DeltaLog instance representing the table located at the provided path.
    
    Parameters:
    
    hadoopConf - Hadoop Configuration to use when accessing the Delta table
    
    path - the path to the Delta table
    
    Returns:
    
    the DeltaLog for the provided path

Interface DeltaLog

Method Summary

Method Detail

snapshot

update

getSnapshotForVersionAsOf

getSnapshotForTimestampAsOf

startTransaction

getCommitInfoAt

getPath

getChanges

getVersionBeforeOrAtTimestamp

getVersionAtOrAfterTimestamp

tableExists

forTable

forTable