[DML] Scala/Java APIs for describing table history

[tdas]

Official issue for tracking work for building Scala APIs for describe table history.

[andremesarovic]

Is there a timeline for this? 3.0 perhaps? The ticket title seems backwards - shouldn't it read "SQL for describing table history" since you can already do it with the Scala/Python Dataframe API? The description should be expanded to refer to all Delta-specific SQL constructs such as:

DESCRIBE HISTORY my_table
DESCRIBE DETAIL my_table
SELECT * FROM my_table VERSION AS OF 5 
etc.

[marmbrus]

The description is not backwards here. We do plan to add scala/java APIs here first as this is easier to do without extending Apache Spark.

In the long run I also hope that we can work with the Spark community to add the needed extension points so the same SQL APIs that are available in Databricks can be used by everyone.