replaceWhere

If we want to replace content of table or file in a path below can be possible.

• The replaceWhere option atomically replaces all records that match a given predicate.

• You can replace directories of data based on how tables are partitioned using dynamic partition overwrites.

Python:

replace_data.write
  .mode("overwrite")
  .option("replaceWhere", "start_date >= '2017-01-02' AND end_date <= '2017-01-30'")
  .save("/tmp1/delta/events")

SQL:

INSERT INTO TABLE events REPLACE WHERE start_data >= '2017-01-01' AND end_date <= '2017-01-31' SELECT * FROM replace_data