pyspark.sql.DataFrame.unpersist#

DataFrame.unpersist(blocking=False)[source]#

Marks the DataFrame as non-persistent, and remove all blocks for it from memory and disk.

New in version 1.3.0.

Changed in version 3.4.0: Supports Spark Connect.

Parameters

blockingbool: Whether to block until all blocks are deleted.

Returns

DataFrame: Unpersisted DataFrame.

Notes

blocking default has changed to False to match Scala in 2.0.

Cached data is shared across all Spark sessions on the cluster, so unpersisting it affects all sessions.

Examples

>>> df = spark.range(1)
>>> df.persist()
DataFrame[id: bigint]
>>> df.unpersist()
DataFrame[id: bigint]
>>> df = spark.range(1)
>>> df.unpersist(True)
DataFrame[id: bigint]