pyspark.RDD.unpersist

RDD.unpersist(blocking: bool = False) → pyspark.rdd.RDD[T]

Mark the RDD as non-persistent, and remove all blocks for it from memory and disk.

New in version 0.9.1.

Parameters

blockingbool, optional, default False

whether to block until all blocks are deleted

New in version 3.0.0.

Returns

RDD

The same RDD

See also

RDD.cache()

RDD.persist()

RDD.getStorageLevel()

Examples

>>> rdd = sc.range(5)
>>> rdd.is_cached
False
>>> _ = rdd.unpersist()
>>> rdd.is_cached
False
>>> _ = rdd.cache()
>>> rdd.is_cached
True
>>> _ = rdd.unpersist()
>>> rdd.is_cached
False
>>> _ = rdd.unpersist()

pyspark.RDD.union pyspark.RDD.values