pyspark.sql.DataFrame.persist

DataFrame.persist(storageLevel=StorageLevel(True, True, False, True, 1))[source]

Sets the storage level to persist the contents of the DataFrame across operations after the first time it is computed. This can only be used to assign a new storage level if the DataFrame does not have a storage level set yet. If no storage level is specified defaults to (MEMORY_AND_DISK_DESER)

New in version 1.3.0.

Notes

The default storage level has changed to MEMORY_AND_DISK_DESER to match Scala in 3.0.

previous

pyspark.sql.DataFrame.orderBy

next

pyspark.sql.DataFrame.printSchema