pyspark.resource.ExecutorResourceRequests¶

class pyspark.resource.ExecutorResourceRequests(_jvm=None, _requests=None)[source]¶

A set of Executor resource requests. This is used in conjunction with the pyspark.resource.ResourceProfileBuilder to programmatically specify the resources needed for an RDD that will be applied at the stage level.

New in version 3.1.0.

Notes

This API is evolving.

Methods

`cores`(amount)
`memory`(amount)
`memoryOverhead`(amount)
`offheapMemory`(amount)
`pysparkMemory`(amount)
`resource`(resourceName, amount[, …])

Attributes

requests

pyspark.resource.ExecutorResourceRequest pyspark.resource.TaskResourceRequest