pyspark.sql.functions.exists

pyspark.sql.functions.exists(col, f)

Returns whether a predicate holds for one or more elements in the array.

New in version 3.1.0.

Parameters

colColumn or str

name of column or expression

ffunction

(x: Column) -> Column: ... returning the Boolean expression. Can use methods of Column, functions defined in pyspark.sql.functions and Scala UserDefinedFunctions. Python UserDefinedFunctions are not supported (SPARK-27052).

:return: a :class:`~pyspark.sql.Column`

Examples

>>> df = spark.createDataFrame([(1, [1, 2, 3, 4]), (2, [3, -1, 0])],("key", "values"))
>>> df.select(exists("values", lambda x: x < 0).alias("any_negative")).show()
+------------+
|any_negative|
+------------+
|       false|
|        true|
+------------+

previous

pyspark.sql.functions.encode

next

pyspark.sql.functions.exp