pyspark.sql.DataFrame.colRegex

DataFrame.colRegex(colName: str) → pyspark.sql.column.Column

Selects column based on the column name specified as a regex and returns it as Column.

New in version 2.3.0.

Changed in version 3.4.0: Supports Spark Connect.

Parameters

colNamestr

string, column name specified as a regex.

Returns

Column

Examples

>>> df = spark.createDataFrame([("a", 1), ("b", 2), ("c",  3)], ["Col1", "Col2"])
>>> df.select(df.colRegex("`(Col1)?+.+`")).show()
+----+
|Col2|
+----+
|   1|
|   2|
|   3|
+----+

pyspark.sql.DataFrame.coalesce pyspark.sql.DataFrame.collect