Pyspark: filter dataframe by regex with string formatting?

From neeraj’s hint, it seems like the correct way to do this in pyspark is:

expr = "Arizona.*hot"
dk = dx.filter(dx["keyword"].rlike(expr))

Note that dx.filter($"keyword" ...) did not work since (my version) of pyspark didn’t seem to support the $ nomenclature out of the box.

Leave a Comment

Hata!: SQLSTATE[HY000] [1045] Access denied for user 'divattrend_liink'@'localhost' (using password: YES)