HeartSaVioR commented on code in PR #46351: URL: https://github.com/apache/spark/pull/46351#discussion_r1589714412
########## sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/HDFSBackedStateStoreProvider.scala: ########## @@ -351,7 +351,7 @@ private[sql] class HDFSBackedStateStoreProvider extends StateStoreProvider with } override def close(): Unit = { - synchronized { loadedMaps.values.asScala.foreach(_.clear()) } Review Comment: Wow... this was totally unsafe... It's blindly assuming that after calling close() nothing will read from loadedMaps and its entities. If we were ever doing this we had to also make sure these entries must be removed from loadedMaps as well... (and/or explicitly invalidate this instance so that loadedMaps (and its entities) can never be accessed afterwards.) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org