Techniques are described herein for generating and using in-memory data structures to represent columns in data block sets. In an embodiment, a database management system (DBMS) receives a query for a target data set managed by the DBMS. The query may specify a predicate for a column of the target data set. The predicate may include a filtering value to be compared with row values of the column of the target data set. Prior to accessing data block sets storing the target data set from persistent storage, the DBMS identifies an in-memory summary that corresponds to a data block set, in an embodiment. The in-memory summary may include in-memory data structures, each representing a column stored in the data block set. The DBMS determines that a particular in-memory data structure exists in the in-memory summary that represents a portion of values of the column indicated in the predicate of the query. Based on the particular in-memory data structure, the DBMS determines whether or not the data block set can possibly contain the filtering value in the column of the target data set. Based on this determination, the DBMS skips or retrieves the data block set from the persistent storage as part of the query evaluation.