How Does Our Mind Store Information in Memory?

Say you just returned from a meeting -- how would your mind remember and organize all the information related to the meeting? Who were the key members of the meeting, what was spoken, what was in the presentations, roughly how many people were there, what was the room like, what were the details? Such information needs to be organized and indexed in a way so that it can be quickly accessed in future if say I met someone from the meeting later to chat about some related topic.

We propose that information related to such events and inputs is stored as a sketch -- a summary of the computation of a deep network that processes this input. Since we can often recall and re-visualize the meeting room, the sketch should be approximately reversible; that is, one can approximately reconstruct the original input and/or its crucial properties, summaries and statistics. The sketch needs to be gracefully erasable so that even if a long time has passed and one may have forgotten several details, the high-level properties and statistics of the event may still be preserved in the sketch.

The sketching mechanism is based on random subspace embedding and is able to approximately reconstruct the original input and its basic statistics up to some level of accuracy. The sketching mechanism implicitly enables different high level object oriented abstractions such as classes, attributes, references, type-information, modules without explicitly incorporating such ideas into the mechanism operations. (Based on https://arxiv.org/abs/1905.12730).