Storing pointer value means that after Get(), the value inside map have more than one pointers. This could lead to mistake that trigger data race, for example when upsert is implemented by updating the value inside the map instead of replacing it.
Suggestion: add RetrieveCB()--similar to upsert and remove--which receive callback that can be used to deep clone the pointed value.