"I am writing to you now for the third time without receiving a reply from you. Don't you remember your father anymore? Are we never going to see eachother again?" -- Albert Einstein to Hans Albert, Sept 26, 1916.
This caused Einstein himself considerable distress, ultimately to the point of physical and nervous collapse, at a time when he was in the heat of working on his theory of General Relativity (around 1915 and after).
For more information see, in particular, The Private Lives of Albert Einstein by Highfield & Carter.