The House in Norham Gardens by Penelope Lively

No.40 Norham Gardens, Oxford, is the home of Clare Mayfield, her two aged aunts and two lodgers. The house is a huge Victorian monstrosity, with rooms all full of old furniture, old papers, old clothes, memorabilia - it is like a living museum.
Clare discovers in a junk room the vividly painted shield which her great-grandfather, an eminent anthropologist, had brought back from New Guinea. She becomes obsessed with its past and determined to find out more about its strange tribal origins.
Dreams begin to haunt her - dreams of another country, another clture, another time, and of shadowy people whom she feels are watching her. Who are they, and what do they want?