Did you guess right?  Read the last verse of the poem and see:

But he let slip the secret
That filled me with such doubt
'Cause when I shouted, "Mummy!"
The man next door came out!

The Man Next Door