His grandmother, Margaret Pattison, moved with her husband and son (Gordon’s father) to Los Angeles in 1932, settling on Bunker Hill. She soon divorced her husband and worked at Bullock’s, saving enough money to lease sixy units of housing in the area. She sublet these and eventually accumulated enough money to purchase the Castle, a twenty room Victorian mansion, in 1937.
Pattison lived on Bunker Hill until he was five, when his family moved to Westchester. His family continued renting out rooms in the Bunker Hill properties, and Gordon helped his father maintain the units into his teens.