The pilot ejected safely and was taken to the hospital with minor injuries, Marines said. Three homes were destroyed and five more were evacuated, officials said, but so far, there are no reports of civilian injuries.
The fighter jet, part of 3rd Aircraft Wing, crashed around 4:20 p.m., Marine officials said.
"It felt like a bomb was thrown in the backyard of the house," Adriana Ramos, 45, whose home is less than a block from the crash scene, told The Associated Press. "The whole house moved."
Ramos fled with her 4-year-old granddaughter and 10-year-old daughter, who both cried at the sight outside. Another witness, Jose Santos, was driving nearby and saw the plane flying "really low." Then "it just fell down," he said.
At the crash site, there was chaos as people ran in every direction, he said. The two homes were on fire, and it was unclear whether anyone was inside.
Another Harrier, also stationed in Yuma, crashed in an unpopulated area south of Phoenix in May. That pilot ejected safely and was not injured, Marines said.
Investigators are still reviewing what caused that crash.
Imperial is 4 miles from Naval Air Facility El Centro and about 65 miles from Yuma, Ariz.
The Corps will launch an investigation to determine the cause of the crash.
Information from The Associated Press was used in this report.