First of all, Foreach-Object is not an actual loop and calling break in it will cancel the whole script rather than skipping to the statement after it.

Conversely, break and continue will work as you expect in an actual foreach loop.

Item #1. Putting a break within the foreach loop does exit the loop, but it does not stop the pipeline. It sounds like you want something like this:

$todo=$project.PropertyGroup 
foreach ($thing in $todo){
    if ($thing -eq 'some_condition'){
        break
    }
}

Item #2. PowerShell lets you modify an array within a foreach loop over that array, but those changes do not take effect until you exit the loop. Try running the code below for an example.

$a=1,2,3
foreach ($value in $a){
  Write-Host $value
}
Write-Host $a

I can’t comment on why the authors of PowerShell allowed this, but most other scripting languages (Perl, Python and shell) allow similar constructs.