rename-item and override if filename exists

As noted by @lytledw, Move-Item -Force works great for renaming and overwriting files.

To replace the index. part of the name, you can use the -replace regex operator:

Get-ChildItem index.*.txt |ForEach-Object {
    $NewName = $_.Name -replace "^(index\.)(.*)",'$2'
    $Destination = Join-Path -Path $_.Directory.FullName -ChildPath $NewName
    Move-Item -Path $_.FullName -Destination $Destination -Force
}