After clicking or touching on a Material Design button it stays focused —> to resolve this, you need to add the following code: onclick="this.blur()"
<button mat-raised-button onclick="this.blur()" (click)="onEdit()">Edit</button>
or in your case
<td mat-cell *matCellDef="let element">
<button mat-icon-button type="button" onclick="this.blur()" (click)="downloadStuff(element)">
<mat-icon mat-icon matTooltip="{{element.someId}}">picture_as_pdf</mat-icon>
</button>
</td>