'Escape a single quote '' using a double single quote'
See the help for the quoting rules.
You can check out the help in the powershell command line by typing:
Get-Help about_Quoting_Rules
It explains that backticks are interpreted literally in single-quoted strings.
Because the contents of single-quoted strings are interpreted literally, you cannot use the backtick character to force a literal character interpretation in a single-quoted string.