Registering a new DelegatingHandler in ASP.NET Core Web API

As already mentioned in previous comment, look into Writing your own middleware

Your ApiKeyHandler can be converted into a middleware class that takes in the next RequestDelegate in its constructor and supports an Invoke method as shown:

using System.Threading.Tasks;
using Microsoft.AspNetCore.Http;
using Microsoft.Extensions.Logging;

namespace MyMiddlewareNamespace {

    public class ApiKeyMiddleware {
        private readonly RequestDelegate _next;
        private readonly ILogger _logger;
        private IApiKeyService _service;

        public ApiKeyMiddleware(RequestDelegate next, ILoggerFactory loggerFactory, IApiKeyService service) {
            _next = next;
            _logger = loggerFactory.CreateLogger<ApiKeyMiddleware>();
            _service = service
        }

        public async Task Invoke(HttpContext context) {
            _logger.LogInformation("Handling API key for: " + context.Request.Path);

            // do custom stuff here with service      

            await _next.Invoke(context);

            _logger.LogInformation("Finished handling api key.");
        }
    }
}

Middleware can take advantage of the UseMiddleware<T> extension to
inject services directly into their constructors, as shown in the
example below. Dependency injected services are automatically filled,
and the extension takes a params array of arguments to be used for
non-injected parameters.

ApiKeyExtensions.cs

public static class ApiKeyExtensions {
    public static IApplicationBuilder UseApiKey(this IApplicationBuilder builder) {
        return builder.UseMiddleware<ApiKeyMiddleware>();
    }
}

Using the extension method and associated middleware class, the
Configure method becomes very simple and readable.

public void Configure(IApplicationBuilder app) {
    //...other configuration

    app.UseApiKey();

    //...other configuration
}