The following seems to have worked for a few people so I’m submitting it as the answer:
python manage.py celerybeat --pidfile= --schedule=/var/my_app/celerybeat-schedule
--pidfile= (an empty string as the pidfile arg) seems to stop one being created.