If you are willing to start the rebase over (git rebase --abort), then this should do what you need:
git rebase -X ours upstream
where upstream is the branch you are rebasing onto.
As noted in this answer and elsewhere, the ours vs. theirs labels are slightly more confusing for rebasing than for merging. After starting a rebase, Git creates an anonymous branch and starts applying commits to it. Since ours means “keep changes from the current branch”, that current branch will be HEAD, which contains upstream and any changes already applied by rebase.