The reason is that when instantiating a class template, all its declarations (not the definitions) of its member functions are instantiated too. The class template is instantiated precisely when the full definition of a specialization is required. That is the case when it is used as a base class for example, as in your case.
So what happens is that A<B> is instantiated at
class B : public A<B>
at which point B is not a complete type yet (it is after the closing brace of the class definition). However, A<B>::action‘s declaration requires B to be complete, because it is crawling in the scope of it:
Subclass::mytype
What you need to do is delaying the instantiation to some point at which B is complete. One way of doing this is to modify the declaration of action to make it a member template.
template<typename T>
void action(T var) {
(static_cast<Subclass*>(this))->do_action(var);
}
It is still type-safe because if var is not of the right type, passing var to do_action will fail.