The core portion of the Builder pattern concerns the Abstract Builder and its subclasses (concrete builders). According to GoF’s Design Patterns, director simply “notifies the builder whenever a part of the product should be built”, which can be perfectly done by the client.
The StringBuilder class in the Java API is an example of a builder without the respective director — typically the client class “directs” it.
Also, in Effective Java and Creating and Destroying Java Objects, Joshua Bloch suggests the use of the builder pattern, and he does not include a director.