Where are logs in Spark on YARN?

You can access logs through the command

yarn logs -applicationId <application ID> [OPTIONS]

general options are:

• appOwner <Application Owner> – AppOwner (assumed to be current user if not specified)
• containerId <Container ID> – ContainerId (must be specified if node
  address is specified)
• nodeAddress <Node Address> – NodeAddress in the format nodename:port (must be specified if container id is specified)

Examples:

yarn logs -applicationId application_1414530900704_0003                                      
yarn logs -applicationId application_1414530900704_0003 myuserid

// the user ids are different
yarn logs -applicationId <appid> -appOwner <userid>