Foreign keys are just constraints to enforce referential integrity. You will still need to use JOINs to build your queries.
Foreign keys guarantee that a row in a table order_details with a field order_id referencing an orders table will never have an order_id value that doesn’t exist in the orders table. Foreign keys aren’t required to have a working relational database (in fact MySQL’s default storage engine doesn’t support FKs), but they are definitely essential to avoid broken relationships and orphan rows (ie. referential integrity).