Core includes the dashboard code in wp-admin, the shared libraries in wp-includes and the PHP files in the site root. Site-specific material lives separately in wp-content, and settings live in wp-config.php.
Core receives minor security releases, which WordPress can install automatically, and larger feature releases that usually need more care. Because the official project publishes checksums for each release, the integrity of core files can be verified.
Do not edit core files directly. Changes are overwritten by the next update, and unexpected modifications inside core folders are a common sign of compromise. Put custom code in a plugin or child theme instead.