Hosts commonly use it to keep one website from reading another website's files on the same server. When a script tries to access a path outside the allowed list, PHP refuses the request.
It is a useful layer of defense, for example it can limit what a file inclusion or file read flaw can reach, but it is not a complete security boundary and does not replace patching. Allowed paths that are too broad provide little protection.
Set it per site to the smallest practical list of directories, test the site afterwards, and remember that CLI and web PHP can read different configuration files, so verify the value for the environment that serves the site.