What causes folders PFinfo and files files.info to be generated? What is the purpose of those system folders and files?
There is no explanation to be found in the user's guide.

They store the Program Window state (bookmarks, breakpoints, and so on), so that this state is preserved when you reopen a program file or database source later.

So this is stored when "Preferences -> Windows Types -> General -> Save window state" is checked? This was introduced with PL/SQL Developer version 14?
How come that changing this checkbox makes no difference in a complete export of preferences? Where is this setting stored then instead?

This feature existed before version 14.0. The preference is stored in %APPDATA%\PLSQL Developer [version]\Preferences\[user]\default.ini, like all other preferences.