You are good to go, as far as installation folder with the D:\SteamLibrary\ install. Did you select the SH4.exe app and right-click it, then on the first "General" tab, clear the "Attributes" "Read-only" tick box? After that one, click on the "Compatibility" tab, and do like in this picture:
For most 'modern' computers, you'll have to set the "Disable fullscreen optimizations" and "Change high DPI settings", which are in the middle window there (File Explorer on the bottom). The DPI button will open that upper window, and you usually have to click on that "Override high DPI scaling behavior..." tick box. You may or may not have to "Run as administrator" and / or "Run this program in compatibility mode for:" stuff, and every once in a while, a person might have to use the top button of that top window, with "Program DPI" and set it to "Whenever I open this program". Experimentation is needed sometimes for all of those. You should also have your Windows graphics set to the "native" resolution of your display monitor ("recommended" in the Windows dialog), and the same setting chosen in the game.
In the meantime, did you empty the Save folder (C:\Users \UserName \Documents \SH4), and delete everything below the "SH4" level? If not, then what you had previously and what you activated most recently will be a mix of the two. The game does not overwrite a file that already exists in the Save folder when you apply mods. You have to empty the folder so that the game recreates it from scratch.