Unfortunately no... A whole VHD is changed, so for the file level backup, that specific file has changed.
Now, there is a possibility to go to volume level
Volume level backup is much faster compared to file level. In some cases you might be able to use volume level if your exclusions are limited. The problem with exclusions on volume level is that the system needs to build the exclusions from the snapshot and if you have more exclusions, and the list of exclusions grows, then it will fail. Volume level backup uses the VSS exclusion engine, and it can simply become too much for that engine to handle.
There is not really a hard limit but it can hit it and your backups will fail at a certain point in time.
So my advice would be: Try out and see if it works, but keep the above in mind.