Currently files can be easily delived by agent policy. It would be nice to have an option to remove/delete a specific file from a remote system, defined in the File Delivery policy.

Yes, for time being it's file delivery policy, not file removal policy. Could be nice thing, but development resources are limited and there are more important features to work on, so probably only with sponsored development.

It's probably possible to add files with 0 size - this will at least delete contents of files.
It's possible to configure action on agent that would do deletion of files.
And, also requires quite some effort - nxshell / java app can use file manager api to talk to agent nodes and to the deletion.