It is simply a matter of how you click on the files with your mouse.
When you select a file to be deleted by clicking on it with your mouse and
you are holding the Ctrl key down to go and select the next file, you must
remove your finger from the mouse after clicking on the file and then move
your mouse pointer and click on the next file you want. When you are done
selecting files, press the delete key.
You are dragging the first file you selected, by not removing your finger
from the mouse after clicking on it and this automatically makes copies.
Just change your technique.
Now, select the copies and delete them. They will go to your Recycle Bin.
Open it and delete them, no point keeping all this there.
