Normally the Recycle Bin is not deletable by right-clicking on the desktop and choosing delete. In any case this will get you back a Recycle Bin but it will be deletable:
Right-click desktop | New | Folder | Rename to "Recycle Bin.{645FF040-5081-101B-9F08-00AA002F954E}" w/o the quotes.