There's not much you can do about it slowing down startup. It has to load, and it takes however long it takes. The main purpose of the Security Center is to monitor various functions, such as your AV Software, the Windows firewall and Automatic Updates and inform you if any of them or not working (and this notification is customizable).
Disabling it won't cause any of the functions it monitors to stop working. Its just a monitoring process. With that said, there is enough malware, virus, trojan and spyware out there, and they're getting pretty sophisticated, it wouldn't suprise me to see them start actively disabling services like your AV software and firewall. I'd live with the slow load and go grab a cup of your favorite beverage while the machine reboots.