Windows Server Core

With Windows Server 2012, the default option when installing is Server Core.
Server Core is not a new function it was also included in Windows Server 2008 R2 but if you installed Server Core there you had no option to upgrade to «full GUI» if you wanted to change.
In 2012, you have full options to upgrade / downgrade as you choose.

So why choose Server Core and why does it matter?

Server Core strips a server of its GUI, and all of the unnecessary components (Including Internet Explorer) and It reduces how many patches you would need to install on the server (and thus reducing the needed downtime for patching)
and since it removes Internet Explorer and the GUI it also reduces the attack surface on that server.

Does it improve the performance on the server?
Yes, it does, you do not have the CPU wasting cpu cycles on the GUI and you also get more memory available. Here is the difference between Server 2012 GUI and Core.

In 2008 R2 you had limited options to manage it remotely you would need some good tool or use WS-MAN / PowerShell.
Since WS2012 includes a fully multiserver management tool with Server Manager, it makes it easy to manage servers with Server Core.

So what options do you have for managing a server core?
Server Manager
PowerShell
Command Prompt
sconfig

when you are finished with installing a server core setup you only get this window

(NOTE: If you manage to close the command prompt, press CTRL + ALT + DEL to open task manager and eventually start cmd.exe from there again.

From here we can start for instance sconfig or we can use PowerShell, if we use sconfig we get a command based interface to change the regular settings that appear in server manager.

Or we can use Server Manager, from here we can manage multiple servers (either via the GUI itself, or launch RDP or PowerShell

So from here I can also chose to install Roles and Features as well, I can also choose to shutdown computers as well.
And if you wish to upgrade from Core to GUI you can run the command

install-windowsfeature Server-Gui-Shell, Server-Gui-Mgmt-Infra in PowerShell.

And append a –restart on the end of the PowerShell command.