Having index.php in the Search Engine Friendly URLs is usually not wanted. This tutorial shows how to remove the index.php part for nicer looking SEF links.
For Apache users: The file htaccess.txt in root of Joomla files should be renamed to .htaccess (if you already have an .htaccess file you can add the contents of htaccess.txt to it.)
For Windows IIS 7 users: Rename web.config.txt to web.config and install IIS URL Rewrite Module before activating Joomla SEO URL rewriting.
In Joomla 3 Administrator dashboard go to Global Configuration -> Site tab
In "SEO Settings" section
Set "Use URL rewriting" to Yes
