WordPress creates pages & posts like this http://brianlis.com/?p=123
Not exactly SEO friendly or even human friendly as p=123 probably doesn’t mean anything to anyone.
To change the way your pages are displayed inside your WP admin panel goto Settings>Permalinks.
Switch the settings to custom and use the value /%postname%.htm or /%postname%.html or /%postname%/ depending on how you want your post slug or URL to be displayed. There might be SEO advantages as to the ending you use. I’ve never seen any evidence of the ending making a difference though.
For more on permalinks visit the WP Codex.