One of my mistakes was to basically copy the same layout for each controller, not realising that there's such a thing as application.rhtml.
Application.rhtml allows you to create a global template for the whole site. The problem comes when you want to include controller specific bits on the page (like a side bar).
A trick I used (and I'm sure there's probably a million better ways of doing this) is to put a partial in the application.rhtml that you know will be used in the controllers. For example:
application.rhtml
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="content-type" content="text/html;charset=UTF-8" />
<title>Your application</title>
</head>
<body>
<div id="left">
<%= render :partial => 'sidebar' %><br/>
</div>
<div id="main">
<%= yield :layout %>
</div>
</body>
</html>
Then in your controller view folders, include a _sidebar.rhtml partial like this:
_sidebar.rhtml in views/user
This is the User Controller Sidebar
<%= link_to "logout" %>
_sidebar.rhtml in views/admin
This is the Admin Controller Sidebar
<%= link_to "Add User" %>
<%= link_to "Remove User" %>
This way you can have a global layout, but also unique side bar details.