Thanks to pat's polytope applet that supports clipping (even if it's imperfect), I think I now have a good handle on how to visualize the 24-cell.
The basic idea is to project the 24-cell into 3-space using a vertex-first viewpoint. The envelope of such a projection is the rhombic dodecahedron. (Google for it if you're not sure what exactly a rhombic dodecahedron is).
The first thing to observe is that in 3D, if you have 6 octahedra, each lying along the coordidate axes and touching at a common vertex, and if you can somehow "squish" them together towards the origin so that they touch each other, they will form a rhombic dodecahedron. Of course, the resulting octahedra would not be regular, but would be slightly flattened. But now, imagine that these 6 slightly-flattened octahedra are actually regular, "unsquished" octahedra projected from 4-space. As you know, when you rotate a 3D object into the W-axis, the projection of the object into 3D will "flatten". So each of these flattened octahedra are actually regular octahedra slightly bent into the W-axis, and touching each other. This, in fact, is the first 6 cells of the 24-cell, as viewed vertex-first.
At this point, it's important to understand that there are two ways to bend these cells into the W-axis so that they form a rhombic dodecahedral projection. One way is to rotate them into +W, the other way is to rotate them into -W. Both will give a 4D object that projects on the same volume, but in 4D, one would be curved inwards (into +W) whereas the other would be curved outwards (into -W). So although the 3D projection is a convex rhombic dodecahedron, it's important to understand that in 4D it is actually an open shell (like a hemisphere). For now, let's say we bent the octahedra into +W, and let's say that the vertex projected onto the origin is the "north pole". So these first 6 cells are the cells surrounding the "north pole" of the 24-cell.
Secondly, note that the surface of the rhombic dodecahedron consists of 12 rhombuses. If you take 12 octahedra and rotate them into the W-axis at a 90-degree angle, they will flatten into 12 rhombuses. With the proper arrangement, you can attach them together in 4D such that their projection is a rhombic dodecahedron. Each edge of the rhombic dodecahedron represents a triangular face shared by 2 of the 12 octahedra. This triangular face happens to lie at 90 degrees to our viewpoint, so they appear collapsed into single lines. This leaves 48 "exposed" faces that aren't joined to anything yet: each rhombus represents 2 triangular faces on the octahedron, and the two sides of the rhombus represent opposite pairs of faces on the octahedron. Since there are 12 octahedra, this gives us 48 exposed faces.
Now, note that 24 of these faces are facing "outwards" (i.e., facing -W) and the other 24 are facing "inwards" (facing +W). Because the outward-facing faces form a rhombic dodecahedron, they exactly match the exposed faces on the first 6 cells (the "north pole" cells). Remember that each rhombus on the boundary of the first 6 cells represents two exposed triangular faces of two adjacent octahedra. We can join these faces to the outward-facing faces of the 12 octahedra. This attaches the 12 octahedra to the first 6 octahedra in 4-space. This gives us 2/3 of the 24-cell: the 12 cells we just made are the "equitorial" cells of the 24-cell attached to the "north pole" cells.
Finally, we take another 6 octahedra, and bend them into the W-axis just like that first 6 cells, except that this time, we bend them into -W instead of +W. This gives us a concave arrangement of cells, which form the "south pole" cells of the 24-cell. The outer envelope of these "south pole" cells also project onto a rhombic dodecahedron. The "inward"-facing exposed faces of the equitorial cells match up perfectly, so we join them together, and now we have the complete 24-cell.
So there we have it, the 24-cell, with 6 cells in the first layer (the "north pole" cells), 12 cells in the second layer (the "equitorial" cells - observe how they are precisely at 90-degree angles to our viewpoint, parallel to the W-axis), and 6 cells in the third, final layer (the "south pole" cells).
Does this description make sense to anyone? :-)
(My next task is to try to visualize the 120-cell. Somehow I get the feeling that this won't be quite as easy... Someday, I might try visualizing the 600-cell, someday, if I manage to get a good handle on the 120-cell first.)