It turns out it is really easy to embed certain kinds of things in WordPress. Video sites like YouTube and Vimeo work great. Just cut and paste the URL into the Visual editor, make sure it isn’t a link, then publish! Unfortunately Google Docs embedding doesn’t work at this time, but documents can be uploaded and embedded using Scribd:

And here is an example of Vimeo embedding: