Is it possible to update a template throughout a project? For instance if I modify the base Template within the Template folder is it possible to update those documents already created without manually editing each instance?
Yes it is possible to update a template throughout a project. But these changes would only apply to new documents, and will have no impact to existing documents.
As you surmised, you would have to modify the existing documents manually.