There are a lot of settings out there that are just plain cool. And I enjoy playing or designing within the limits of the canon universe. I also love settings that lend themselves to new creations and a player-driven expansion of the universe or concepts within it.
This, all the way. A great example of this kind of setting is Star Wars. There is a strong canon, but it has left a grand abundance of opportunities for new or branching lore.
I also think that a mutually-loved fictional world is one of the strongest foundations you can have for developing a group dynamic around a shared vision for a game. People are going to have different ideas about how to approach most game worlds because creating a shared imaginary with any degree of similarity from one mind to another is a tremendous challenge. But, if everyone in the group loves Indiana Jones or Buffy the Vampire Slayer or The Stormlight Archive, you’ve got a stew going on!
As I work on adapting literature to games, I am experimenting with different ways to restrict or encourage creation.
Interesting. I’m picturing the As I Lay Dying RPG or the Don Quixote RPG.
So far, I find it easier to encourage sandbox play either indirectly, such as minimal lore or implied setting in tables and descriptions. Direct encouragement through tangible, valuable rewards for creating new things (for example, taking time to document a species of flora/fauna in UVG = XP), in my limited experience, is very effective.
I suspect the [do thing = get XP] loop would likely lose its appeal after a certain amount of repetition.
To build out some more ideas, I thought of an opposite question: How would one enforce sticking to canon, either through the setting or mechanically?
I mean, this is what PbtA is all about. The whole idea of PbtA is that there are thousands of great worlds to explore and for each a thousand ways to explore them, but each way of exploring them is a bit lacking in elbow room. You enforce the canon by making the options available to the players reinforce the predetermined themes and activities.
Probably the best example of this is Bluebeard’s Bride. The game doesn’t deviate from the bride’s fate in the story, but rather builds the tragedy into the rails of the mechanics. It might be a bit extreme as examples go, but it’s fitting.
I am not suggesting that PbtA is that best way to go for your game, but you might look at the work done in that space for ideas of how to achieve your goals here.
Magpie Games is especially good at designing game mechanics around supporting a core game world. They made the aforementioned Bluebeard’s Bride, as well as the licensed RPG for Avatar and Korra. That game is specifically designed around exploring and expanding a richly canonical world without breaking the canon.