Copilot was fine at bounded tasks. Give it a single deep research question, or ask it to go and find some information and synthesise it into a detailed response, and it did a reasonably good job. What it couldn’t do was hold a long conversation across multiple ideas and actually build them on each other. Ask it to help develop several connected ideas over a longer thread, and it would drift toward whatever we were talking about most recently and lose the earlier parts — it couldn’t integrate and relate ideas across the course of a conversation the way that kind of work needs. I don’t know for certain that this was a context-window limitation specifically, but that’s what it looked like from the outside, and it’s a reasonable explanation for the behaviour.
I wasn’t the only one hitting it. I know for example some of the commissioners had tried using Copilot to produce documents directly and found the output was poor quality for the same kind of reason.
The workaround: chunk, save, consolidate
What worked was breaking the work down rather than trying to have one long conversation do everything. I’d keep individual chat threads short — roughly ten to twenty turns — focused on one thing at a time: generating ideas, a piece of research, a look at some enterprise data. Copilot let you save an entire chat thread as a page, so each of those bounded conversations became its own page once it was done.
Then I’d bring the relevant pages together into a notebook, and get Copilot to consolidate across the notebook rather than trying to do everything live in one conversation. That produced noticeably better results — longer, more coherent documents built from several good short conversations, rather than one long conversation slowly losing the thread.
It was frustrating to have to work around at the time, but it was a useful lesson to learn early — it’s essentially where my working understanding of context, context windows, and how to design a task around a model’s actual limits started. Looking back, it’s probably no accident that mem-local ended up caring so much about persistent, structured memory. This method of working in chunks has also now become routine.
Sources: written from my own recollection. There’s no chat record for this work, because it predates my exported chat history and involved NHS operational data, so the underlying documents, data and the exact prompts aren’t reproduced here.