First of all, what the heck is a product backlog?
Backlog isn't a term you use regularly. It's kind of an old term or used in particular niches like manufacturing or supply chain.
It's now been popularized by process frameworks like Scrum or Agile. Almost exclusively people are are using the term backlog to refer more to software and product development nowadays.
Which brings up a good point that it is truly like manufacturing. There's lean manufacturing which is like Lean frameworks for working with product teams. Things like Kanban too, all came from manufacturing. So does the concept of a backlog.
During the industrial revolution -- think Henry Ford and his assembly line, when he was making Model T Fords, it was specialization and manufacturing and division of labor that make available the cheap productive capacity of humans and machines. We are still creating and producing things but this time they are not tangible or real coming off of a conventional assembly line. This time, the assembly line is organized by Trello, JIRA or any number of Product Teams can get their hands on and use. That's why we Plans, why there are phases, why there is testing and delivery processes etc. What the product manager does is like a plant facility manager - overseeing the "product" that is being "manufactured".
Scrum teams generally finalize estimates and commit to a Sprint Backlog during their bi-weekly Sprint Planning sessions. Ideally most (if not all) User Stories will already have been discussed and estimated before Sprint Planning. Trying to have the needed conversations, estimate and commit all at once often results in exhausting, marathon sessions that result in inconsistent velocity and frustrated teams.
The vehicle for pre-estimating User Stories is typically a weekly Backlog Grooming session facilitated by the Scrum Master. The agenda for these recurring meetings would usually include reviewing in detail Product Backlog items that are nearing the top of the list, then using Planning Poker to estimate and record the relative effort required to deliver them.
User Stories are often split during Backlog Grooming, and Assessment Criteria are established based on the conversation. Most teams leave more granular, task-level details for Sprint Planning.
One thing that I think all of us who have been in product before know is that nothing goes according to plan, at least most of the time. Defects come up that are unintended, there's blindspots on the team and all sorts of things that unless you have an magnifying glass, it really is not something you or any other person can foresee.
First of all it's not your fault. We put a lot of pressure on Product Managers and Product Owners to own their product. While ultimately the responsibility lies with us, it's not something we can all foresee. In fact, in the spirit of Agile and Scrum we should be making hypotheses or educated guesses as my Grade 8 teacher taught us, use the scientific method and see what comes out of the experiment.
Sometimes you're right, sometimes you're wrong and sometimes the users just don't the product or feature or service that they told you they wanted when they finally see it in production. There's sometimes pressure to get things right the first time and that leads down the path of going back to "waterfall" or full requirements before any type of work or releasing and it's just not healthy. More on that in other articles.
The great thing about software is that you can "fix" your mistakes in the next iteration. You can remove a feature if no one uses it (though it takes a lot of courage to do this). You can fix the defects or fix the workflow to improve engagement or adoption. Nothing should be set in stone and A/B testing helps as this mentality and mindset of experimentation.
The same is true about things in your backlog. You should always err on the side of having way too much on there than to have nothing to do. The backlog itself should be fluid, should be just enough to convey the information needed and nothing more and should be a big big laundry list of all sorts of crap like defects, changes, new functionality.
Why do you need all this crap in your backlog?
There's 2 main reasons why it's crucial to do this.
1. You have unexpected delays on the big plans you had for the next few releases.
This comes up all the time. Your feature is dependent on some 3rd party be it a internal team or external vendor. You cannot predict when they will get it to you and sometimes they don't run agile. And even if they have Sprints, they also have their backlog and they won't be able to intake your work until X weeks or months. You need to then pivot, be ready to pivot to something else or your just "walking" instead of sprinting. Even the best laid plans go to ruin and it happens ALL THE TIME.
2. The backlog should be visible and provides a sense of the roadmap.
Features don't live in isolation. In a complex system like software applications, each introduction of a new feature introduces what in physics is called, entropy. Every time you introduce a new element, it inevitably disrupts the systems (i.e. your application) and changes the whole application. I know in reality it doesn't seem like it does because this so-and-so feature is independent of this other so-and-so feature. But when you look at the code, when you look at people use it, when you look at your QA testers doing their regression they would argue otherwise.
With a full backlog, it's now possible to see how the dynamic changes release-over-release can have on adjacent features so you're better prepared for how much entropy you are introducing. Feature X should be released with Defect Y or Defect A is going to affect Feature B so you better move that to the next release. It's crucial you see your app as a complex, holistic system.