The point where things really started to go wrong was in the summer of 2012. Prior to that, we'd had 3 years or so of steady progress - not flying up the league like we did from 2004 to 2007/08, but slowly building something. If Miller hadn't got that injury against Scunthorpe, we'd probably have made the play-offs in the 11/12 season.
What we should have done that summer was build on what we had, kept most of the squad and add a couple more players to kick us on. What happened is we lost three players who were regulars in that squad (Lubo, Taiwo and Zoko) and didn't properly replace them. The budget was cut, no doubt about it. As a result, Abbott had to take a few punts - Bugno to left-back moving Chantler to centre mid to replace Taiwo, Cademarteri (as good as he looked early on two-year deal was a big mistake with hindsight). They didn't work, and the quality of loan signings (bar Garner) weren't as good as what we'd had before. Then there was Miller's injury in the first game - you can't predict something like that.
Since then it's been nothing but downhill. People will talk about us "squandering cup money", but the majority of that will have gone on keeping up the wage bill - we didn't really get big enough crowds or commercial revenue to justify the quality of squad we had. Things could have been rectified (probably with an experienced appointment either instead of Kavanagh or as assistant to support him) to avoid relegation. We're pretty steady now though - this division is a poor one and a few canny signings can turn you around easily enough.
Edit - [censored] hell, forgot all about Mike Edwards as Lubo's replacement!