I've been involved with hundreds of mid range business application development projects plus a few big ones and my universal experience with all of them is first that process blocks progress; and secondly that the more formal the specifications process, the less realistic the usage specification and therefore the less successful the eventual application.
A big part of the problem is that users and analysts simply don't understand the same words in the same ways. At the obvious level this has the consequence that data dictionary definitions don't apply either to all users of the data or consistently over time -and at the more subtle level it means that users will describe their jobs in terms of the official procedure manual while expecting you to understand that they don't actually follow it.
In a work to rule strike, union members explicitly follow every rule or procedure laid down by management - and thus bring the organization to a grinding halt. Sit down with users within any kind of formal setting or process and what they tell you will reflect the rule book - and if you implement it in your code you'll be enforcing a work to rule process: meaning that your application will bring the organization to a grinding halt.
The better answer is continuous prototyping - a process which starts by establishing long term user relationships within which you do initial development and then continues as you revise your prototype after it goes into production.
In the initial development stage for continuous prototyping you work in the user environment, build your initial screens and attach actions interactively under user direction - then, just as you're beginning to think you might have a clue, you'll run into a roadblock showing that you've got it wrong.
Getting the first few wrong that way is normal - because users don't understand the bigger context any more than you do and so they'll applaud your efforts, and then find niggling little maybes, how abouts, and surprises that cumulatively cause you to throw away what you've done and start over. Two or three starts later, you'll have a first screen or set of screens along with some appreciation of the complexities and overall flow of the work - and the rest of the thing will start to fall into place.
My favorite tool for this has long been Unify's Vision/U2K combination - because the on screen create and drag was both fast and highly intuitive for the users watching me work; because the source files produced were text (meaning, among other things, that a simple script could extract the data references and build the schema); because search by example worked across everything; because you could make large, complex, screens accessing as many tables (and databases) as you wanted to; because you could cheerfully mix system generated data with user generated data; because a host of external variables let you customize almost everything for the specific end user; and, because you could tie in any Unix process you wanted to from anywhere in the operational process.
Now if you haven't seen or used one of these toolsets this will all seem rather foreign to you and there's not much I can do to help. You might, however, want to consider these two factoids and then use your imagination a bit:
- Microsoft Access is a limited volume, single user, single data set, single screen, single language copy of the typical 1980s 4GL/RDBMS application builders without the multi-processing, language linkages, and user specific customization capabilities that come with Unix - think of it as a kind of thumbnail demo for a true 4GL/RDBMS combination and you'll get the idea.
- The Microsoft small screen pop-up and click routine is an outgrowth of graphics limitations on the 386/486 based PCs Windows 3.X was aimed at - because, on those, small pop-ups inside rectangles drawn in primary colors gave the impression of speed and thus validated the purchase decision. That worked because the bright pop-up window frame itself registered with the viewer - while the switch to pastels or other light colors on putting something inside that window hid the delay in getting the text and data in place.
One consequence of the interactive, user driven, design approach is that you find yourself making bigger and bigger screens with increasing levels of apparent complexity - because users simualtaneously demand that you add just one more thing, and decry "computer hassles" like having to click a mouse as time and concentration killers. As a result, you'll typically find yourself pushed to the limits of your hardware - the screen below, for example, illustrates this because it's designed for NCD's 21", 1280 x 1640, 1990s HMX screens as an absolute functional minimum.
This kind of thing is counter-intuitive for people raised on the Microsoft desktop, but real users (i.e. people who care about their jobs, and not the PC) don't care that it's big, ugly, and intimidating: they care that it's hassle free, menu free, and fast -things which, at least according to the users in the pilot group this was tried with, would have made the time and billings manager it fronted for extremely effective in large scale use.
Screens like this are a consequence of how active prototyping works - users always want both more functionality and less computer hassle, so you end up finding ways to make the computer function as a tool for them to get their work done - something that reverses the traditional developer's view of the user as the computer program's input device.
To make the process work you define the screens and actions interactively with users; giving them things they can see and react to as you talk - something that requires you to fake in the back-end stuff until you get time to do it off-line. That gives them a continuously evolving view of what they'll get - and, of course, that process makes them think: meaning that while you're off filling in those back end holes they'll change their minds about some things, add a few things you've never dreamt of, and invite a couple of new players into the game - thus quite possibly forcing you to start over on either the whole thing or parts of it.
As a result the process is often extremely frustrating; but when you're done you usually have a prototype that works - one users already trust, one that reflects the actual work flow; one that has been tested for the overwhelming majority of usage cases, and one that's really fundamentally ready for production - provided you remember that today's production system is just a prototype for tomorrow's because neither their workload nor your application is ever static for long.