The astute reader will notice that the list of problems encountered by the QAMC project is substantially longer than the advantages offered by the systems it has developed. This is hardly surprising, given the exploratory nature of the project and the honesty that non-commercial research promotes.
Successful development of diagnostic systems trained on large medical databases demands the collection of relevant, reliable, data - free from systematic defects. This is a tall order, as many of those who handle medical data will attest. However, it is important to communicate that the limiting factor in creating accurate prediction systems out of large databases is (in this instance) the data itself, rather than the modelling method.
One technological improvement to a system like the QAMC web page would be to implement a more interpretable approach. Bayesian networks seem to have appeal in this regard, but it is not clear (a) whether meaningful conditional independence relations could be found among the variables available, (b) whether training such systems is feasible with over 770 000 cases, and (c) whether any greater diagnostic accuracy could be achieved.
Other issues, like stationarity and generalisation to other patient populations will always be a challenge. As more and more people gain access to the Internet, many of the legal questions we discussed will have to be addressed.
It is our hope that this paper and the QAMC web page will promote thought and discussion about the relevant issues. The reader may be interested to write to us (via the feedback section of the web page ), or to read some of the comments that we have received.