In 1978, a report by Lord Franks recommended reform - "Departmental Committee on section 2 of the Official Secrets Act 1911" - Cmnd 7285, 1978. However, it was not until 1988 that a white paper emerged - "Reform of Section 2 of the Official Secrets Act 1911" - Cm. 408, June 1988. The Official Secrets Act 1989 was enacted to replace section 2. It has been claimed that the 1989 Act is an "armalite rifle" by comparison with the section 2 "blunderbuss." On its face, the 1989 Act is more limited - (i.e. more precisely targeted) - than section 2 but it is still wide in scope.
Turning to the 1989 Act, it covers Security and Intelligence (s.1); Defence (s.2); International Relations (s.3); Crime and Special Investigation Powers (s.4); Information resulting from unauthorised disclosures or entrusted in confidence (s.5); Information entrusted in confidence to other States or international organisations (s.6).
The Act does not contain any public interest defence. This is deliberate. When the Bill was in Parliament, the government resisted amendments for such a defence. This has the consequence that disclosure of a scandal within government would be a criminal offence (subject to any defences provided within the Act). There is a great deal of difference between protecting information relating to serious matters such as disposition of the armed forces etc. and protecting information so as to avoid embarrassment to Ministers. The OSA 1989 is capable of protecting both.
Since the enactment of the 1989 Act there have been some moves toward greater openness on the part of government such as placing the Security Services on a statutory basis - see the Intelligence Services Act 1994, Security Service Acts of 1989 and 1996 and enabling greater freedom of information albeit with considerable exceptions - Freedom of Information Act 2000.
The Official Secrets Act 1989 was enacted as the old Cold War world - (c. 1945-1991) - was drawing to a close. However, that world has been replaced by other challenges and we continue to live in difficult times. Post 9/11, the UK has engaged in military action in Iraq and Afghanistan and, during 2011, has been involved (with NATO partners) in connection with Libya. The nation has also had to live with much serious crime including terrorism and intelligence obtained in such matters must remain wrapped in secrecy. That necessary "core of secrecy" is likely to remain and those "in the know" about serious matters will be expected to take their knowledge with them to their graves.
Telegraph - "The slow road to reform in a nation once ruled by secrecy" - 15th Augsut 2011
For a commentary on the Official Secrets Bill see Campaign for Freedom of Information, 1988.
Troubled history of the Official Secrets Act - BBC 18th November 1998
MI5 - the Security Service
MI6 - Secret Intelligence Service
The Guardian - "Let's free the Official Secrets Act from its cold war freeze."