The report by the UK government's independent reviewer of terrorism legislation, David Anderson QC, found that the mass interception of online traffic by government surveillance agency GCHQ should be allowed to continue but under tighter "strict additional safeguards".
Key among those safeguards is the requirement that bulk collection should be approved by serving or retired judges, sitting on the Independent Surveillance and Intelligence Commission (ISIC), reducing the role of government ministers in the approval process.
In addition security services should only collect bulk data where it is targeting communications of people outside of the UK at the time, it said. Where there is the need to look at the communication of a person believed to be in the UK, a specific interception warrant must be authorised by a judge.
GCHQ should also have to provide "a tighter definition of the purposes for which it is sought, defined by operations or mission purposes" in relation to bulk data collection, it recommends.
A new form of bulk warrant, the bulk communications data warrant, which would be limited to the acquisition of communications data, should be introduced, according to the report, which argues this "could thus be a proportionate option in certain cases".
The report says ISPs and telecoms companies should continue to keep communications data for 12 months. That includes information which makes it easier to identify who was using a computer at any one time, such as the IP address a subscriber was assigned at a point in time. It would also include details of who called or emailed whom and when. The requirement was pushed through under the hastily-introduced Data Retention and Investigatory Powers Act (DRIPA).
But Anderson is less supportive of proposals in forthcoming legislation dubbed the "Snoopers Charter". He said that ISPs should only be compelled to keep data about what sites people have visited if "a detailed operational case can be made out and a rigorous assessment has been conducted of the lawfulness, likely effectiveness, intrusiveness and cost".
Data that doesn't "originate or terminate" on the ISP's network should not be retained "before a compelling operational case for it has been made out (as it has not been to date) and the legal and technical issues have been fully bottomed out".
Another controversial introduction under DRIPA, the extension of powers to overseas communications services companies that provide services to UK citizens, should eventually be replaced by a "multilateral arrangement" between states regulating access to information held across borders, according to Anderson.
Anderson says a new law covering data gathering online is required, both to improve the capabilities of law enforcement and to better safeguard individual privacy.
"The current law is fragmented, obscure, under constant challenge and variable in the protections that it affords the innocent. It is time for a clean slate. This report aims to help Parliament achieve a world-class framework for the regulation of these strong and vital powers," he said.
Commenting on the revelation by NSA whistleblower Edward Snowden that western security services were engaged in mass surveillance, Anderson said: "One impact of the leaks in the Snowden Documents in the UK is that they damaged people's belief in the safety of their data."
Eric King, deputy director of campaigning body Privacy International, said: "We applaud David Anderson's report as we applauded the ISC report from just a few months ago. The ISC called our surveillance laws 'unnecessarily complicated'. David Anderson calls them 'undemocratic'. The message cannot be clearer: wholesale reform of Britain's surveillance laws is needed."
The 379-page report, which makes various other recommendations, is expected to be used to inform future legislation.