The directive will require Internet service providers and phone companies to keep data on every electronic message sent and phone call made for between six months and two years.
"Agreement on retaining communications data places a vital tool against terrorism and serious crime in the hands of law enforcement agencies across Europe," British Home Secretary Charles Clarke said in a statement. "Modern criminality crosses borders and seeks to exploit digital technology."
However, the data retention has been criticized as a threat to the personal privacy of European citizens.
The parliament voted 378 to 197, with 30 abstentions, for a package already agreed to by the assembly's two biggest groups and member states, with European Commission backing.
Telecommunications providers will now have to keep data such as the time of each fixed and cell phone call made in Europe; whether a call is answered or not; the duration of the call; and other details that can help trace the caller. On the Internet side, they will be required to retain information on the times people connect to the Internet, people's IP addresses, and details pertaining to e-mail messages and VoIP calls. The content of the communications will not be recorded.
The legislation is being championed by the U.K. and other governments. They say it will help trace terrorists through communications records. The change in the law was proposed during the U.K.'s presidency of the European Union in the wake of the July 7 bombings in London.
Telecommunications companies and ISPs expressed concerns about the financial impact of the parliament's decision, as the new law will drastically increase companies' storage costs, but makes no move to compensate providers.
"This directive will impose a significant burden on the European e-communications industry, impacting on its competitiveness," Europe's telecoms and Internet industries said in a joint statement.
According to the British government's Home Office, EU member states could begin implementing the directive as early as next year.
A separate directive from the European Commission to keep data for six to 12 months was passed earlier this year, despite fierce criticism from both the public and private sector.
Jo Best of Silicon.com reported from London. Reuters contributed to this report.