Ministers will respond in parliament on Monday to claims that UK intelligence agencies have gained access to a vast reservoir of private data relating to people living in Britain – including emails and phone logs – collected by spies at the US National Security Agency.

Amid growing calls for the government to reveal what it knows about the data interception scandal revealed by the Guardian last week, senior Whitehall sources said it was "highly likely" that a statement would be delivered to MPs in the Commons either by the foreign secretary, William Hague, whose department is responsible for GCHQ, or the home secretary, Theresa May.

May is scheduled to answer Home Office questions at which the issue will dominate, if no formal statement is to be made. Among questions that MPs want answered as a matter of urgency is whether any of the intelligence supplied by the US about British citizens was handed over to GCHQ in breach of British interception legislation, or whether British intelligence officials attempted to bypass British law by sending requests for intercepts to the US.

The calls follow several days of leaks of top-secret documents from within the NSA to the Guardian, painting in shocking detail for the first time the vast reach of the NSA's electronic spying operations conducted as part of anti-terrorism investigations and foreign intelligence gathering. Included have been revelations of how the NSA has collected huge amounts of so-called "metadata" from telephone companies such as Verizon. This is data that gives details of calls made, numbers dialled and their duration, rather than the content of the conversations.

A second leak to the Guardian was contained in a PowerPoint presentation detailing the secret NSA programme called "Prism", which appears to make use of vast swaths of email and instant chat messaging data supplied by technology companies including Google, Facebook, Microsoft and Apple.

Although the Obama administration has admitted that the NSA mines telephony and other communications data – including from US citizens – senior executives at technology companies have denied in the most emphatic terms that the programme amounts to a direct pipeline from their servers to the NSA. US spies have been accused of tapping into servers of nine US internet giants including Apple, Facebook, Microsoft and Google in a massive and wide-ranging anti-terror sweep. All deny giving government agents access to servers.

On Saturday Google said that the US government had no access – "not directly, or via a back door, or a so-called drop box". Instead the companies – which say they have not heard of Prism – insist that the only material they hand over is that demanded by a court order, under the US foreign intelligence surveillance act (FISA).

Mark Zuckerberg, the Facebook founder and CEO, said: "Facebook is not and has never been part of any programme to give the US or any other government direct access to our servers." Google and Facebook insisted that they only turned over data when they received court orders, and frequently challenged them. But the New York Timesreported on Saturday that key tech firms, including Google, Facebook and Microsoft, had co-operated with the NSA "at least a bit". The story describes a close working relationship between top intelligence officials and the companies. The chairman of the US joint chiefs of staff, General Martin E Dempsey, met "personally with executives including those at Facebook, Microsoft, Google and Intel," the paper reported. Twitter is said to have declined to participate.

In addition to the denials of the companies, the office of the US director of national intelligence on Saturday night moved to declassify aspects of the controversial Prism programme. A fact sheet released by the office of James Clapper, the director of US national intelligence, insisted that no "unilateral" unregulated capability exists that provides a direct feed of internet data for US intelligence analysis. It insisted that all information demanded was subject to specific court requests under FISA, overseen by Congress. "Under section 702 of FISA, the US government does not unilaterally obtain information from the servers of US electronic communication service providers. All such information is obtained with FISA court approval and with the knowledge of the provider based upon a written directive from the attorney general and the director of national intelligence."

Since the revelation of Prism and the court order demanding that Verizon hand over its telephony metadata, the Obama administration has tried to justify the need for the NSA's interception operations, insisting that all collection is approved by secret FISA courts and overseen by Congress.

But in Britain MPs of all parties have demanded clarity. Douglas Alexander, Labour's shadow foreign secretary, said: "Our intelligence agencies do vital work to keep our country safe from harm, but it is also vital that they operate within a framework of legality and accountability.

"These reports have raised serious public concern, so I am now calling on the foreign secretary, William Hague, to come to the Commons on Monday to make an urgent statement to MPs.

"In that statement he must explain the government's position and tell MPs how the government will work with the intelligence and security committee to address these public concerns."

GCHQ has agreed to send a full report about the controversy to the all-party intelligence and security committee chaired by a former Conservative foreign secretary, Sir Malcolm Rifkind. By coincidence Rifkind and members of his committee are flying to Washington on Monday to meet members of the intelligence committees of the Senate and the House of Representatives as well as other US officials involved in the intelligence field. The committee will be sent the GCHQ report via the British embassy in Washington.

The committee will be keen to know if British intelligence agencies used requests for information under Prism to circumvent the Regulation of Investigatory Powers Act (2000), which requires ministerial authority for intercepting data content such as emails, or a senior intelligence officer's agreement to reveal metadata.

In the past there have been suggestions that the NSA and GCHQ have had reciprocal arrangements which see them issue requests for intercepts.

In 2003 Katharine Gun, a translator at GCHQ, leaked a request from an NSA officer, Frank Koza – first revealed by the Observer – that requested GCHQ's assistance in the illegal and secret bugging of UN delegations in the runup to the Iraq war.

That leak suggested that both agencies had in the past exchanged requests for assistance in interception operations against targets.

A potential cut-out allowing deniability does, however, exist under the so called "control principle". That regulates the arrangement under which the UK and US intelligence agencies share material, which means that the organisation supplying intelligence, even following a direct request, "owns" the intelligence and does not have to disclose how the information was acquired.

Former shadow home secretary David Davis said the US monitoring programme appeared to allow the state to "spy on who they like". He told Sky News that if reports of its operations were accurate then "it is actually quite a scandal".

David suggested that May and Hague must have had knowledge if the system was used by GCHQ to provide information on UK citizens. "Presumably they at least would have had to sign an authorisation for this to take place," he said.

"[There were] nearly 200 British citizens under surveillance of one sort or other in one year so they must have known something about it."

The ICIJ's exploration of offshore secrets began when a computer hard drive packed with corporate data arrived in the post. Gerard Ryle, ICIJ's director, obtained the small black box as a result of his three-year investigation of Australia's Firepower scandal, a case involving offshore havens and corporate fraud.

The hard drive contained more than 260 gigabytes, the equivalent of half a million books. Its files included 2m emails, four large databases. There were details of more than 122,000 offshore companies or trusts, and nearly 12,000 intermediaries (agents or "introducers").

Unlike the smaller cache of US cables and war logs passed in 2010 to WikiLeaks, the offshore data was not structured or clean, but an unsorted collation of internal memos and instructions, official documents, emails, large and small databases and spreadsheets, scanned passports and accounting ledgers.

Analysing the immense quantity of information required "free text retrieval" software, which can work with huge volumes of unsorted data. Such high-end systems have been sold for more than a decade to intelligence agencies, law firms and commercial corporations. Journalism is just catching up.

The named people who administered offshore companies included shareholders, directors, secretaries, lawyers, accountants, nominees and trustees. But many of such structures were simply legal devices designed to conceal. The real beneficial owners proved often to be the so-called "settlors" or "protectors" of offshore trusts, and those holding legal powers of attorney which enable them to exert secret control over the bank accounts.

China, Hong Kong, Taiwan, the Russian Federation and former Soviet republics appeared to provide the majority of secret offshore owners. The British Virgin Islands are the second-largest source of capital investment in China – on paper at least. Cyprus, an offshore island currently in financial crisis as a result, is also identified in the data as a huge source of Russian investment.

ICIJ's collaborating journalists from 46 countries constituted one of the largest groups ever to have worked together on a data project.

Interestingly, the team's attempts to use encrypted email systems such as PGP ("Pretty Good Privacy") were abandoned because of complexity and unreliability that slowed them down.

Meanwhile, computer programmers in Germany, the UK and Costa Rica also designed sophisticated data mining and cleaning software for ICIJ. Manual analysis in New Zealand proved crucial in early decisions on what countries ICIJ needed reporters.

ICIJ's own search system – named Interdata – was developed by a British programmer as dozens of new journalists joined the expanding project. Interdata allowed them to download copies of those of the 2.5m offshore documents relevant to their countries.

ICIJ rebuilt some of the databases in an effort to run them in their original format. There were surprises. The databases were formatted to record who really lay behind each entity, as required by international regulations on money laundering and "due diligence". Journalists hoped the truth was just a click away.

In fact, entries for "beneficial owners" were often empty. The offshore agencies had frequently passed off their supposed legal responsibility to intermediaries in other countries. The lesson was that the empty fields were not an accident; it was the design.

Only occasionally would an alert screen pop up, giving contact details for the persons who really owned the assets. ICIJ's fundamental lesson therefore had to be patience and perseverance.

But persistently following leads through incomplete data yielded some great rewards: not just occasional and unexpected top names, but also the inside details of many nuanced and complex schemes for hiding wealth.

• Duncan Campbell was the data journalism manager for the ICIJ project.

Britain is to "sweep away" tax secrecy by introducing a new central register that will ensure all the true owners of "shadowy" shell companies have to be declared to the tax authorities, David Cameron has announced.

In one of the biggest steps by Britain to crack down on "aggressive" tax avoidance and money-laundering, the prime minister pledged in a Guardian interview to end the era of "secretive companies in secretive locations" that cost exchequers around the world billions of pounds in lost revenue.

The prime minister will ask G8 leaders, including Barack Obama and Angela Merkel, to sign up to a new set of core principles on tax on the second and final day of the summit, which opens in Northern Ireland on Monday.

The summit will also be dominated by Syria after the White House announced that the regime of Bashar al-Assad had crossed a "red line" by using chemical weapons against rebel forces. Cameron told the Guardian that Britain shared the "candid assessment" of the US that the Assad regime had used chemical weapons against the rebels.

He said Britain had proof that Damascus had used the nerve agent sarin on at least two occasions. "It does constitute a war crime," he said. "As the statement from America has put it, it crosses a line as defined by all sorts of international agreements over many decades." He said Britain now had to do more to help the rebels, but said no decision to arm them had been made.

His remarks came as the US offered to provide arms to the rebels in the country, a dramatic change of heart that brings Obama into line with the position forged by Cameron and the French leadership in recent weeks. Discussions about how, when and where to give what to whom are certain to encroach on the G8 talks. Merkel said the situation was serious enough to require UN security council deliberation. The Syrian government rejects the accusation that it has used chemical weapons.

The prime minister, who will also hold talks about Syria in Downing Street on Sunday with the Russian president, Vladimir Putin, before travelling to Northern Ireland, believes that significant steps can be taken on tax avoidance at the G8 after the EU agreed new rules on exchange of information.

Cameron said he wanted to set an example to fellow G8 leaders by cracking down on British accountants, lawyers and business figures who use shell companies – often located in offshore tax havens – to hide the identity of ultimate beneficiaries.

Echoing Graham Greene's description of Monaco as a "sunny place for shady people", the prime minister told the Guardian: "We need to know more about who owns which company – beneficial ownership – because that is how a lot of people and a lot of companies avoid tax, using secretive companies in secretive locations.

"The way to sweep away the secrecy and get to the bottom of tax avoidance and tax evasion and cracking down on corruption is to have a register of beneficial ownerships so the tax authorities can see who owns beneficially every company."

Under the changes, companies registered in Britain would come under a legal obligation to obtain and hold adequate, accurate and current information on the ultimate owner who benefits from the company – and be required to place the information on a central register that would be maintained by Companies House. The central registry would be introduced into law by incorporating the EU's fourth money-laundering directive.

The central register will initially only be made available in Britain to authorities such as Revenue & Customs. But the government is to hold a consultation on whether it should be made public, a step the prime minister said he would only take if it would not harm the interests of British businesses.

"I am sure that is where I would like to end up, but I do not want to disadvantage Britain by doing something others won't do," he said. "I don't also want to give up our leverage on others by trying to make them move at the same time."

Brendan Cox, a spokesman for the If campaign, said: "Public registries would make a major difference in the fight against tax evasion. They would mean all countries had access to the data and the 'many eyes' principle would mean more effective scrutiny."

On Friday night, an interactive database allowing users to search through more than 100,000 secret companies, trusts and funds created in offshore locations including the British Virgin Islands was launched online by the Washington-based International Consortium of Investigative Journalists (ICIJ). It is hoped that the release of the data – part of a cache of 2.5m leaked files which the non-profit ICIJ has analysed with the Guardian and other global news groups – will lead to further tip-offs and investigations into the role of offshore tax havens.

Cameron is to ask fellow G8 leaders to follow his example by signing up to core principles similar to the proposed binding rules in Britain. G8 countries will be asked to agree that companies should "know who owns and controls them" and that "basic information should be adequate, accurate and current". Their records should be accessible to law enforcement and tax authorities, Cameron said.

He added: "When you look at how individuals and companies sometimes evade tax or aggressively avoid it – that is a problems one and two. Problem three, in many ways even more serious, is corrupt payments from extractive companies or dictators or middlemen in the developing world. They use shadowy nominee companies where you can't see who is the beneficial owner – that is why who benefits is so important."

Britain expects the G8 to agree to call on the Organisation for Economic Cooperation and Development to produce a new agreement modelled on the US-led Financial Action Task Force on tax transparency, which is supported by 75 countries. The OECD proposal, supported by 28 states including Britain's main crown dependencies, would be multilateral and lead to "automatic exchange", in which countries pass on tax details without a request having to be made.

The prime minister is confident that the leaders of Britain's crown dependencies and territories will take steps at a meeting in Downing Street towards agreeing the automatic exchange of tax information. He said: "They are now agreeing to tax exchange of information, not just on request but what matters is automatic exchange of information. You don't know what you have not go as it were, to make one of the Rumsfeldian type remarks … When you have got automatic exchange you can see what you are missing."

But there were signs of resistance from some of the crown dependencies after the chief minister of Bermuda called for clarification before agreeing to sign.

Hubert Hughes, the chief minister of Anguilla, said the Caribbean island did not have the resources to uphold an OECD agreement that would involve liaising with 80 countries.

The chief minister told the Guardian: "We don't have the financial resources, we don't have the economy to deal with that. Our British governor has done his best to create a failed state in Anguilla – our British governor has done everything to drive good investment from Anguilla."

Hughes added that he resented being lectured by Britain on tax transparency in light of the City of London's role. "It is a matter of hypocrisy because when I look at the City of London – they should do something about that first and not try to shelter behind these little territories that are just trying to survive.

"We are not tax havens as such, we are offshore centres. The City of London is one of the biggest tax havens in the world – it is the biggest money laundering centre in the world."

But the chief minister said he would sign the OECD framework if Bermuda, whose population is nearly five times that of Anguilla, accepts the measure. "We tend to walk behind them," he said.

In his Guardian interview, Cameron praised the work of the Oxford economic professor Paul Collier, who wrote in Prospect magazine recently that lawyers and accountants who create shell companies should "hang their heads in shame" for creating artificial structures that facilitate "evil". The prime minister said: "I feel very passionately about it. The more I have looked into it the more I can see that unless you take action on the tax and transparency agendas you will never get the sort of development and change we want to see. Does it raise moral issues? Yes it does. Tax evasion is illegal so it is not immoral, it is criminal. But aggressive tax avoidance raises moral issues."

The prime minister also hopes to use the G8 summit to launch negotiations on a new EU-US trade pact, though this has run into difficulties as Paris seeks to exclude its film industry from the agreement. Cameron said he is willing to be patient. "I am a great admirer of French films. But I don't believe that you save industries by protecting them. Would the British film industry be stronger if we didn't have this amazing exchange between the British film industry and the strength and power of Hollywood? But I totally understand the French feel very passionately about this. What I just hope is we want to get the trade talks off to a good start. Both sides want [the negotiations] to get off to the right start. But we shouldn't go for artificial deadlines."